Additionally, we have Scott Schuchart, who writes:
“I’m trying to compress three largest files, about 20 to 60 gigabytes each, using the Finder, and they’re taking forever. According to Activity Monitor, Archive Service, which I assume is a compressor, is running six threads but only using between 95 and 140 percent CPU. This is in a MacBook Pro M2 Pro with 10 cores. Nothing else significant is happening, and it’s plugged in with no fan noise. Why can’t it spin up at least three performance cores to get this done faster? What are these chips for?”
I think it’s an interesting question, because a lot of the times, in people’s regular computing lives, especially if they’re using phones, there’s this question that I always ask to make people think about: is there anything you do with your computer that makes you wait?
A lot of people will say, honestly, no. Like, unless I’m waiting for an animation to complete, I don’t do anything with my computer where I’m like, come on, come on, this is taking too long. The magic of modern technology is a lot of the times, people will say, waiting, they used to say, waiting for a web page to load, or like a cellular is bad, waiting for something to load, like lots of network-related stuff, but like, if you really want to get down to it, is there anything you’re waiting for the CPU for?
Gamers would say, yeah, I’m waiting for it to draw the next frame faster, because I’m getting 20 frames per second on this thing, and I really wish I could get 60 or whatever. Like, there are answers to this question. It’s not like we have infinite performance, and this is another interesting one. This is a thing I think a lot of people who use just, you know, Macs and PCs find themselves doing, which is like, I asked my computer to do a thing, and because of the size of the data involved, like trying to, you know, do something with a 20 or 60 gig file, I’m waiting for it to be done. I’m like, why, you know, I see a progress bar, I have to wait several minutes.
Sometimes it’s, you know, network-related, but sometimes even just on a local thing, you’re like, is this, what’s the bottleneck here? Is it, as we would say in the business, IO bound? Am I waiting for things to be read off of disk or written to disk? Is it CPU bound?
Scott is asking, my computer seems like it’s doing nothing. It’s chill, like the fans aren’t running. It’s not using all of its resources, and yet here I am looking at a progress bar. Here I am waiting, I’m waiting for my computer, and I don’t understand why, because it seems like it’s got more resources that could be putting towards this.
If you are a computer science major, or ever studied this at all, you’ll inevitably run into Amdahl’s Law, which is, we’ll put a link to the Wikipedia page. It’s often used in the context of parallel computing, which they used to call it back in the day when you had more than one processor, although now everything does, to predict the theoretical speed-up.
I’m reading from the Wikipedia page here: “When using multiple processors, for example, if a program needs 20 hours to complete using a single thread, and a one-hour portion of the program cannot be parallelized, then only the remaining 19 hours of execution can be parallelized. Therefore, regardless of how many threads are devoted to a parallelized execution of this program, the minimum execution time is always more than one hour.”
So it’s basically saying, there are some things that you can do in parallel and some things that can’t. And even if you make the things that can be done in parallel happen instantaneously, which you’re not going to, but if you can make them done instantaneously, because you deploy seven million processors and there’s no overhead to distributing the work to them, which also doesn’t happen, you’re never gonna get it faster than the time it takes for those serial portions. Sometimes running a particular compression algorithm has parts of it that are parallelizable and parts of it that are not. And you’re never going to make it run faster than the parts that are not. They are the long pole. They are the thing that you can’t shrink. No matter how many more threads you throw at it, there are certain portions of it that you can’t work on at the same time.
Now, this particular task, compressing this, is this compression algorithm parallelizable? If so, is the thing that is running it in parallel not devoting enough threads to it? This can happen, especially in macOS. There’s lots of things that I’ve complained about this before where it’s like, there’s a task and there’s a portion of it that is parallelizable, but it is intentionally run by the program or the OS in a way that it uses fewer resources. It’s run at low priority. It’s run at a low number of threads so that it doesn’t make your computer feel unresponsive. But sometimes this is the only thing you want your computer to do and you want it to do it right now.
I talk about like, you know, the photos, please recognize faces now. Please analyze my photos now. It’s literally the only thing I want you to do. I’m gonna walk away, use all my resources to do it and macOS and Apple’s apps in particular are terrible at that. I’m going to guess in this case, the main problem is the right-click finder thing that compresses files, has a cap on how many threads it will ever use and therefore is never gonna go faster than that.
Now, it may also be the case that this algorithm is not parallelizable sufficiently, so more threads wouldn’t help. I don’t know specifically, but these are the potential scenarios. One of them is just like theoretical, like if you pick an algorithm that you can’t break into like 75 chunks, you can only break it into like four chunks, right? There’s not parallelizable past that because there’s data interdependencies between the tasks and they need to communicate and synchronize with each other and there’s serial portions, then you’re stuck.
But specifically in Apple platforms, Apple is good/bad about making choices for you that ensure that the system remains responsive under all circumstances by limiting the amount of parallel work that can be done. And you would like more options, like for example, past sponsor Backblaze that I use on my computer to back it up. It actually has a pop-up menu in the little like settings that says, “Hey, when Backblaze is running and backing up your stuff, how many threads do you want it to use?” And the default is pretty low because you don’t like, as we say in the ad reads, you don’t even know Backblaze is running half the time. The default is like, just do your work, it’ll back up your stuff behind the scenes or whatever, it’s using a small number of threads to not disturb you, right?
The way I personally run Backblaze on one of my computers is because I have a million backups running all day, every day anyway. I make Backblaze run only at 3 AM and I give it like 99, like whatever the maximum number of threads is. Obviously, there’s a point of diminishing returns, you shouldn’t give it way more threads than you have CPU cores, but I have a lot of CPU cores. I give it a huge number of cores. So it wakes up at 3 AM, runs my Backblaze thing, and it runs it using every resource on the system. So much so that if I was to come downstairs at 3 AM and try to use my computer, I would notice that Backblaze is running because I’ve intentionally said, use it all, peg everything, go as fast as you can and believe me, it makes a big difference because I have a fast internet connection, but just the process of like finding all the change files, reading all those change files and sending them up to fill my one gigabit upwards pipe, I need to be reading and sending from as many files at once as I want. Backblaze gives you that option.
The right-click menu in the finder for compressing files does not give you that option. So the answer, I think in this case, without knowing anything about the compression algorithm or how parallelizable zip is, is that Apple is doing this to try to help you. And in this case, you do not want this help from Apple.
