Process stuck waiting

[gothickitty93]

So, running the application in small batches works, but when I try to do a large directory, the process waits. Pressing CTRL + T shows this.

load: 3.16 cmd: afsctool 12486 waiting 52.97u 10.25s

Running dtruss reveals

psynch_cvwait(0x600000E1D2A0, 0x9C0100009D00, 0x0) = -1 Err#316 gettimeofday(0x7FF7B6CBA018, 0x0, 0x0) = 0 0

Now, I'm no programmer, so I don't really understand what is going on. Please advise if there is a better way I can create more useful output for diagnosis.

[RJVB]

For now I cannot make anything of the output you show, so I can only ask a few questions.

What hardware is this on, Intel or Apple silicon, and what OS version? How large is that large directory and at what percentage of progress does the hang occur? What's the exact invocation? Does it make a difference if you add -f (detect hard links)?

A typical reason for this kind of behaviour would be if the application tried to compress a named pipe or some other kind of special file. A priori that shouldn't happen but one never knows. You could add -v options until you get a meaningful output. The good news is that afsctool will skip files that are already compressed, so if this is indeed a problem with a particular file the hang should happen (almost) immediately if you relaunch afsctool on the same directory. Another reason could be that your drive has a bad sector or some other I/O error. In that case you should be seeing meaningful output in the system log (since you're already in a terminal the quickest way to start monitoring that file would be open /var/log/system.log ... if SIP still allows that kind of thing).

[gothickitty93]

OK,

My system is a 2018 Intel Mac mini, i7, 32GB. I am on macOS Monterey 12.0.1.

The directory sizes that I am trying to compress is essentially the whole system. I've tried to break it up into subfolders like /Library, /System, /Applications, /Users, etc, but usually run into the issue of hanging. Similar strategy and result for my external storage too.

The incantation I’m using is sudo afsctool -cfvvvvvvvv -j10 -T ZLIB -9

Syslog doesn’t show anything related to I/O or afsctool.

[RJVB]

I see only 2 ways to move forward with this: point afsctool to a directory where you are certain to get the issue, and 1) use -Jn instead of -jn for a less strict locking for I/O 2) launch it in serial mode

If my locking (thread synchronisation) approach is the problem then 1) might already solve your issue. If a file is the culprit, you should be able to identify it with 2).

BTW, "-T ZLIB" is the default, and I doubt you'll get more useful output beyond 3 -v options (-vvv).

[gothickitty93]

launch it in serial mode

How would I go about doing this?

[RJVB]

How would I go about doing this?

Omit the -j/-J option; it will then process only a single file at a time.

[gothickitty93]

If a file is the culprit, you should be able to identify it with 2).

Looks like the tool is having issues with files located in /Library, /System/Volumes/Data/Library, and ~/Library,

[RJVB]

Looks like the tool is having issues with files located in /Library, /System/Volumes/Data/Library, and ~/Library,

Can you identify them?

NB: it is not a good idea to compress locations that have open files in them. I'm not aware of any way the Mac OS allows to determine if a file is open in another process so I'm not checking for this.

Note to @RJVB: I think the documentation must or if not, should mention this somewhere.

[gingerbeardman]

I believe these folders are now (since Catalina or Big Sur?) protected by macOS, which could be the problem.

[RJVB]

I believe these folders are now protected in macOS, which could be the problem.

That could definitely be the problem. I knew that SIP Sucks (TM) ;) (~/Library can become huge; periodic compression of the caches in there alone would be beneficial).

[gingerbeardman]

Hmm, I'm not able to find relevant info right now.

https://support.apple.com/en-gb/HT204899

But, I'm also unable to compress ~/Library in one go.

[gothickitty93]

I was thinking something like that too. I tried mounting the system volume as read/write, and it still had the same problem (in addition to breaking my macOS install). I suppose maybe afsctool should know what OS version it is on and skip known protected paths. The average user shouldn't be as adventurous with the boot volume as I am.

[RJVB]

my macOS install). I suppose maybe afsctool should know what OS version it is on and skip known protected paths. The average user shouldn't be as adventurous with the boot volume as I am.

Or rather use some API call to determine if it has write permissions in the target directory. You'd expect though that this is already being done by trying to open the file in read/write mode. I'll have to have another look at exactly how the I/O is done; it's still the old code I didn't write myself.