Add support for extracting Apple Archives (.aar files)

[zorgiepoo]

This is an efficient Apple archive format available since versions of macOS 10.15. From my testing it is much faster / compresses better than zip, tar.*, dmg (tried with lzma and lzfse compression).

In macOS 10.15, the aa utility used to be called yaa. (While older OS versions have yaa, they are not compatible with newly created archives).

Check man aa and https://developer.apple.com/documentation/applearchive for more details.

Like the other archive files we deal with, we are using the piped unarchiver. I didn't want to adopt Swift APIs in this framework and deal with those implications. (edit: seems like there is a C API too. I may explore that another day).

Misc Checklist

• [x] My change requires a documentation update on Sparkle's website repository
• [x] My change requires changes to generate_appcast, generate_keys, or sign_update

Testing

I tested and verified my change by using one or multiple of these methods:

• [ ] Sparkle Test App
• [x] Unit Tests
• [x] My own app
• [ ] Other (please specify)

Need to test:

• [x] Added unit tests here for aar format.
• [x] Extracting aar files on macOS 10.15.7
• [x] Updating/installing an app that uses aar
• [x] Failing to extract aar archive on macOS 10.14
• [x] Test extraction support in generate_appcast

macOS version tested: 14.5 (23F79) 12.0 VM (works) 10.15.7 VM (works) 10.14.6 VM (fails)

[zorgiepoo]

@aonez @kornelski for potential curiosity.

[kornelski]

Seems reasonable, given that it's just shelling out to an Apple tool. If Apple uses it themselves for packages they hopefully pay attention to security of this decompressor.

From what I could find it uses https://en.m.wikipedia.org/wiki/LZFSE so it's not the best compressing algorithm, but should be fast.

[zorgiepoo]

It also supports LZMA and a couple more algorithms. From what I see it has a more favorable (or as favorable) decompression speed/compression size than other LZMA or LZFSE-competitive based archives. I think that is partly due to the multithreaded nature.

Here's some random decompression timings on my M3 MBA running macOS 14.5 (compressing Arc.app, a ~776 MB app) using default parameters:

• arc-lzfse.aar (381 MB, 441 milliseconds)
• arc-lzma.aar (302 MB, 1.53 seconds)
• arc.zip (387 MB, 1.73 seconds, using ditto)
• arc.zip (387 MB, 1.34 seconds using bsdtar)
• arc.tgz (386 MB, 1.37 seconds, using tar)
• arc.txz (292 MB, 8.25 seconds, using tar)
• arc.tar.zst (374 MB, 995 milliseconds, using tar)
• arc-lzma-apfs.dmg (314 MB, 415 milliseconds attach, 5.04 seconds extract, 285 milliseconds eject, using diskutil)
• arc-lzfse-apfs.dmg (382.7 MB, 366 milliseconds attach, 1.14 seconds extract, 279 milliseconds eject, using diskutil)

[aonez]

Nice one @zorgiepoo. Just thinking out loud, since AAR supports encryption maybe it can be implemented as well in a future upgrade, although I don't know if it will make much sense since the XML will need to provide the password and will be as public as the compressed file. Also ZSTD is a clear winner lately but for macOS only AAR is usually faster.

[zorgiepoo]

I'm not a fan of supporting a decryption password, although we support it for DMGs because a couple of people wanted it some years ago (developers need to provide a password themselves via a delegate callback). So I'll hold off unless people want it (it also has steeper OS requirements).

I may also later look into BinaryDelta performance as decompression is pretty single threaded and I think primarily bottlenecked there (followed after by bspatch). That might mean using multiple compression streams and splitting the payload into chunks.

[zorgiepoo]

As far as any corner cases for Security concerns go, I'm going to require aar files to be validated before extracted (#2588). And I plan/want to move towards doing this for all other archive formats (#2590).