GrapheneOS-Archive / legacy_bugtracker

See the new issue tracker for GrapheneOS at https://github.com/GrapheneOS/os_issue_tracker.
112 stars 11 forks source link

Release-Script: Slow, single-threaded xz compression of factory images #850

Closed kuleszdl closed 6 years ago

kuleszdl commented 6 years ago

The last line of the release script does only use one cpu thread for compressing the factory image:

xz -v --lzma2=dict=512MiB,lc=3,lp=0,pb=2,mode=normal,nice=64,mf=bt4,depth=0 $DEVICE-factory-$BUILD_NUMBER.tar

In theory, xz supports multiple threads using the -T parameter. I tried setting it to -T0 but then xz failed as it was out of memory (I tried in a VM with 30G of RAM and 15 cpu cores). Is there a way how this could be improved?

thestinger commented 6 years ago

It's intentionally using a huge dictionary and spending a lot of time compressing because it only happens for release builds, not development builds. I have more than enough memory to use some threads but that would hurt the compression ratio.

You can just comment it out. In fact, you don't need to generate factory images at all after your initial build so you can comment out everything but generating over-the-air updates.

thestinger commented 6 years ago

And technically, for the first build, you didn't need to generate an over-the-air update.

Rudd-O commented 6 years ago

Glad to take patches here https://github.com/Rudd-O/copperheados-build/ to skip certain steps via build variables.