search

hashicorp / packer

Packer is a tool for creating identical machine images for multiple platforms from a single source configuration.
http://www.packer.io
Other
15.09k stars 3.33k forks source link

Don’t use upx to reduce binary size #8971

Open symgryph opened 4 years ago

symgryph commented 4 years ago

For anyone considering using upx To reduce binary size, It seems that packer does not like being compressed by UPX. It does all kinds of strange things and doesn’t work very well. This is mostly an FYI so other people don’t waste their time

SwampDragons commented 4 years ago

Hi -- do you think you can share a little more information? What specific command are you using to compress the binary, and what weird behavior are you seeing as a result?

symgryph commented 4 years ago

sure, I used the command upx —lzma packer The behavior is extremely slow start times, and never processing any input. The system just hangs there after compressing the binary and running it. If you run the unmodified binary it works fine. to reproduce:

upx —lzma packer

I didn’t try lzma —ultra-brute packer

I was using the most current version of upx, 3.96

On Mar 31, 2020, at 12:00 PM, Megan Marsh notifications@github.com wrote:

Hi -- do you think you can share a little more information? What specific command are you using to compress the binary, and what weird behavior are you seeing as a result?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/hashicorp/packer/issues/8971#issuecomment-606719581, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABCSE3X2JTUWG6XNE32I3HDRKIHSRANCNFSM4LW3GGLA.

arizvisa commented 4 years ago

As slow start times are expected due to the need for the elf-segments/pecoff-sections to be decompressed... What platform are you using, or rather what executable format are you decompressing from exactly?

Packer plugins are started or executed by exec'ing packer plugin /path/to/plugin/, so they're not getting the benefit of COW (copy-on-write) due to the address-space of the process being replaced during the exec(). This means that the segments will end up being re-decompressed every single time a plugin does its thing. it's likely that this is amplifying the decompression during runtime.

If this does turn out to be the case, a potential solution (or hacky solution, most-likely) would be to do a "soft" exec, where you fork(), set argv to the required plugin parameters, and then branch to main somehow.. I'm not sure how to do this with golang. Plus, i'm not sure whether plugins expect the packer host process to have a completely resetted state at startup.

You don't get fork() semantics on windows with pecoff.. Hence the question about which platform/format.