search

criblio / appscope

Gain observability into any Linux command or application with no code modification
https://appscope.dev
Apache License 2.0
267 stars 33 forks source link

Binary file size too large #531

Closed coccyx closed 3 years ago

coccyx commented 3 years ago

In issue #449 we stopped upx compression of the binaries, which has resulted in a bloating of the current binary to 53MB, which is far too large.

-rwxr-xr-x 1 clint clint 53M Sep  6 13:27 bin/linux/x86_64/scope

We need to find a solution to binary size, since we now put multiple copies of the binary and the library into the container which has ballooned to a whopping 218MB:

cribl/scope   dev-x86_64                 b653c6394caf   3 minutes ago   218MB
ledbit commented 3 years ago

Root cause of this is likely related to upx/upx#298 - @pdugas can you please confirm?

pdugas commented 3 years ago

I was able to build UPX from source on an ARM machine and use that to compress the scope binary from 33M down to 20M. The resulting binary appears to work with some cursory manual testing. Running the full suite of integration tests now.

If this works, my plan is to add UPX as a new contrib and use that to restore the earlier behavior of the build system to compress scope on both x86 and ARM.

pdugas commented 3 years ago

Unit and integration tests all pass on ARM using the compressed scope binary on ARM and the pre-4.0.0 unreleased UPX built from source. Will get a PR out shortly.

pdugas commented 3 years ago

Before we pull the trigger on merging #547 to address this, I think we need to get some consensus on whether we want to use the unreleased version of UPX that's been added as a contrib on that PR.

I see four options here:

  1. Do Nothing - Abandon the PR and wait for a newer release of UPX that supports both x86 and ARM. The scope binary stays large like it is now.
  2. UPX-3.96 x86 Only - Use the current released version of UPX to compress the x86 version of scope like before but leave the ARM version uncompressed.
  3. UPX-3.96 x86, UPX-4.x ARM - Use the current released version of UPX to compress the x86 version of scope like before and the 4.x development (unreleased) version of UPX to compress the ARM version.
  4. UPX-4.x x86 and ARM - Use 4.x development (unreleased) version of UPX to compress both versions.

The PR is currently setup for option 4. Anybody have a strong opinion here?

ghost commented 3 years ago

Moving this under 0.7.6 milestone since the PR #547 didn't close by 0.7.4 milestone timing

pdugas commented 3 years ago

Direction from @ledbit in stand-up today was to add back compression of the x86 version of scope using the released version of UPX and leave the ARM version uncompressed. When UPX releases a version for that supports ARM, we can return to compressing the ARM version.

pdugas commented 3 years ago

Given the direction above, I'm abandoning the work on the feature/531-upx branch where we added UPX as a contrib. I will close that PR then open a new one for a new branch that only makes the Makefile changes needed for the simpler approach.

seanvaleo commented 3 years ago

QA INSTRUCTIONS

Do: On an ubuntu machine: git clone git@github.com:criblio/appscope cd appscope make build cd bin/linux/x86_64 ls -hl scope

Expect: scope binary is less than 30MB in size