search

praetorian-inc / noseyparker

Nosey Parker is a command-line program that finds secrets and sensitive information in textual data and Git history.
Apache License 2.0
1.66k stars 79 forks source link

Release static binaries #177

Open SkyperTHC opened 5 months ago

SkyperTHC commented 5 months ago

The x86_64 release binary rarely works on Linux because of libc mismatches. It would be desirable to have a statically compiled binary as well.

I'm currently using these steps to compile a static binary using your Dockerfile.alpine:

# Inside Dockerfile.alpine:
cd /src
RUSTFLAGS='-C target-feature=+crt-static' cargo build --locked --profile release --target x86_64-unknown-linux-musl --features release
strip target/x86_64-unknown-linux-musl/release/noseyparker-cli

The statically compiled binary is then created in target/x86_64-unknown-linux-musl/release/noseyparker-cli

Happy to fiddle it into hour GitHub CI/CD but feels like it would be easier for you - gimme a shout.

bradlarsen commented 5 months ago

@SkyperTHC thank you for opening this. Yes, I'd like to solve this here, as the glibc mismatch has continued to be a pain point from time to time (e.g., #103), making it difficult to run on older Linux systems.

I'll take a look at this.

bradlarsen commented 5 months ago

*The desired end state: Nosey Parker releases will include an additional prebuilt releases for `-unknown-linux-musl` that is statically linked.**

The v0.17.0 release has the following prebuilt artifacts:

We want to include an additional file like noseyparker-v0.17.0-x86_64-unknown-linux-musl.tar.gz in future releases, which is fully statically linked.

Notes

Okay, so I tried a few things. Simply enabling the crt-static target feature for the x86_64-unknown-linux-gnu target fails early on:

error: cannot produce proc-macro for `clap_derive v4.5.4` as the target `x86_64-unknown-linux-gnu` does not support these crate types

A few minutes of research indicates that this may be a limitation of the *-linux-gnu targets...

Instead, using the x86_64-unknown-linux-musl target seems like it works, but setting up an Ubuntu-based build environment with all the necessary tools appears to be more complicated than simply doing an apt install of a few packages. (I get errors about musl-g++ missing...)

@SkyperTHC does this match your experience, or am I missing something about building statically-linked Rust programs on Linux?

Fully static x86_64 Linux binaries do get built within the Alpine-based Docker image. One option for building the *unknown-linux-musl release would be to use the Dockerfile.alpine build and simply copy the release out of there. Another option might be using cross.

bradlarsen commented 2 months ago

I looked at this issue again just now. Had trouble building out of the box with cross, and was unable to build successfully when setting RUSTFLAGS='-C target-feature=+crt-static' in the Alpine Docker image (or indirectly via .cargo/config.toml).

It does look like the binary that is included in the Alpine-based Nosey Parker Docker image is statically linked. It seems like the simplest approach to get static noseyparker binaries for Linux will be to build the Alpine-based Docker image and copy the release out of there. We should do this to produce releases for both arm64 and x86_64 Linux variants.

Some GitHub Actions plumbing could do this. Might make sense to do that work while simultaneously fixing #156 to get much faster builds of the Docker images.