Ensure SecHub release builds are reproducible

[Jeeppler]

Problem

Reproducible builds means, we can reproduce the same artifacts based on the same git commit and java version regardless of the machine we build it (more information can be found at: https://reproducible-builds.org/).

There are some challenges with reproducible builds and Java. Those challenges are explained here: https://reproducible-builds.org/docs/jvm/.

Gradle supports reproducible builds since 3.4: https://docs.gradle.org/current/userguide/working_with_files.html#sec:reproducible_archives.

For the Go programming language there is some article: https://words.filippo.io/reproducing-go-binaries-byte-by-byte/.

Wanted

• SecHub Server, PDS, and other Java artifacts are build reproducible
• The SecHub Client is build reproducible

[de-jcup]

Our builds should be reproduceable since the first initial commit SecHub:

• Our complete build mechanism is inside code
• We use dedicated library versions (we do not use dependencies ala x.y.* or something else).
• No timestamps or something changing inside the builds

So normally a restart of a build must produce the same jar/go file.

The only pain points are IMO:

• github would change their build system in the meanwhile
• jenkins configurations/plugins change
• documentation (maybe)

But I would leave that out here.

So IMO we have this already, so my questions: If you build the same commit two times with our GitHub workflows, do we currently have

• different artifacts on Java side ?
• different artifacts on Go side ?

Is there any other reason for this issue?

[Jeeppler]

First, while in theory the builds should be byte-to-byte reproducible, I am almost sure nobody ever verified that.

Second, there are issues with the ordering in ZIP files and timestamps with Java. In addition, the SecHub executables are zipped and might have the same problems.

[company-jcup]

We verified this inside the team and...

user> sha256sum sechub-pds/build/libs/sechub-pds-0.38.0.jar
c7fe96cdf07510999e9e2a58ef046538e790f6dc7e233d90f5ecc3ec473bb30b  sechub-pds/build/libs/sechub-pds-0.38.0.jar
fc902fe289170c3e5db0d3f177765f029304db730b9586fdd9fb4180310566a1  sechub-pds-0.38.0.jar

user> sha256sum sechub-server/build/libs/sechub-server-0.43.0.jar
7aaacfec41ff8f285e190acba0d0747831d2b00427c18feec9280264f1c36ef9  sechub-server/build/libs/sechub-server-0.43.0.jar
7440ff5cd6896ddebf9e61486188e43849caa9e909445d5a69a32ff74d9ecbdf  sechub-server-0.43.0.jar

unfortunately local builds and github actions build provide different binaries.

We tried also to build the jars locally multiple times and the sha256 checksum did change here as well!!

[sven-dmlr]

Same for the Go client:

> ./gradlew buildGo
> sha256sum sechub-cli/build/go/platform/linux-386/sechub
04a398c857df676650c558797435a29ea568be819ab8f4467cdedc8f879b37bf  sechub-cli/build/go/platform/linux-386/sechub

> ./gradlew buildGo
> sha256sum sechub-cli/build/go/platform/linux-386/sechub
50e758b13d89b180d2bdb8e466cf447f3389b980cd10948ac778271a781101ac  sechub-cli/build/go/platform/linux-386/sechub