Reproducible builds

[IzzySoft]

I've rebuilt your app from the commit behind the latest tag/release, but it failed to be reproducible:

-------------------------------
--- /dev/fd/63  2024-07-19 00:04:22.195372633 +0200
+++ /dev/fd/62  2024-07-19 00:04:22.195372633 +0200
@@ -3,7 +3,7 @@
   META-INF/version-control-info.textproto
   32-bit CRC value (hex):                         c16a89e6
   assets/dexopt/baseline.prof
-  32-bit CRC value (hex):                         190816fb
+  32-bit CRC value (hex):                         bbcb104b
   assets/dexopt/baseline.profm
   32-bit CRC value (hex):                         20e4a7fd
   classes.dex
@@ -11,7 +11,7 @@
   classes2.dex
   32-bit CRC value (hex):                         76bc99fd
   classes3.dex
-  32-bit CRC value (hex):                         54f43960
+  32-bit CRC value (hex):                         716c4b22
   lib/arm64-v8a/libmapbox-gl.so
   32-bit CRC value (hex):                         0d7ff9e5
   lib/armeabi-v7a/libmapbox-gl.so
@@ -2026,9 +2026,3 @@
   32-bit CRC value (hex):                         0ef1b92c
   resources.arsc
   32-bit CRC value (hex):                         f2fd0cb2

Was the APK attached to that release really built from that commit – or do I maybe miss some "environment settings" or build parameters? I've been simply running ./gradlew assembleRelease.

If you wonder about the background of this, be welcome to take a look at Reproducible bulds, special client support and more in our repo :wink: And picking a core element which could help you here, that would be the link to our hints on reproducible builds.

Thanks in advance for your help!

[dbrgn]

The app should be built from the same commit as the release tag, yes. How did you compare the builds?

IIRC the baseline.prof file is not reproducible, for reproducible builds it must be turned off.

In general, this app is not currently optimized for reproducible builds. I'll treat this like a feature request, not like a bug report 🙂

[IzzySoft]

I'll treat this like a feature request, not like a bug report 🙂

That's exactly how it was meant, thanks!

How did you compare the builds?

I use rbtlog. Put into the shortest words: build the APK from source, then copy over the signature from the APK you provide using apksigcopier, then see if it verifies. So if yours does have the baseline.prof and I'd turn it off at my end, it wouldn't be RB either.

In your case, it's not the baseline only, it's also the classes3.dex that differs. Which suggests your APK was "built differently": either from a different commit, or from a "dirty tree" (e.g. cached left-overs), or with a different JDK. As you ruled out the first, may I ask for the other two? I usually build with Java-17, but could also build with 21 (or 11 if must-be). If DEX differs, baseline usually differs, too.

[dbrgn]

I built from within Android Studio ("Generate Signed APK") with Target JDK 17. It's possible that there were cached leftovers. For reproducible builds, we'd need to build from a well-defined environment (e.g. using a Dockerfile).

[dbrgn]

Details about RB at IzzyOnDroid: https://android.izzysoft.de/articles/named/iod-rbs-mirrors-clients?lang=en

[IzzySoft]

It's possible that there were cached leftovers

Then this advice might help:

So, maybe you can ask developers if they are using android studio to do a clean project > rebuild project > generate signed apk and then upload the generated apk to github, so that most rb are a success.

(source)

build from a well-defined environment (e.g. using a Dockerfile).

rbtlog here indeed does exactly that, just instead of Docker it uses a Podman container. I was using debian:bookworm-slim as base image, and openjdk-17-jdk-headless as JDK.

Details about RB at IzzyOnDroid:

Yupp, that's the link from my opening post. Apologies, mine had a typo, which I just fixed now.

[dbrgn]

Yupp, that's the link from my opening post.

Oops sorry, I totally missed that...

I'll revisit the topic once we create the next release 🙂

[IzzySoft]

Thanks! Just give me a ping when I shall try again :smiley:

[obfusk]

IIRC the baseline.prof file is not reproducible, for reproducible builds it must be turned off.

The .profm (not .prof) used to be unreproducible. You used my workaround for that for Threema before upgrading to AGP with the fix (the version used by this app should have that fix and indeed we see that file isn't affected here).

But the .prof will (almost) always differ when any of the .dex files do as it contains checksums for those. So as Izzy said, with a differing .dex a differing .prof is expected.

We have seen odd .prof differences without corresponding .dex differences with recent versions (>= 8.4) of AGP. We'd be happy to investigate that with you if you happen run into that. But for now it looks like the .dex differences here are the cause of the .prof differences.