Open IzzySoft opened 1 month ago
Hi,
Reproducible Builds sound great for open-source community. And you are correct. selfFossRelease
task is disabled on the public repo. self
builds are those built by us and signed with our own keys. fdroid
builds are meant to be built by F-Droid (or the open-source community) and signed by their keys. This is to identify the builds e.g. when we get crash logs or support requests for different builds.
To enable the self
builds one can simply change:
if (!names.contains('fdroid') || !names.contains('foss')) {
variant.setIgnore(true)
}
to
if ((!names.contains('fdroid') && !names.contains('self')) || !names.contains('foss')) {
variant.setIgnore(true)
}
in build.gradle.
Thanks! But oh, now I need to get hold of all the RegEx I can gather… oof:
's/if (\(\!names\.contains\('"'"'fdroid'"'"'\)) (..) \!names\.contains\('"'"'foss'"'"'\)\).+/if (\1 \&\& !names.contains('"'"'self'"'"')) \2 !names.contains('"'"'foss'"'"')) {/'
Took me a while to cobble that together. Single quotes and pipes are tricky. OK, now let's see, build is running… Well, actually it's taking ages to download all those gradle plugins, at around 100kB/s (my line would easily give 35MB/s, so this looks like throttling) :disappointed: The "durian" one is downloading now for several minutes already.
(3 builds later)
OK, now the build succeeds (had a typo in the original RegEx, already corrected above). 4th build running now as the output file wasn't where it was expected.
Finally, there we go: it's not RB. Here comes the APK diff:
-------------------------------
--- /dev/fd/63 2024-08-04 16:37:12.983054700 +0200
+++ /dev/fd/62 2024-08-04 16:37:12.983054700 +0200
@@ -1,11 +1,11 @@
META-INF/com/android/build/gradle/app-metadata.properties
32-bit CRC value (hex): 41228e1e
assets/dexopt/baseline.prof
- 32-bit CRC value (hex): 3a96f3c4
+ 32-bit CRC value (hex): 27206cc5
assets/dexopt/baseline.profm
32-bit CRC value (hex): ab338062
classes.dex
- 32-bit CRC value (hex): 86203828
+ 32-bit CRC value (hex): 24b49127
classes2.dex
32-bit CRC value (hex): 35d55042
META-INF/androidx.activity_activity.version
@@ -1844,9 +1844,3 @@
32-bit CRC value (hex): 9f343fb7
resources.arsc
32-bit CRC value (hex): 1705207c
Putting that in order: baseline.prof
differs because of classes.dex
does. Let's start with that… hm, two pages of diff. Comparing red (from your APK) vs. green (from my build) it seems the APK was either built from a later commit with things added, or from an earlier one and things got dropped. Example:
|: invoke-static {}, Lv2/a;.i:()Z
|: move-result v0
-|: if-eqz v0, 0022 // +0014
-|: invoke-static {}, Ly1/i;.u:()Z
-|: move-result v0
-|: if-eqz v0, 0022 // +000e
+|: if-eqz v0, 001c // +000e
|: iget-object v0, v2, Lcom/mirfatif/mylocation/MainActivity;.v:Lu2/a;
|: iget-object v0, v0, Lu2/a;.d:Landroid/widget/TextView;
Looking at the timestamps I bet the former: tag is from November 2023, APK from January 2024. But there's no commit after the tagged one – so maybe you didn't push that change? §1 of Izzy's RB hints:
§1: Always build the APK from exactly the commit your release-tag will point to!
:wink:
So how can we get that fixed? Any hints from your side?
Looking at the timestamps I bet the former: tag is from November 2023, APK from January 2024. But there's no commit after the tagged one – so maybe you didn't push that change?
There's no change in source code after Nov'23. But APK was rebuilt in Jan'24 with v1 signing scheme enabled as you had reported an issue earlier (https://github.com/mirfatif/MyLocation/issues/16).
So how can we get that fixed? Any hints from your side?
Compiled code for selfFoss
and fdroidFoss
variants won't ever be identical, even if built at the same time. The files generated by Android build system could be different for different variants. In our case, the contents of BuildConfig.java
file, for instance, are different due to this and this. So the resulting dex files would also differ.
Compiled code for
selfFoss
andfdroidFoss
variants won't ever be identical
No, and I would not expect them to be. That's why I asked you how to build the selfFoss
variant, as that's what is listed at IzzyOnDroid. Here's the recipe so you can see what is compared:
---
repository: https://github.com/mirfatif/MyLocation.git
updates: releases
versions:
- tag: v1.05
apks:
- apk_pattern: MyLocation.*\.apk
apk_url: https://github.com/mirfatif/MyLocation/releases/download/v1.05/MyLocation_v1.05.apk
build:
- sed -r 's/if (\(\!names\.contains\('"'"'fdroid'"'"'\)) (..) \!names\.contains\('"'"'foss'"'"'\)\).+/if (\1 \&\& !names.contains('"'"'self'"'"')) \2 !names.contains('"'"'foss'"'"')) {/' -i app/build.gradle
- ./gradlew --info assembleSelfFossRelease
- find . -name '*.apk'
- mv app/build/outputs/apk/selfFoss/release/app-self-foss-release-unsigned.apk /outputs/unsigned.apk
build_home_dir: /build
build_repo_dir: /build/repo
build_user: build
provisioning:
android_home: /opt/sdk
build_tools:
cmake:
cmdline_tools:
version: '12.0'
url: https://dl.google.com/android/repository/commandlinetools-linux-11076708_latest.zip
sha256: 2d2d50857e4eb553af5a6dc3ad507a17adf43d115264b1afc116f95c92e5e258
extra_packages: []
image: debian:bookworm-slim
jdk: openjdk-17-jdk-headless
ndk:
platform:
platform_tools:
tools:
verify_gradle_wrapper: true
It compares the APK attached to your release here (apk_url
) with the one created by the recipe. You see I've used the above RegEx to make sure it can build selfFoss
, then called it with ./gradlew assembleSelfFossRelease
(the --info
was just in for debugging). Finally used is app/build/outputs/apk/selfFoss/release/app-self-foss-release-unsigned.apk
– which should be the selfFoss
build. Isn't that what is attached to your release here? If not, maybe I wasn't clear enough with my question: How would I build the APK you've attached to releases here? Apologies for any confusion I might have caused!
You were perfectly clear. Probably I was sleepy. :smile:
Ok. I've synced my private branch (on which I work and build APKs, and which contains extra code for non-FOSS variants) with the public one (which I push to GitHub) and ran assembleSelfFossRelease
on both branches. The resulting classes.dex
still differs. At the moment I don't know why. And don't have time to investigate. Probably there's something wrong with Java API desugaring.
For now I've built the APK on public branch and released v1.06. See if it fixes the problem.
What shall I say? Welcome to the collection of Reproducible Builds at IzzyOnDroid then, Irfan – and thanks for your help! This was a full success.
Err… but a flaky one, second try failed. Lemme see what the 3rd one says… OK, success again. The failure was because some short network hick-up it seems. Should that happen more often, I guess I could simply alter the relevant build step to
- ./gradlew --info assembleSelfFossRelease || ./gradlew --info assembleSelfFossRelease
i.e. repeat the build if it fails :man_shrugging: But for now: v1.06 will show up as RB with the next sync around 6 pm UTC :smiley:
Feel free to close this ticket. I left it open in case you think there's something left to do on your end (e.g. making a note to process future releases like this one :stuck_out_tongue_winking_eye:)
Hi Irfan, you might have noticed that the IzzyOnDroid repo is now publicly featuring Reproducible Builds (see: Reproducible bulds, special client support and more in our repo). So of course I wanted to get your apps in, but have a problem with MyLocation: How do I build
SelfFossRelease
– assuming that is the one listed at IzzyOnDroid? I failed miserably, only succeeding withFdroidFossRelease
.We'd appreciate if you could help making your build reproducible. We've prepared some hints on reproducible builds for that.
Looking forward to your reply!