Reproducible Builds

[IzzySoft]

I've checked your app if its build is reproducible (see: Reproducible bulds, special client support and more in our repo), but while I was able to successfully generate the APK using ./gradlew assembleRelease, but there were differences between the APK generated here and the one you provide. Most noticable, your APK seems to contain an additional database:

-------------------------------
--- /dev/fd/63  2024-07-28 18:14:04.132439662 +0200
+++ /dev/fd/62  2024-07-28 18:14:04.132439662 +0200
@@ -18,8 +18,6 @@
   32-bit CRC value (hex):                         54a56b50
   assets/database/arru.db
   32-bit CRC value (hex):                         d9f32e86
-  assets/database/baka-arru.db
-  32-bit CRC value (hex):                         5e6c9cfc
   DebugProbesKt.bin
   32-bit CRC value (hex):                         d5ac4dc2
   META-INF/androidx.activity_activity-compose.version
@@ -173,9 +171,9 @@
   META-INF/kotlinx_coroutines_core.version
   32-bit CRC value (hex):                         8f7d2509
   META-INF/services/O4.B
-  32-bit CRC value (hex):                         b728e702
+  32-bit CRC value (hex):                         04b5dcf1
   META-INF/services/P4.a
-  32-bit CRC value (hex):                         b56e595b
+  32-bit CRC value (hex):                         2f988f32
   kotlin-tooling-metadata.json
   32-bit CRC value (hex):                         68ab5328
   kotlin/annotation/annotation.kotlin_builtins

Could it be you've built from a "dirty tree" and that assets/database/baka-arru.db should not even be there?

The other differences are easy to fix on our end (you probably build on Windows, as those files have DOS line endings – while we build on Linux and thus the files here have Linux file endings).

We'd appreciate if you could help making your build reproducible. We've prepared some hints on reproducible builds for that.

Looking forward to your reply!

[KSSidll]

Thanks for noticing that that happened, the file wasn't supposed to be there as it's a database I used for testing the app

I updated the latest v2 release with a clean build from the same commit

It should be good now

[IzzySoft]

Congrats :partying_face: That did it! Now there's just one thing left I see:

! repo/com.kssidll.arru_35.apk contains signature block blobs: 0x504b4453 (DEPENDENCY_INFO_BLOCK; GOOGLE)

And that's easily fixed at your end, by a little modification to your build.gradle:

android {
    dependenciesInfo {
        // Disables dependency metadata when building APKs.
        includeInApk = false
        // Disables dependency metadata when building Android App Bundles.
        includeInBundle = false
    }
}

For some background: that BLOB is supposed to be just a binary representation of your app's dependency tree. But as it's encrypted with a public key belonging to Google, only Google can read it – and nobody else can even verify what it really contains. More details can be found e.g. here: Ramping up security: additional APK checks are in place with the IzzyOnDroid repo.

Thanks a lot!

One more thing: while it helped for a quick fix now, please never replace an already distributed file at a release. Better make a new release (can be a "maintenance release"). Those already having updated to the latest release won't get this APK – and now that your app has RB enabled, it would void RB and no longer match the "proof".

Starting with the next sync around 6 pm UTC, your app's latest version will carry the "green shield" – and future versions hopefully will hold it up :smiley: Preview from staging:

And here comes your "welcome message": https://floss.social/@IzzyOnDroid/112899790609030217

Next (!) release then even without that blob? :smiley_cat:

[KSSidll]

Thank you ❤️

I made a new release without the blob

[IzzySoft]

Then let's see if the updater works, shall we?

$ iod repo get com.kssidll.arru
com.kssidll.arru: looking for 'https://api.github.com/repos/KSSidll/Arru/releases'
com.kssidll.arru: checking tag 'v2.4.7'
com.kssidll.arru: lastRelNo set to '2.4.7', checking for files
com.kssidll.arru: Upstream file date (2024-08-03 21:59) is newer than ours (2024-08-03 20:27).
com.kssidll.arru: returning ['2.4.7','https://github.com/KSSidll/Arru/releases/download/v2.4.7/Arru-2.4.7.apk',1722715174]
com.kssidll.arru: 2.4.6/2.4.7, https://github.com/KSSidll/Arru/releases: https://github.com/KSSidll/Arru/releases/download/v2.4.7/Arru-2.4.7.apk
- Grabbing update for com.kssidll.arru: OK
- Checking 'repo/com.kssidll.arru_36.apk' for libraries and malware …
- Checking the app's AndroidManifest.xml …
com.kssidll.arru: check if repo contains FUNDING.yml
com.kssidll.arru: looking for 'https://api.github.com/repos/KSSidll/Arru/contents/.github'
com.kssidll.arru: Github reports "Not Found" for https://api.github.com/repos/KSSidll/Arru/contents/.github
com.kssidll.arru: looking for 'https://api.github.com/repos/KSSidll/Arru/contents/'
com.kssidll.arru: looking for 'https://api.github.com/repos/KSSidll/.github/contents/'
com.kssidll.arru: Github reports "Not Found" for https://api.github.com/repos/KSSidll/.github/contents/
com.kssidll.arru: no FUNDING.yml detected.
com.kssidll.arru: no Fastlane configured, skipping Fastlane check.

Looks good! And no mention of that blob :partying_face: Want a PR with a Fastlane starter kit? Fastlane would allow you to maintain how your app is presented (description, graphics etc). I could send you what is currently set up with IzzyOnDroid – and you can then use e.g. the IzzyOnDroid Fastlane Documentation to guide you with building on it. Takes me no more than 5 minutes to have the PR open – should you accept. As you see, your adjustments to it would then be checked whenever an update is pulled, and would be pulled along.

If you wonder about FUNDING.yml: that is if you want to propagate that and how one can support you with donations.

[KSSidll]

Yeah, that would be very helpful

Thanks again for reaching out! Never expected anything like that to happen lol

[KSSidll]

I will close this as everything seems to have been resolved

[IzzySoft]

RB check for the new release is running already :crossed_fingers: Oh, and be welcome to pick a badge to link to Arru's presence at IzzyOnDroid if you wish – multiple variants to choose from, including "shields" from shields.io.

I'll prepare the PR soon, just need a little break now. But expect it shortly!

Oh, and in case I didn't yet say so: welcome aboard IzzyOnDroid with Arru! :star_struck:

[IzzySoft]

And there you go: PR is open! RB for the new release succeeded btw, so with the next sync Arru will show up publicly with 2 RB releases :smiley: Oh, and your shield will of course reflect that then, thanks for adding it!

[KSSidll]

Happy to be part of the Izzy Repo 😄 Thank you for trying to make Android floss ecosystem more available to the average user

we need more floss apps on android fr

[IzzySoft]

we need more floss apps on android

Working on it to make more of them available – and the available ones more floss – and all of them more visible plus the system as safe as we can, as you've noticed :wink:

Thanks for your help!

[IzzySoft]

@KSSidll v2.4.8 failed RB again. Did you really build from a clean tree at the commit the tag points to? APK diff:

-------------------------------
--- /dev/fd/63  2024-08-13 20:05:34.755419539 +0200
+++ /dev/fd/62  2024-08-13 20:05:34.755419539 +0200
@@ -3,11 +3,11 @@
   META-INF/version-control-info.textproto
   32-bit CRC value (hex):                         f6fecc61
   assets/dexopt/baseline.prof
-  32-bit CRC value (hex):                         fb927fe9
+  32-bit CRC value (hex):                         ded44b15
   assets/dexopt/baseline.profm
   32-bit CRC value (hex):                         75395d5a
   classes.dex
-  32-bit CRC value (hex):                         e9fb32eb
+  32-bit CRC value (hex):                         69228c44
   lib/arm64-v8a/libdatastore_shared_counter.so
   32-bit CRC value (hex):                         4f56ac38
   lib/armeabi-v7a/libdatastore_shared_counter.so
@@ -836,9 +836,3 @@
   32-bit CRC value (hex):                         87d58c52
   resources.arsc
   32-bit CRC value (hex):                         5010ded9

I'll reset today's run in the hope you can tell me the commit to use (do not replace the APK please as that's already distributed now), and a subsequent run succeeds then.

[KSSidll]

huh that's weird i built after everything was merged and on the upstream so it should be the exact same commit with clean tree i will try to investigate what could have happened

[KSSidll]

i built the apk again from the d60393e commit and it has the same SHA256 🤔

[KSSidll]

this is quite interesting

i did a clean clone and then built with that, same CRC as what you got but i can't seem to find where the difference is from

somehow deleting gradle build files and building anew created a completely different CRC (classes.dex got 77C71FAF) even thought this shouldn't affect it at all

i think the best way would be for me to containerize the build process instead of building from android studio level with all that said not sure RB for this version is achievable

[obfusk]

Yes, unfortunately (for Reproducible Builds) Android Studio tends to do "incremental builds", resulting in these kinds of odd differences -- usually in DEX files -- that shouldn't happen. Which is why we recommend cleaning the project and cache before building the release APK, though that isn't always sufficient. And even CLI builds will re-use build artefacts and gradle cache. Containerising your build -- like our completely clean builds in a fresh container -- should indeed help.

We do find that CLI builds with ./gradlew clean assembleRelease --no-build-cache --no-configuration-cache --no-daemon often produce clean results identical to ours, without the need to containerise the build process (though the latter would be the "cleanest").

[obfusk]

FYI: I'm planning on making it easy to use our rbtlog tooling to simply build an APK in a container using the same build recipe (and built-in provisioning and everything) we use, with the RB check being optional: https://github.com/obfusk/rbtlog/issues/471

[KSSidll]

Using the same tool (and recipes) would certainly make it easier for everyone yeah for now I will test apk build via docker compose but I will follow the progress on rbtlog

[IzzySoft]

rbtlog does support Docker as well, if you prefer that over Podman (we mostly use Podman there).

[KSSidll]

i thought the tool supports RB verification and logging for now, with local apk build coming later, didn't notice that you can save the built apk locally until just now that's why i made my thing (lol)

i will experiment with that later

[IzzySoft]

That's what Fay just plans to implement. Once the new feature is ready, you can e.g. call the build script telling it "just build from that commit and give me the APK". I't not yet there; Fay was reaching out for feedback before starting implementation, to know what other "steak holders" (intentional misspelling) think about it. After all, this part is for you who might want to use it to "simply build", so it should cover your needs :wink:

[IzzySoft]

v2.4.8 is marked failed here now as we cannot heal it anymore. Let's see we get this solved for the next release.

[KSSidll]

next release will be built in a container so it should be RB unless something else goes wrong

i will leave this open until there is an actual confirmation of RB (hopefully next release)

[IzzySoft]

Be welcome to give me a ping if I shall test before you release.

[obfusk]

FYI: I'm planning on making it easy to use our rbtlog tooling to simply build an APK in a container using the same build recipe (and built-in provisioning and everything) we use, with the RB check being optional: obfusk/rbtlog#471

FWIW, this is implemented now :)

[KSSidll]

From what i see RB passed without issues So seems to be fine for now

[IzzySoft]

Yupp, went through without any issue today, thanks!