APK files missing at releases sinve v3.8.1

[IzzySoft]

Could you please add the APK files again to your releases (starting with the current one would be fine, no need to fill up the older ones)? Else Kiwix cannot be kept updated at e.g. IzzyOnDroid (which currently is stuck at 3.8.0 due to this).

Thanks in advance!

[kelson42]

I don't understand the big report, all APK are here https://download.kiwix.org/release/kiwix-android/

[MohitMaliFtechiz]

@kelson42 I think @IzzySoft is talking about our release section.

Release 3.11.1	Release 3.8.0

[MohitMaliFtechiz]

@IzzySoft We have updated the our release 3.11.1 with APK's files https://github.com/kiwix/kiwix-android/releases/tag/3.11.1.

[kelson42]

@MohitMaliFtechiz Can you easily do that? Do we know why this is not there anymore?

[MohitMaliFtechiz]

@MohitMaliFtechiz Can you easily do that? Do we know why this is not there anymore?

Yes, I was checking why it was not there and I found, in those releases which we self made these APKs are not available because these APKs are uploaded by our release.yml file after generating the APK https://github.com/kiwix/kiwix-android/blob/15760953d9d3e68b7fd26bf8ff62412bdee2a893/.github/workflows/release.yml#L49-L55 which are available in our release section but currently in draft.

It is not occurring after the release 3.8.1 if we see the releases before it which is self-made (not pushed by the Github action) these APKs are not available there as shown in the below screenshot.

[MohitMaliFtechiz]

@MohitMaliFtechiz Can you easily do that?

It needed the manual steps to do it:

1. Download all artifacts from the draft release.
2. Upload them in the existing release.
3. Delete the draft release.

So to reduce the manual steps, we have now directly uploaded the APKs to the existing release instead of creating the new draft release in https://github.com/kiwix/kiwix-android/pull/4054.

[IzzySoft]

I don't understand the big report, all APK are here https://download.kiwix.org/release/kiwix-android/

@kelson42 the updater at the IzzyOnDroid repo looks for them at the releases of the repo hosting the source code.

@MohitMaliFtechiz thanks for fixing! 🤩 Funnily the updater here now picks 3.8.1 as latest one (most likely because the Github API returns the releases by date, and 3.8.1 has the newest timestamp), but that should "fix itself" with the next release then hopefully:

$ iod repo get org.kiwix.kiwixmobile
org.kiwix.kiwixmobile: looking for 'https://api.github.com/repos/kiwix/kiwix-android/releases'
org.kiwix.kiwixmobile: checking tag '3.8.1'
org.kiwix.kiwixmobile: lastRelNo set to '3.8.1', checking for files
org.kiwix.kiwixmobile: Upstream file date (2023-12-03 16:14) is newer than ours (2023-11-25 19:40).
org.kiwix.kiwixmobile: returning ['3.8.1','https://github.com/kiwix/kiwix-android/releases/download/3.8.1/kiwix-armeabi-v7a-release.apk',1701616476]
org.kiwix.kiwixmobile: 3.8.0/3.8.1, https://github.com/kiwix/kiwix-android/releases: https://github.com/kiwix/kiwix-android/releases/download/3.8.1/kiwix-armeabi-v7a-release.apk
- Grabbing update for org.kiwix.kiwixmobile: OK
- Checking 'repo/org.kiwix.kiwixmobile_5230801.apk' for libraries and malware …
- Checking the app's AndroidManifest.xml …
! repo/org.kiwix.kiwixmobile_5230801.apk declares flag(s): usesCleartextTraffic
! repo/org.kiwix.kiwixmobile_5230801.apk declares sensitive permission(s): android.permission.ACCESS_FINE_LOCATION android.permission.MANAGE_EXTERNAL_STORAGE android.permission.READ_EXTERNAL_STORAGE android.permission.NEARBY_WIFI_DEVICES android.permission.ACCESS_COARSE_LOCATION
! repo/org.kiwix.kiwixmobile_5230801.apk contains signature block blobs: 0x504b4453 (DEPENDENCY_INFO_BLOCK; GOOGLE)
org.kiwix.kiwixmobile: check if repo contains FUNDING.yml
org.kiwix.kiwixmobile: looking for 'https://api.github.com/repos/kiwix/kiwix-android/contents/.github'
org.kiwix.kiwixmobile: FUNDING.yml already up-to-date.
org.kiwix.kiwixmobile: calling 'getFastlaneMeta(github,[host:github.com,owner:kiwix,repo:kiwix-android,path:/fastlane/metadata/android])'
org.kiwix.kiwixmobile: FastlaneFeatures shortdesc,fulldescMD,icon,screenshotsJPG
org.kiwix.kiwixmobile: looking for 'https://api.github.com/repos/kiwix/kiwix-android/contents/fastlane%2Fmetadata%2Fandroid'
org.kiwix.kiwixmobile: looking for 'https://api.github.com/repos/kiwix/kiwix-android/contents/fastlane%2Fmetadata%2Fandroid%2Fde'
...

Oops, some exclamation marks 😱 Let's see… I cannot find details on permissions in the repo (or in its wiki), so:

• what for are location permissions (especially ACCESS_FINE_LOCATION) needed?
• similarly, NEARBY_WIFI_DEVICES?
• does it really need MANAGE_EXTERNAL_STORAGE (I have not checked where files are stored, but shouldn't that be possible via SAF?)
• what kind of cleartext (unencrypted) network connections are made?

As for DEPENDENCY_INFO_BLOCK, that can easily be avoided with a minor addition to your build.gradle:

android {
    dependenciesInfo {
        // Disables dependency metadata when building APKs.
        includeInApk = false
        // Disables dependency metadata when building Android App Bundles.
        includeInBundle = false
    }
}

For some background: that BLOB is supposed to be just a binary representation of your app's dependency tree. But as it's encrypted with a public key belonging to Google, only Google can read it – and nobody else can even verify what it really contains. More details can be found e.g. here: Ramping up security: additional APK checks are in place with the IzzyOnDroid repo.

Thanks in advance!

[IzzySoft]

PS: I've just checked why 3.8.1 is considered the newest release: the publish date (published_at) is the newest there.

Pulled in 3.11.1 now manually, so that's the one going live here with the next sync around 7 pm UTC today.

[kelson42]

We have a mess here (also to many workflow files). We have to clean and republish the APK (but manual release triggering). I will have to look

[MohitMaliFtechiz]

what for are location permissions (especially ACCESS_FINE_LOCATION) needed? similarly, NEARBY_WIFI_DEVICES?

The ACCESS_FINE_LOCATION permission is used to discover peer-to-peer (P2P) devices for transferring ZIM files to nearby devices. For Android 13 and above, we request the NEARBY_WIFI_DEVICES permission, as recommended by the WifiP2pManager API, which allows us to directly find nearby devices without needing ACCESS_FINE_LOCATION. Since the NEARBY_WIFI_DEVICES permission was introduced in Android 13, we continue to request ACCESS_FINE_LOCATION for devices running versions before Android 13.

does it really need MANAGE_EXTERNAL_STORAGE (I have not checked where files are stored, but shouldn't that be possible via SAF?)

@IzzySoft We require the MANAGE_EXTERNAL_STORAGE to load the sideloaded ZIM files in our application, no currently, it is not possible to load the sideloaded ZIM files with SAF. Currently, libkiwix needs direct filePath to properly work with the Xapain index(We have an issue open for this https://github.com/openzim/libzim/issues/852). But as of now, we need this permission.

what kind of cleartext (unencrypted) network connections are made?

For this, we have opened a ticket https://github.com/kiwix/kiwix-android/issues/4055.

As for DEPENDENCY_INFO_BLOCK, that can easily be avoided with a minor addition to your build.gradle:

Thanks for this, we will fix this.

PS: I've just checked why 3.8.1 is considered the newest release: the publish date (published_at) is the newest there.

@IzzySoft, @kelson42 It was published by mistake by github action while fixing and testing this issue. My bad :disappointed_relieved:

Pulled in 3.11.1 now manually, so that's the one going live here with the next sync around 7 pm UTC today.

Thanks for doing this :heart:

[IzzySoft]

Thanks @MohitMaliFtechiz 😍 Permissions are added to Kiwix' "green list" now:

android.permission.ACCESS_COARSE_LOCATION: used on Android < 13 to discover peer-to-peer (P2P) devices for transferring ZIM files to nearby devices
android.permission.ACCESS_FINE_LOCATION: used on Android < 13 to discover peer-to-peer (P2P) devices for transferring ZIM files to nearby devices
android.permission.NEARBY_WIFI_DEVICES: used on Android ≥ 13 to discover peer-to-peer (P2P) devices for transferring ZIM files to nearby devices
android.permission.MANAGE_EXTERNAL_STORAGE: used to load the sideloaded ZIM files on Android ≥ 11
android.permission.READ_EXTERNAL_STORAGE: used to load the sideloaded ZIM files on Android < 11

Already visible if you visit Kiwix in the IzzyOnDroid repo browser. Please correct if I abbreviated/deduced wrongly. Thanks for splitting off the cleartextTraffic to follow it up – and for taking care for the DEPENDENCY_INFO_BLOCK!

It was published by mistake by github action while fixing and testing this issue. My bad 😥

No worries, it's fixed here by now. And as I wrote, the next release will "close the gap" again.

Thanks for doing this ❤️

Gladly – thanks for your support and swift action!

[IzzySoft]

Thanks again! Btw, the F-Droid badge on your Readme links to a 404. Maybe you want to pick an IzzyOnDroid badge and link it to your app there instead, @MohitMaliFtechiz?

[MohitMaliFtechiz]

Thanks again! Btw, the F-Droid badge on your Readme links to a 404. Maybe you want to pick an IzzyOnDroid badge and link it to your app there instead, @MohitMaliFtechiz?

@IzzySoft Thanks. We have updated our badge in https://github.com/kiwix/kiwix-android/pull/4100.

[IzzySoft]

Thanks!

[MohitMaliFtechiz]

@IzzySoft We have just released the version 3.12.0 and it has all the APKs artifact in the release https://github.com/kiwix/kiwix-android/releases/tag/3.12.0. So it could be now directly updated to the IzzyOnDroid without needing manual pulling?

Edited @IzzySoft One question, do you need output-metadata.json file for uploading the application on IzzyOnDroid? Since this file have the information about the all APKs.

[IzzySoft]

So it could be now directly updated to the IzzyOnDroid without needing manual pulling?

Triggering manually what would have run automatically later today:

$ iod repo get org.kiwix.kiwixmobile
org.kiwix.kiwixmobile: looking for 'https://api.github.com/repos/kiwix/kiwix-android/releases'
org.kiwix.kiwixmobile: checking tag '3.12.0'
org.kiwix.kiwixmobile: lastRelNo set to '3.12.0', checking for files
org.kiwix.kiwixmobile: Upstream file date (2024-11-27 12:28) is newer than ours (2024-11-05 19:44).
org.kiwix.kiwixmobile: returning ['3.12.0','https://github.com/kiwix/kiwix-android/releases/download/3.12.0/3.12.0-armeabi-v7a-standalone.apk',1732706891]
org.kiwix.kiwixmobile: 3.11.1/3.12.0, https://github.com/kiwix/kiwix-android/releases: https://github.com/kiwix/kiwix-android/releases/download/3.12.0/3.12.0-armeabi-v7a-standalone.apk
- Grabbing update for org.kiwix.kiwixmobile: OK
- Checking 'repo/org.kiwix.kiwixmobile_5231234.apk' for libraries and malware …
- Checking the app's AndroidManifest.xml …
? repo/org.kiwix.kiwixmobile_5231234.apk no longer carries sensitive permission(s): android.permission.ACCESS_COARSE_LOCATION
! repo/org.kiwix.kiwixmobile_5231234.apk contains signature block blobs: 0x504b4453 (DEPENDENCY_INFO_BLOCK; GOOGLE)
org.kiwix.kiwixmobile: check if repo contains FUNDING.yml
org.kiwix.kiwixmobile: looking for 'https://api.github.com/repos/kiwix/kiwix-android/contents/.github'
org.kiwix.kiwixmobile: FUNDING.yml already up-to-date.
org.kiwix.kiwixmobile: calling 'getFastlaneMeta(github,[host:github.com,owner:kiwix,repo:kiwix-android,path:/fastlane/metadata/android])'
org.kiwix.kiwixmobile: FastlaneFeatures shortdesc,fulldescMD,icon,screenshotsJPG
org.kiwix.kiwixmobile: looking for 'https://api.github.com/repos/kiwix/kiwix-android/contents/fastlane%2Fmetadata%2Fandroid'
org.kiwix.kiwixmobile: looking for 'https://api.github.com/repos/kiwix/kiwix-android/contents/fastlane%2Fmetadata%2Fandroid%2Fde'
...

Looks good so far. But, we don't have that app:

2024-11-27 14:35:32,250 WARNING: org.kiwix.kiwixmobile_5231234.apk (org.kiwix.kiwixmobile.standalone) has no metadata!
    Use `fdroid update -c` to create it.

What we have here is org.kiwix.kiwixmobile – which is not available at your releases, so the update will fail 🤷‍♂ Any reason those APKs have a different packageName/applicationId? To avoid the non-matching APK being downloaded over and again, I've told the updater to ignore /standalone/ for now:

$ iod repo get org.kiwix.kiwixmobile
org.kiwix.kiwixmobile: looking for 'https://api.github.com/repos/kiwix/kiwix-android/releases'
org.kiwix.kiwixmobile: checking tag '3.12.0'
org.kiwix.kiwixmobile: lastRelNo set to '3.12.0', checking for files
No true match found for org.kiwix.kiwixmobile
org.kiwix.kiwixmobile: Upstream file date (2024-11-05 19:44) not newer than ours (2024-11-05 19:44), skipping.
org.kiwix.kiwixmobile: returning local appdata as ['3.11.1','',0]
org.kiwix.kiwixmobile: 3.11.1 up-to-date.

(and btw, there's no direct update path from org.kiwix.kiwixmobile to org.kiwix.kiwixmobile.standalone – a different packageName means it's a different app)

One question, do you need output-metadata.json file for uploading the application on IzzyOnDroid? Since this file have the information about the all APKs.

No, the updater here checks the releases for files matching the defined pattern, For Kiwix, that pattern is /armeabi-v7a/(and of course restricted to *.apk, and ignoring /debug/ (hopefully) – and now also ignores /standalone.

[MohitMaliFtechiz]

@IzzySoft Thank you for your response.

What we have here is org.kiwix.kiwixmobile – which is not available at your releases, so the update will fail 🤷‍♂ Any reason those APKs have a different packageName/applicationId? To avoid the non-matching APK being downloaded over and again, I've told the updater to ignore /standalone/ for now: (and btw, there's no direct update path from org.kiwix.kiwixmobile to org.kiwix.kiwixmobile.standalone – a different packageName means it's a different app)

It was due to https://github.com/kiwix/kiwix-android/issues/3933 and I have opened a separate ticket for this problem https://github.com/kiwix/kiwix-android/issues/4114.

[MohitMaliFtechiz]

@IzzySoft We have updated our release with org.kiwix.kiwixmobile package name https://github.com/kiwix/kiwix-android/releases/tag/3.12.0, and from the next release it will automatically publish these APKs to release.

No, the updater here checks the releases for files matching the defined pattern, For Kiwix, that pattern is /armeabi-v7a/(and of course restricted to *.apk, and ignoring /debug/ (hopefully) – and now also ignores /standalone.

@IzzySoft Why it is only taking the /armeabi-v7a/ APK, this APK is only for the armeabi-v7a architecture devices. For example, I have an arm-64-v8a device and I have downloaded the kiwix application from IzzyOnDroid https://apt.izzysoft.de/fdroid/index/apk/org.kiwix.kiwixmobile, but it is not compatible with my device.

So either it should upload all abi APKs on IzzyOnDroid or should take the universal APK from github release so that it can be installed on different architecture devices.

Like it is uploading different abi APKs for the ADB over Network application. But for Kiwix it is only uploading the armeabi-v7a APK.

[IzzySoft]

We have updated our release with org.kiwix.kiwixmobile package name

Thanks! That will then be picked up with the next run of the updater later today.

Why it is only taking the /armeabi-v7a/ APK

We have a size limit of 30 MB per app, so we had to decide for one. Back then it was decided for armeabi because it offers the widest compatibilities (excluding only very few devices like the Pixel 7a plus, where vendors decided to make the OS "arm64 only". At IzzyOnDroid, we want to support longevity, to enable the use of older devices (e.g. with custom ROMs) for as long as they last (I e.g. still use a 2015 Fairphone 2, which is 32bit and thus cannot support 64bit apps).

So either it should upload all abi APKs on IzzyOnDroid or should take the universal APK

Multiple APKs per release are not supported (even at F-Droid they'd then need a unique versionCode to be distinguishable). Apart from that, see what I just wrote about the size limit: even the armeabi is already scratching that. All ABIs together would be much beyond it, and the universal one is factor 3+ beyond it – so no, unfortunately not.

Like it is uploading different abi APKs for the ADB over Network application.

That's a single APK covering all ABIs ("universal APK"). You've cut off the screenshot before the APK size, but that APK there is much smaller: just 2 MB per APK.

[MohitMaliFtechiz]

Multiple APKs per release are not supported (even at F-Droid they'd then need a unique versionCode to be distinguishable).

It is already there in our code, all the APKs are generated with unique versionCode.

We have a size limit of 30 MB per app

Ohh!! I was not aware of that, Thanks for the explanation. Now I can understand why only armeabi-v7a is published on the IzzyOnDroid. Closing the issue.

[IzzySoft]

With 32bit devices slowly aging out (and in case of Kiwix, I'm not even sure if they're still capable of running the app with an acceptable "fluidity"), we can of course switch to the arm64 build later.

Btw, as you might be aware of, it's the native libs which per APK make up ~18 MB, and are stored uncompressed. I've no idea if any size optimization is possible there, but you've probably checked that already.

[MohitMaliFtechiz]

Btw, as you might be aware of, it's the native libs which per APK make up ~18 MB, and are stored uncompressed. I've no idea if any size optimization is possible there, but you've probably checked that already.

Yes, We have done the all optimization to minimize the APK size.