Build issue

[IzzySoft]

I've tried to build codec2_talkie from source, but I failed with all attempts. How do you build it? Your workflow only specifies "ubuntu:latest", which would mean JDK-21 – but that cannot work with Gradle 6.5 (which your wrapper specifies). Injecting a newer gradle (I tried 8.9) fails with

FAILURE: Build failed with an exception.

* What went wrong:
org/gradle/initialization/BuildCompletionListener
> org.gradle.initialization.BuildCompletionListener

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
> Get more help at https://help.gradle.org.

Deprecated Gradle features were used in this build, making it incompatible with Gradle 9.0.

Build mostly works when I go down to JDK-11 however (to meet the ancient gradle version), though then it fails at the end:

Execution failed for task ':libcodec2-android:stripReleaseDebugSymbols'.
> No toolchains found in the NDK toolchains folder for ABI with prefix: arm-linux-androideabi

Any hints?

[sh123]

Just building it in Android Studio (Arctic Fox), probably this IDE version is very outdated.

[IzzySoft]

Ah, probably. And not sure what SDK etc. it uses, or other modifications it makes on the way. I vaguely remember several things which ended up not being reproducible when the "original" was build with that IDE (or some other as well).

It should basically build with simply calling ./gradlew assembleRelease from the command line. Maybe you could give that a try and follow the hints it might show? I'm no Android dev, so some of them are rather hard for me to understand or even interpret what to do.

Btw, just found something that might be a clue:

Latest NDK removed support for mips abi, and earler version of android gradle plugin still check for the existance of mips toolchain

We're not talking about "mips" here, but still: maybe arm-linux-androideabi is using a different name nowadays? Your gradle-wrapper.properties points to a rather ancient gradle version (6.5 – we meanwhile passed 8.9 IIRC). So your "very outdated" might be a euphemism there :speak_no_evil:

If you accept the challenge: can you give me a ping once you updated your tooling and succeeded with a new release using the new environment? I'd then check again.

Background of my question: codec2_talkie is listed at IzzyOnDroid, where we support Reproducible Builds (see e.g. Reproducible Builds, special client support and more at IzzyOnDroid). I just tried to get the "green shield" up for your app :wink:

PS: be welcome to pick a badge to link to your app at IoD e.g. from your Readme :smiley:

[sh123]

No toolchains found in the NDK toolchains folder for ABI with prefix: arm-linux-androideabi

@IzzySoft , there must be some problem with NDK, this version is hardcoded in gradle build file android.ndkVersion "21.4.7075529", this needs to be installed with sdkmanager, it will appear in $ANDROID_HOME/ndk/21.4.7075529, not sure if gradle can download it and install automatically.

I think working combination is JDK-11 + NDK 21.4.7075529 + Gradle 6.5 + <???>, at least it compiles from command line this way (also need to disable lint otherwise it will fail, committed this change to disable lint):

# ANDROID_HOME=~/Android/Sdk JAVA_HOME=~/Downloads/android/android-studio/jre ./gradlew build
> Task :libcodec2-android:compileCodec2
-- codec2 version: 1.0.5
.............
BUILD SUCCESSFUL in 1m 29s
194 actionable tasks: 59 executed, 135 up-to-date

I'll try to upgrade, but not to latest versions they are not backwards compatible, there are just too many code changes needed.

[IzzySoft]

I think working combination is JDK-11 + NDK 21.4.7075529 + Gradle 6.5 + <???>, at least it compiles from command line this way

Funny. Your gradle-wrapper.properties file specifies Gradle 7.0 – but OK, at the tag it's still 6.5, right. So OK, I can try specifying NDK 21.4.7075529 explicitly and use OpenJDK-11. Running now… Oh, looking better it seems – building things. Let's see if it gets through… lots of (c)makes… Build succeeds! The only change on my end was making the NDK version explicit. Well, not the first time that did the trick:

BUILD SUCCESSFUL in 5m 47s
92 actionable tasks: 92 executed

Now let's see if it's RB (need to run again as I didn't have the resulting outfile name and thus couldn't catch it out of its container)… Unfortunately not: all the libraries differ, so does the classes.dex and one of the PNGs:

  -rw----     2.4 fat       24 b-       19 defN 1980-00-00 00:00:00 9accf7c0 META-INF/window_release.kotlin_module
- -rw----     2.4 fat  8716304 b-  3292869 defN 1980-00-00 00:00:00 f5a492d2 classes.dex
+ -rw----     2.4 fat  8716420 b-  3292899 defN 1980-00-00 00:00:00 84ec4782 classes.dex
  -rw----     2.4 fat      204 b-      171 defN 1980-00-00 00:00:00 230d79dd kotlin/ArithmeticException.kotlin_metadata
  -rw----     2.4 fat      135 b-      118 defN 1980-00-00 00:00:00 86a4ca4e kotlin/AssertionError.kotlin_metadata
  -rw----     2.4 fat      443 b-      306 defN 1980-00-00 00:00:00 aec5a097 kotlin/BuilderInference.kotlin_metadata
@@ -386,22 +384,22 @@
  -rw----     2.4 fat      356 b-      269 defN 1980-00-00 00:00:00 4ed3b1cf kotlin/time/TimeSource.kotlin_metadata
  -rw----     2.4 fat      760 b-      423 defN 1980-00-00 00:00:00 eb779507 kotlin/time/TimeSourceKt.kotlin_metadata
  -rw----     2.4 fat      492 b-      355 defN 1980-00-00 00:00:00 8b0e8e2d kotlin/time/TimedValue.kotlin_metadata
- -rw----     2.4 fat    15696 b-     5085 defN 1980-00-00 00:00:00 3d9d3d62 lib/arm64-v8a/libCodec2JNI.so
- -rw----     2.4 fat     6576 b-     2348 defN 1980-00-00 00:00:00 ee49a32b lib/arm64-v8a/libOpusJNI.so
- -rw----     2.4 fat  1412648 b-   724242 defN 1980-00-00 00:00:00 cf9facae lib/arm64-v8a/libcodec2.so
- -rw----     2.4 fat   548544 b-   230551 defN 1980-00-00 00:00:00 5e5a2111 lib/arm64-v8a/libopus.so
- -rw----     2.4 fat     9576 b-     4045 defN 1980-00-00 00:00:00 0f8ea133 lib/armeabi-v7a/libCodec2JNI.so
- -rw----     2.4 fat     4480 b-     2093 defN 1980-00-00 00:00:00 29e7d271 lib/armeabi-v7a/libOpusJNI.so
- -rw----     2.4 fat  1334316 b-   708185 defN 1980-00-00 00:00:00 a6076eb4 lib/armeabi-v7a/libcodec2.so
- -rw----     2.4 fat   430420 b-   219439 defN 1980-00-00 00:00:00 e22d07ec lib/armeabi-v7a/libopus.so
- -rw----     2.4 fat    12232 b-     4968 defN 1980-00-00 00:00:00 50d54531 lib/x86/libCodec2JNI.so
- -rw----     2.4 fat     5024 b-     2261 defN 1980-00-00 00:00:00 f4f1e457 lib/x86/libOpusJNI.so
- -rw----     2.4 fat  1435540 b-   729455 defN 1980-00-00 00:00:00 611352e3 lib/x86/libcodec2.so
- -rw----     2.4 fat   623344 b-   240259 defN 1980-00-00 00:00:00 672bc768 lib/x86/libopus.so
- -rw----     2.4 fat    16368 b-     5628 defN 1980-00-00 00:00:00 7f1f6862 lib/x86_64/libCodec2JNI.so
- -rw----     2.4 fat     6736 b-     2414 defN 1980-00-00 00:00:00 0de8e8ce lib/x86_64/libOpusJNI.so
- -rw----     2.4 fat  1497496 b-   760995 defN 1980-00-00 00:00:00 e16fdf68 lib/x86_64/libcodec2.so
- -rw----     2.4 fat   591688 b-   233263 defN 1980-00-00 00:00:00 34f979ae lib/x86_64/libopus.so
+ -rw----     2.4 fat    19024 b-     5396 defN 1980-00-00 00:00:00 16502f32 lib/arm64-v8a/libCodec2JNI.so
+ -rw----     2.4 fat    10160 b-     2528 defN 1980-00-00 00:00:00 39bf5195 lib/arm64-v8a/libOpusJNI.so
+ -rw----     2.4 fat  1421536 b-   726681 defN 1980-00-00 00:00:00 5d059e75 lib/arm64-v8a/libcodec2.so
+ -rw----     2.4 fat   587744 b-   256009 defN 1980-00-00 00:00:00 4412622d lib/arm64-v8a/libopus.so
+ -rw----     2.4 fat    18392 b-     8830 defN 1980-00-00 00:00:00 2a34c7a0 lib/armeabi-v7a/libCodec2JNI.so
+ -rw----     2.4 fat    13960 b-     6948 defN 1980-00-00 00:00:00 2d9f6740 lib/armeabi-v7a/libOpusJNI.so
+ -rw----     2.4 fat  1339040 b-   708410 defN 1980-00-00 00:00:00 bffebe7f lib/armeabi-v7a/libcodec2.so
+ -rw----     2.4 fat   444176 b-   228161 defN 1980-00-00 00:00:00 eafa4e71 lib/armeabi-v7a/libopus.so
+ -rw----     2.4 fat    14188 b-     5117 defN 1980-00-00 00:00:00 81920ee9 lib/x86/libCodec2JNI.so
+ -rw----     2.4 fat     5660 b-     2399 defN 1980-00-00 00:00:00 70a61664 lib/x86/libOpusJNI.so
+ -rw----     2.4 fat  1449568 b-   733651 defN 1980-00-00 00:00:00 84f517ee lib/x86/libcodec2.so
+ -rw----     2.4 fat   653008 b-   252400 defN 1980-00-00 00:00:00 dee27084 lib/x86/libopus.so
+ -rw----     2.4 fat    19296 b-     5915 defN 1980-00-00 00:00:00 1111e7bf lib/x86_64/libCodec2JNI.so
+ -rw----     2.4 fat    10432 b-     2617 defN 1980-00-00 00:00:00 10438f89 lib/x86_64/libOpusJNI.so
+ -rw----     2.4 fat  1499688 b-   759529 defN 1980-00-00 00:00:00 cf939f67 lib/x86_64/libcodec2.so
+ -rw----     2.4 fat   608496 b-   253713 defN 1980-00-00 00:00:00 ecc0bbfc lib/x86_64/libopus.so
  -rw----     0.0 fat      616 b-      299 defN 1980-00-00 00:00:00 c605ed21 res/anim-v21/design_bottom_sheet_slide_in.xml
  -rw----     0.0 fat      616 b-      298 defN 1980-00-00 00:00:00 75f51489 res/anim-v21/design_bottom_sheet_slide_out.xml
  -rw----     0.0 fat      364 b-      218 defN 1980-00-00 00:00:00 3931a311 res/anim-v21/fragment_fast_out_extra_slow_in.xml
@@ -547,7 +545,7 @@
  -rw----     0.0 fat      872 b-      497 defN 1980-00-00 00:00:00 d7570dff res/drawable-anydpi-v21/ic_app_action_disconnected.xml
  -rw----     0.0 fat     5696 b-      982 defN 1980-00-00 00:00:00 0c9d0923 res/drawable-anydpi-v21/ic_launcher_background.xml
  -rw----     0.0 fat     1056 b-      563 defN 1980-00-00 00:00:00 a84f87a8 res/drawable-anydpi-v24/ic_app_action.xml
- -rw----     2.4 fat      272 bx      272 stor 1980-00-00 00:00:00 e0e00b8a res/drawable-hdpi-v4/abc_ab_share_pack_mtrl_alpha.9.png
+ -rw----     2.4 fat      272 b-      272 stor 1980-00-00 00:00:00 e0e00b8a res/drawable-hdpi-v4/abc_ab_share_pack_mtrl_alpha.9.png
  -rw----     2.4 fat      227 bx      227 stor 1980-00-00 00:00:00 55e57039 res/drawable-hdpi-v4/abc_btn_check_to_on_mtrl_000.png

Now wondering for the reason.

• was the APK attached to releases built from (a clean tree at) the commit the tag points to?
• are the .so files generated from a "fixed version" – or might I have gotten a newer version today than you might have built a month ago, when creating that release?
• might also be that build IDs were included in the libs (there's --build-id=none with cmake for that IIRC).

Looking at the dex it seems there was something removed before your APK was built which was still present at the tag (where I built from):

   VISIBILITY_SYSTEM Ldalvik/annotation/InnerClass; accessFlags=25 name="attr"
+  VISIBILITY_RUNTIME Ljava/lang/Deprecated;
+  VISIBILITY_RUNTIME Ljava/lang/Deprecated;
+  VISIBILITY_RUNTIME Ljava/lang/Deprecated;
+  VISIBILITY_RUNTIME Ljava/lang/Deprecated;
+  VISIBILITY_RUNTIME Ljava/lang/Deprecated;

   Class descriptor  : 'Lcom/radio/codec2talkie/R$attr;'
   Access flags      : 0x0011 (PUBLIC FINAL)
@@ -665714,6 +665719,14 @@

   VISIBILITY_SYSTEM Ldalvik/annotation/EnclosingClass; value=Lcom/radio/codec2talkie/R;
   VISIBILITY_SYSTEM Ldalvik/annotation/InnerClass; accessFlags=25 name="styleable"
+  VISIBILITY_RUNTIME Ljava/lang/Deprecated;
+  VISIBILITY_RUNTIME Ljava/lang/Deprecated;
+  VISIBILITY_RUNTIME Ljava/lang/Deprecated;
+  VISIBILITY_RUNTIME Ljava/lang/Deprecated;
+  VISIBILITY_RUNTIME Ljava/lang/Deprecated;
+  VISIBILITY_RUNTIME Ljava/lang/Deprecated;
+  VISIBILITY_RUNTIME Ljava/lang/Deprecated;
+  VISIBILITY_RUNTIME Ljava/lang/Deprecated;

[sh123]

Your gradle-wrapper.properties file specifies Gradle 7.0 – but OK, at the tag it's still 6.5, right

@IzzySoft , you asked for newer gradle, I've upgraded yesterday, also pumped up SDK version to 31 and made some other changes (check git log), did not make any releases with that, if you want to replicate older build then you need to get code published together with the APK release or clone from release tag and build from it. APK published here to project are built from my local working master, might be not fully clean cloned copy, not sure it could cause any difference.

might also be that build IDs were included in the libs (there's --build-id=none with cmake for that IIRC).

Not sure about that, not sure this flag is set, should it be set? To which value?

I've also published version 1.73 from master, you can also cross check it against master inside your build environment.

[IzzySoft]

if you want to replicate older build

No need from our end for that (as we weren't able to do that yet). If we'd need that, we'd use what's available at the corresponding tag.

APK published here to project are built from my local working master, might be not fully clean cloned copy

That could indeed be the cause of the issue – see the "first basic rule" in our hints on reproducible builds.

Not sure about that, not sure this flag is set, should it be set?

I've used Github's search to find out, but found no hint. Would definitely be worth a try I guess. Value as pointed out above: --build-id=none (so value is "none", i.e. "do NOT include it).

PS: Maybe you could link the IoD shield in the Readme to your app at IzzyOnDroid instead of to its image?

I've also published version 1.73 from master, you can also cross check it against master inside your build environment.

Does not build at all:

> Task :libcodec2-android:buildCMakeRelWithDebInfo FAILED
C/C++: ninja: error: '/build/repo/libcodec2-android/build/imported-lib/armeabi-v7a/libcodec2.so', needed by '/build/repo/libcodec2-android/build/intermediates/cxx/RelWithDebInfo/6t2rc5q5/obj/armeabi-v7a/libCodec2JNI.so', missing and no known rule to make it

> Task :codec2talkie:processReleaseManifestForPackage

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':libcodec2-android:buildCMakeRelWithDebInfo'.
> Build command failed.
  Error while executing process /opt/sdk/cmake/3.10.2.4988404/bin/ninja with arguments {-C /build/repo/libcodec2-android/.cxx/RelWithDebInfo/6t2rc5q5/armeabi-v7a Codec2JNI}
  ninja: Entering directory `/build/repo/libcodec2-android/.cxx/RelWithDebInfo/6t2rc5q5/armeabi-v7a'

  ninja: error: '/build/repo/libcodec2-android/build/imported-lib/armeabi-v7a/libcodec2.so', needed by '/build/repo/libcodec2-android/build/intermediates/cxx/RelWithDebInfo/6t2rc5q5/obj/armeabi-v7a/libCodec2JNI.so', missing and no known rule to make it

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 1m 19s
32 actionable tasks: 32 executed

[sh123]

PS: Maybe you could link the IoD shield in the Readme to your app at IzzyOnDroid instead of to its image?

Do you have link to instruction how to add it?

Does not build at all:

Added fix in master, error is gone for me.

I've used Github's search to find out, but found no hint. Would definitely be worth a try I guess. Value as pointed out above: --build-id=none (so value is "none", i.e. "do NOT include it).

Where this should be added, is there equivalent variable in gradle script or it should be done in cmake or .. ? What's the value instead of none?

[IzzySoft]

Whatever happened to your signing key? Today's update was rejected at IoD because the key doesn't match:

2024-10-17 19:50:06,730 WARNING: "com.radio.codec2talkie_173.apk" is signed by a key that is not allowed:
b065a941bd9a5ac5ae50d02d6e6bd0d126a85517351cbd9e15e8edcb4ae3a59e

Signing keys are pinned at IoD, so you cannot simply change them (also, changing them makes seamless updates impossible but requires uninstall/reinstall).

Do you have link to instruction how to add it?

How to add a link?

[<img src='https://img.shields.io/endpoint?url=https://apt.izzysoft.de/fdroid/api/v1/shield/com.radio.codec2talkie'>](https://apt.izzysoft.de/packages/com.radio.codec2talkie)

Added fix in master, error is gone for me.

I will check again once the signing issue is solved. Would need an APK from you then (renamed to *.zip and attached here is fine) and the commit it was built from.

Where this should be added, is there equivalent variable in gradle script or it should be done in cmake or .. ? What's the value instead of none?

No other value you should use there (idk what is the default, but it often uses "sha1"). For instructions, see e.g. here.

[sh123]

Whatever happened to your signing key? Today's update was rejected at IoD because the key doesn't match:

Yes, I was using new signing key, had to regenerate it after upgrades on devenv, but it is self signed anyway, can it be updated on your side?

How to add a link?

Updated in README.md, should have correct link.

I will check again once the signing issue is solved. Would need an APK from you then (renamed to *.zip and attached here is fine) and the commit it was built from.

Built from cc2867e49cfc542c7236f99270a8a69ef248f587 with command ANDROID_HOME=~/Android/Sdk JAVA_HOME=~/.jdks/jbr-21.0.4 ./gradlew assembleRelease

codec2talkie-release-unsigned.zip

This one is signed from android studio.

codec2talkie-release.zip

[IzzySoft]

Yes, I was using new signing key, had to regenerate it after upgrades on devenv, but it is self signed anyway, can it be updated on your side?

Tell me an app that's not "self signed" – that's not the point, see e.g. How to keep your key safe and what measures to take for the event of loss? Do you still have the original key? If not, we have a problem. To switch to a different key we need to confirm it. You do not sign your commits (I strongly recommend starting that), there's nobody "registered" who can vouch for you, and no alternative way to contact you has been established. So if someone "hacked" your Github account pretending it's you, we have no way to verify – as outlined by the linked article.

So: do you still have the original key? Why did you have to replace it, was it no longer accepted? Upgrading the environment to my knowledge does not render a key unusable.

Updated in README.md, should have correct link.

Thanks, looks good!

Built from cc2867e49cfc542c7236f99270a8a69ef248f587

Thanks! Will check once we got the signing issue tackled.

[sh123]

Sorry, it becomes too complicated and time consuming for me, I do not have that much free time to deal with all that maintenance, framework updates work and now InfoSec scenarios, I suggest you to remove this app from Izzy, that's amateur app and I see that it is not widely used and popular as nobody ever contributed single PR here and people who really need it can build/fork on their own.

[IzzySoft]

That would be a sad result. You still didn't say if you still have the original key – because if you have, we can either move to the new one (using the original to confirm), or switch back to the original one (and avoid having the uninstall/reinstall for the move). Before I outline the (rather simple) way to do that, can you please confirm the key would still be available to you, and you could use it (at least once) for signing?

[sh123]

No I do not have it after setting up new dev env, upgrades and performing clean build, so I had to regenerate new one in another location, it was self-signed so I did not pay attention that it could be used elsewhere.

[sh123]

I'm closing it as build issue is resolved and I've also added --build-id=none for the latest build.

[IzzySoft]

I'm still pondering what we can do about the signing issue, without being able to "verify". You know what signing is about, do you?

had to regenerate it after upgrades on devenv

I wonder what made you think that. The keystore should not have been affected by upgrades or whatever – it's just an "encrypted file" holding the key(s). And as signing is used a.o. to prevent malicious actors to ship malicious updates for your app, an APK signed with a different key is rejected for security reasons. Even if I'd modify the config for your app here to allow the new key, devices having a previous version installed would (and should!) refuse it – so folks had to uninstall and re-install. While that cannot be helped if the previous key was lost, it should at least be made sure not to happen again: mistakes happen, but we must learn from them. Hence my question if you understand what signing is for and what it does (in principle), and that you can carry a keystore over to an updated/moved/changed environment.

[sh123]

That's malicious that I have to worry about all that and explain something while my intention is to store it as source code here on github if I will need to publish public apk when app will be mature enough I can do it through google store, right? As I wrote before, you are free to remove this app from your package manager, myself I'm not using 3rd party Android package managers for security concerns you explained above.

[IzzySoft]

I can do it through google store, right?

You assume everyone is using that. And everyone is happy with Google's tracking. Yes, you can publish it there – but that will not make it available to those having cut their ties to the Google ecosystem (like me for example: I don't even have a Google account anymore, for about 10 years now – and without an account, you cannot get any app there). That's where FOSS solutions like IzzyOnDroid (and F-Droid) come in, offering a solution without tracking, without needing an account. Would be a sad world where a FOSS app must use a proprietary distributor, which in addition intransparently ships so many tracker-ridden apps that you hardly can be sure what you get.

And don't think uploading new releases to Google will always go easy. I've heard enough complaints about quite stupid demands – like blocking them as they want login credentials for an offline app not even having the internet permission or using any accounts (don't believe me? See e.g. Your app is not compliant with Google Play Policies: A story from hell). Apart from that, you're aware of the demands Google has nowadays towards developers wanting to publish there? Like your full address and phone number and, if I remember correctly, even a copy of your ID card? You're happy with that? Devs start pulling their apps from Play already as they are not willing to give Google those details. Apart from that: ever tried to reach a human at Playstore support? Good luck. I do not remember having heard from any developer managing that. Just a lot of fighting with "robot mails". Try explaining yourself to a robot… Think of A.I. there? Right: All Insane :see_no_evil:

I'm not using 3rd party Android package managers for security concerns you explained above.

Ah. Now guess why we have security measures like this in place :wink: Want a short reading? See e.g. Ramping up security: additional APK checks are in place with the IzzyOnDroid repo – or the short summary on the repo's help page. It's not just "lazy side-loading". There's a lot of work done on our (the repo maintainers') ends. Including the RB stuff we were dealing with above. We're doing our best to keep our repos as safe and secure as possible. Why else might I have asked for verification when your key changed? :wink:

I won't push or even force you, the decision is of course yours. If you indeed want us to remove your app here, we'll comply. But think it over thoroughly and be sure you want that :wink:

[sh123]

I won't push or even force you, the decision is of course yours. If you indeed want us to remove your app here, we'll comply. But think it over thoroughly and be sure you want that 😉

Yes, I want this app to be removed from Izzy, please, comply. Thanks.

[IzzySoft]

Done, effective with the next sync. No bad feelings of course, wishing you the best for your projects as well as personally! Thanks for all the efforts you've put into this – even if the outcome was not what we've hoped for…