Publish on F-Droid - Githubissues

Integral-Tech commented 2 weeks ago

It would be better if ServerBox is submitted to F-Droid.

lollipopkit commented 2 weeks ago

That's a good idea. Could you please help me submit this app to F-Droid, or should I do it myself?

Integral-Tech commented 2 weeks ago

I have submitted an issue to request for packaging (https://gitlab.com/fdroid/rfp/-/issues/2780) Your main git submission email is used for filling AuthorEmail

lollipopkit commented 2 weeks ago

Thank you for your efforts. :) I have read the doc and it states that a Fastlane config should be created inside this repo. I would like to get more people to contribute, so could you open a PR for that?

Integral-Tech commented 2 weeks ago

Thank you for your efforts. :) I have read the doc and it states that a Fastlane config should be created inside this repo. I would like to get more people to contribute, so could you open a PR for that?

Yes, I have opened a pull request

Integral-Tech commented 2 weeks ago

In the RFP issue, the bot detected non-free dependencies (seems to be GMS-related), which should be removed:

com.google.android.c2dm.permission.RECEIVE

There are also some dependencies / services that could be used to track users, which should be minimized:

ly.count.android.sdk.messaging.CountlyPushActivity
ly.count.android.sdk.ReferrerReceiver
https://firebase.google.com/support/privacy/init-options
https://plus.google.com/

lollipopkit commented 2 weeks ago

It's related to #379, i will solve it later.

licaon-kter commented 2 weeks ago

trying to build https://github.com/lollipopkit/flutter_server_box/commit/64ce3638cb8942499800875876d3a6fbe657516d

these proprietary libs are still pulled in via https://github.com/Countly/countly-sdk-flutter-bridge/blob/24.4.1/android/build.gradle#L38 and https://github.com/Rexios80/watch_connectivity/blob/watch_connectivity-0.2.0/watch_connectivity/android/build.gradle#L52

lollipopkit commented 2 weeks ago

trying to build 64ce363

these proprietary libs are still pulled in via https://github.com/Countly/countly-sdk-flutter-bridge/blob/24.4.1/android/build.gradle#L38 and https://github.com/Rexios80/watch_connectivity/blob/watch_connectivity-0.2.0/watch_connectivity/android/build.gradle#L52

Thanks for the tip.

licaon-kter commented 2 weeks ago

rebuilt from 5059872c3f7343054b702de8b629d53b303a8aad

2024-06-11 15:30:19,612 INFO: Successfully built version 1.0.963 of tech.lolli.toolbox from 5059872c3f7343054b702de8b629d53b303a8aad
2024-06-11 15:30:29,778 DEBUG: Checking build/tech.lolli.toolbox/build/app/outputs/flutter-apk/app-release.apk
2024-06-11 15:30:29,835 INFO: Scanning APK with dexdump for known non-free classes.
2024-06-11 15:30:29,951 DEBUG: > /opt/android-sdk/build-tools/33.0.0/dexdump /tmp/tmp2z8b7fta/classes.dex
2024-06-11 15:30:35,878 DEBUG: Problem: found class 'com/google/android/play/core/splitinstall/SplitInstallSessionState'
2024-06-11 15:30:35,901 DEBUG: Problem: found class 'com/google/android/play/core/splitinstall/SplitInstallStateUpdatedListener'
2024-06-11 15:30:35,927 DEBUG: Problem: found class 'com/google/android/play/core/tasks/OnSuccessListener'
2024-06-11 15:30:35,947 DEBUG: Problem: found class 'com/google/android/play/core/tasks/OnFailureListener'
2024-06-11 15:30:35,952 DEBUG: Problem: found class 'com/google/android/play/core/splitcompat/SplitCompatApplication'
2024-06-11 15:30:36,105 DEBUG: Problem: found class 'com/google/android/play/core/splitinstall/SplitInstallManager'
2024-06-11 15:30:36,118 CRITICAL: Found 6 problems in build/tech.lolli.toolbox/build/app/outputs/flutter-apk/app-release.apk
2024-06-11 15:30:36,120 ERROR: Could not build app tech.lolli.toolbox: Found blocklisted packages in final apk!
2024-06-11 15:30:36,120 DEBUG: Error encountered, stopping by user request.

which looks like a flutter bug https://gitlab.com/fdroid/fdroiddata/-/issues/2949

Integral-Tech commented 2 weeks ago

rebuilt from 5059872

2024-06-11 15:30:19,612 INFO: Successfully built version 1.0.963 of tech.lolli.toolbox from 5059872c3f7343054b702de8b629d53b303a8aad
2024-06-11 15:30:29,778 DEBUG: Checking build/tech.lolli.toolbox/build/app/outputs/flutter-apk/app-release.apk
2024-06-11 15:30:29,835 INFO: Scanning APK with dexdump for known non-free classes.
2024-06-11 15:30:29,951 DEBUG: > /opt/android-sdk/build-tools/33.0.0/dexdump /tmp/tmp2z8b7fta/classes.dex
2024-06-11 15:30:35,878 DEBUG: Problem: found class 'com/google/android/play/core/splitinstall/SplitInstallSessionState'
2024-06-11 15:30:35,901 DEBUG: Problem: found class 'com/google/android/play/core/splitinstall/SplitInstallStateUpdatedListener'
2024-06-11 15:30:35,927 DEBUG: Problem: found class 'com/google/android/play/core/tasks/OnSuccessListener'
2024-06-11 15:30:35,947 DEBUG: Problem: found class 'com/google/android/play/core/tasks/OnFailureListener'
2024-06-11 15:30:35,952 DEBUG: Problem: found class 'com/google/android/play/core/splitcompat/SplitCompatApplication'
2024-06-11 15:30:36,105 DEBUG: Problem: found class 'com/google/android/play/core/splitinstall/SplitInstallManager'
2024-06-11 15:30:36,118 CRITICAL: Found 6 problems in build/tech.lolli.toolbox/build/app/outputs/flutter-apk/app-release.apk
2024-06-11 15:30:36,120 ERROR: Could not build app tech.lolli.toolbox: Found blocklisted packages in final apk!
2024-06-11 15:30:36,120 DEBUG: Error encountered, stopping by user request.

which looks like a flutter bug https://gitlab.com/fdroid/fdroiddata/-/issues/2949

But there are some Flutter apps submitted successfully to F-Droid, such as FluffyChat. How did they solve this problem?

licaon-kter commented 2 weeks ago

Not sure what's their magic sauce :(

linsui commented 2 weeks ago

FluffyChat emptied their proguard-rules.pro.

Integral-Tech commented 2 weeks ago

@licaon-kter I have cleaned proguard-rules.pro. Try to rebuild it :)

licaon-kter commented 2 weeks ago

Builds fine now

@lollipopkit up next, setup Flutter as a submodule (and use it yourself in your CI and build recipes) or at least have the exact version written somewhere

tag and build the APK from the tagged commit so we can test for reproducibility (ref: https://f-droid.org/docs/Inclusion_How-To/#reproducible-builds and https://f-droid.org/2023/09/03/reproducible-builds-signing-keys-and-binary-repos.html)

lollipopkit commented 2 weeks ago

@lollipopkit up next, setup Flutter as a submodule (and use it yourself in your CI and build recipes) or at least have the exact version written somewhere

Thanks, i have two question:

After read the ref, I can’t figure out where(exact version written somewhere) to specify the Flutter version. Is it in pubspec.yaml? If possible, use the latest stable release of Flutter.
If fdroid builds with my sign key, must I write the plain passwd to metadata?

linsui commented 2 weeks ago

If possible, use the latest stable release of Flutter.

We need to pin the version. The latest stable release is changing.

If fdroid builds with my sign key, must I write the plain passwd to metadata?

We don't need your key. Do not send your key to anyone. :)

linsui commented 2 weeks ago

For example, there is the revision of the flutter in https://github.com/lollipopkit/flutter_server_box/blob/main/.metadata. If you keep the file up-to-date we can use the revision specified there.

licaon-kter commented 2 weeks ago

After read the ref, I can’t figure out where(exact version written somewhere) to specify the Flutter version. Is it in pubspec.yaml?

Where you want, eg. some use the CI, say put it here: https://github.com/lollipopkit/flutter_server_box/blob/main/.github/workflows/release.yml#L52 but not stable but 3.22.1

We need to pin the version. The latest stable release is changing.

yes, eg. yesterday was 3.22.1 you tagged (example) a version and built it

F-Droid builds today, and today... surprise 3.22.2 is stable. Maybe it's not an issue between .1 and .2 but there are issues between major versions

licaon-kter commented 2 weeks ago

We don't need your key. Do not send your key to anyone.

exactly, do read those two links :smile:

lollipopkit commented 2 weeks ago

Thank you. I understand.

I still have questions about how to build with my sign key (if possible)? Or to release with fdroid signature (only in fdroid)?

linsui commented 2 weeks ago

We only use your signature if we can make the build reproducible. See https://f-droid.org/docs/Reproducible_Builds/.

licaon-kter commented 2 weeks ago

You already host apks like: https://github.com/lollipopkit/flutter_server_box/releases/tag/v1.0.949

$ apksigner verify --print-certs ServerBox_949_amd64.apk
Signer #1 certificate DN: CN=JunyuanFeng, OU=ToastStudio, O=ToastStudio, L=ChengDu, ST=Sichuan, C=86

so.. you're already setup, build and sign as usual, F-Droid will use your verified reproducible APK :wink:

lollipopkit commented 2 weeks ago

Sorry... I thought the apk released in fdroid would be built in fdroid ci, so was wondering how to sign it.

licaon-kter commented 2 weeks ago

@lollipopkit look at the graph https://f-droid.org/docs/Reproducible_Builds/ it's YES and YES and ends up with "publish @lollipopkit's APK" and not "sign something"

lollipopkit commented 2 weeks ago

It's my problem...
Before you explained it, i thought build reproducible could only be fdroid ci

Now the flutter version is specified in the .github/workflows/release.yml

licaon-kter commented 1 week ago

is there a reason why the Tag name and the versionName don't match? https://github.com/lollipopkit/flutter_server_box/blob/v1.0.967/pubspec.yaml#L4

lollipopkit commented 1 week ago

The version in pubspec.yaml is generated by a script before pushing, but the version in the tag is generated by CI after pushing. So there is a discrepancy.

Is this important? Should I fix it?

linsui commented 1 week ago

Yes, it's important because we need a correct tag name to get your apk.

licaon-kter commented 1 week ago

eg: https://github.com/lollipopkit/flutter_server_box/releases/download/v1.0.967/ServerBox_967_amd64.apk

we can use versionCode and versionName as templates for your APK url

say https://github.com/lollipopkit/flutter_server_box/releases/download/v%v/ServerBox_%c_amd64.apk

but... if those do not match your Tag and are +1 we can't

licaon-kter commented 1 week ago

regarding versionCode of the per arch APKs, please move the arch number at the end so instead of the current 1967/2967/4967 it's better to have 9671/9672/9674

it used to be here https://github.com/flutter/flutter/blob/40e3489254b2d0ffedf840bd9233e5e6d06d09dd/packages/flutter_tools/gradle/flutter.gradle#L749 kinda so instead of abiVersionCode * 1000 + variant.versionCode have it variant.versionCode * 10 + abiVersionCode

not sure where this was moved in 3.22.x

Integral-Tech commented 1 week ago

regarding versionCode of the per arch APKs, please move the arch number at the end so instead of the current 1967/2967/4967 it's better to have 9671/9672/9674

it used to be here https://github.com/flutter/flutter/blob/40e3489254b2d0ffedf840bd9233e5e6d06d09dd/packages/flutter_tools/gradle/flutter.gradle#L749 kinda so instead of abiVersionCode * 1000 + variant.versionCode have it variant.versionCode * 10 + abiVersionCode

not sure where this was moved in 3.22.x

@licaon-kter I have fixed this in https://github.com/lollipopkit/flutter_server_box/pull/391

lollipopkit commented 1 week ago

v1.0.971 solved these two.

licaon-kter commented 1 week ago

Ok, builds, some notes:

the APK names, since versionCode is different per arch, maybe name them based on that instead being 971, as they are 9713 9711 etc, or better yet just remove the 971
it's almost repro, here's the diff log: tech9713.log.7z.zip (remove .ZIP from filename)

@linsui this diff looks familiar?

/LE: recipe updated here: https://gitlab.com/fdroid/rfp/-/issues/2780#note_1946717714

linsui commented 1 week ago

No. @lollipopkit Can you explain how you build your apk?

lollipopkit commented 1 week ago

I built the apks as shown in Actions.

The reason why this diff occurs, may due to (i guess):

Zulu jdk17:https://github.com/lollipopkit/flutter_server_box/blob/108d0a5a5b4a6889ffbc801df4a95142910c49d2/.github/workflows/release.yml#L27 My custom build script: https://github.com/lollipopkit/flutter_server_box/blob/108d0a5a5b4a6889ffbc801df4a95142910c49d2/.github/workflows/release.yml#L34

linsui commented 1 week ago

I want to know what your script does. :) Can we also run

dart run fl_build -p android

to build the apk?

lollipopkit commented 1 week ago

Yes, dart run runs a pkg(bin) named fl_build, it can be found at: https://github.com/lollipopkit/flutter_server_box/blob/108d0a5a5b4a6889ffbc801df4a95142910c49d2/pubspec.yaml#L79

linsui commented 1 week ago

Does it do anything special? How can I build only 1 abi with it?

lollipopkit commented 1 week ago

It updates metadata used by update checker and others. It can't build 1 abi...

I think just use zulu to build, then the diff will decrease much.

linsui commented 1 week ago

Zulu is only used to build the java part but the libapp.so has difference. The BuildData.build is different. In your apk it's 971 and in our apk it's 970.

lollipopkit commented 1 week ago

Ok, i will update it to support only 1 abi.

lollipopkit commented 1 week ago

But it can't completely repro.

The buildAt vary every build time. https://github.com/lollipopkit/flutter_server_box/blob/108d0a5a5b4a6889ffbc801df4a95142910c49d2/lib/data/res/build_data.dart#L7

linsui commented 1 week ago

Why is https://github.com/lollipopkit/flutter_server_box/blob/v1.0.971/lib/data/res/build_data.dart not updated to 971? Does your script update it when building the apk?

lollipopkit commented 1 week ago

Does your script update it when building the apk?

Yes

linsui commented 1 week ago

That's the problem... We need to build the apk from the same code. If it update the code everytime we can't get the same apk.

lollipopkit commented 1 week ago

Can i delete buildAt but leave build?

We both use fl_build, so the build would be the same.

linsui commented 1 week ago

I thought it should work.

lollipopkit commented 1 week ago

Now the build script will output apk at build/app/outputs/flutter-apk/app-release.apk

linsui commented 1 week ago

How should we build, e.g., an arm64 apk with fl_build?

lollipopkit commented 1 week ago

dart run fl_build -p android -ooa

lollipopkit / flutter_server_box

Publish on F-Droid #381