F-Droid inclusion - Githubissues

IzzySoft commented 2 years ago

Someone anonymously requested to have your app included with F-Droid – but then magically disappeared. So as one of the F-Droid maintainers, I'm carrying this request over to you:

are you OK with having your app included?
if yes: could you please establish Fastlane structures here to provide metadata (description, screenshots etc)? (Note the very same can also be used for PlayStore and other places)

Thanks in advance – and of course you're welcome to chime in to the linked issue directly :wink:

DonovanMaillard commented 2 years ago

Not planned yet

camillemonchicourt commented 2 years ago

It should be discussed, as I think it is a good idea to have Occtax-mobile on F-droid.

IzzySoft commented 2 years ago

Thanks @camillemonchicourt! I've set that request on our end to wait for your decision, and put a reminder on top to check up with you again in about a month (in case notifications from this issue here, which is linked from there, should be missed).

IzzySoft commented 2 years ago

Hi @camillemonchicourt, just wanted to see if any decision has been made? The RFP is now entering the closure phase and will be closed due to inactivity in 4 weeks – unless there is activity of course :wink: So your help is much appreciated.

camillemonchicourt commented 2 years ago

Thanks, I started a first fastlane structure : https://github.com/PnX-SI/gn_mobile_occtax/pull/157 Will try to complete it asap.

IzzySoft commented 2 years ago

Thanks! A few notes here as I seem unable to comment there:

please do not add empty files, rather leave them out entirely
full_description.txt and short_description.txt in en-US MUST NOT be empty: everything in en-US is used as fallback for languages missing it. So leaving the two empty here would make the app appearing without summary/description in those languages.
I cannot preview the images nor do I get any details on them – except for the fact their size suggests they are empty, too. Please check with my short documentation of Fastlane structures to make sure the sizes fit (eg. icons must be square and between 48x48 and 512x512 px, featureGraphic either 512x250 px or 1024x500 px, and so on)
if you format the full description the way you did the changelogs, they should be fine :wink:

Feel free to ping me for another review when you think it's ready. I meanwhile edited the RFP, cancelling the closure phase and moving up the due date for another month to give you some time with things here. If you need assistance, just let us know!

camillemonchicourt commented 2 years ago

Yes, I checked your very well done documentation. It is not finished yet, and I will check why there was a problme with screenshots. Thanks a lot for the help.

IzzySoft commented 2 years ago

As the due date of the RFP has been reached again: any progress, @camillemonchicourt? If there are any issues with the screenshots, maaybe you could at least add short_description.txt and full_description.txt for en-US (this would be the bare minimum to get it working) so we can process the RFP? Screenshots could then be added later.

I'll move the due-date now for a last time (cannot continue that forever). Would be really nice if we get this tackled. Thanks in advance!

camillemonchicourt commented 2 years ago

OK, I couldn't work on it earlier in due time. But I just added the minimal information and it should be OK now. I have no problem with screenshots preview.

If it is OK for you in my PR (https://github.com/PnX-SI/gn_mobile_occtax/pull/157/files), I will merge it in master in order to process it.

DonovanMaillard commented 2 years ago

Thank you @camillemonchicourt for this PR.

Screenshots will be updated on next releases if necessary because of frontend improvements under development.

IzzySoft commented 2 years ago

Thanks @camillemonchicourt – that looks good! A few short remarks, though:

it seems like en-US/short_description.txt is 2 chars to long, so the last 2 chars would be truncated. As it's clear this is an Android app (what else is listed at F-Droid :smile:), may I suggest making that simply "collect biodiversity data in GeoNature Occtax module"? All else is rather self-evident :wink: Similar for the French one, please (which otherwise would be cut right after "module" and thus missing essential details).
changelogs: remember those text files are limited to 500 chars (and will be truncated after that). Your most recent changelog is almost twice that size, and would be truncated in the last line of the first block. Not a big deal, and definitely no stopper – just to keep in mind for future releases
fr-FR/full_description.txt is an empty file – bad idea, as that means French users don't see any description. Better remove that file entirely, so the fall-back to en-US takes place
same for all other files you have no language-specific variant for – so no need to duplicate icon and screenshots, just keep them in en-US and they will be shown for all missing localizations.

@DonovanMaillard a note on updating screenshots: due to a current bug in fdroidserver, take care to re-use existing names (otherwise the old images remain and you cannot remove them). So always keep using existing names; currently you cannot remove any existing screenshots, only overwrite them or add more. Apologies for the inconvenience!

That said: Looking forward to the merge. Please drop us a note (either here or at the RFP) so we know we can pick up again. Thanks!

camillemonchicourt commented 2 years ago

OK thanks for the feedbacks.

I reduced the short_description.txt as suggested
I reduced changelogs to new features only
I filled fr-FR/full_description.txt
I removed screenshots and incons in FR as they are the same as you saw

I merged the PR in master so everything should be good. Thanks for helping and waiting and keep us updated.

IzzySoft commented 2 years ago

Gladly– and thanks to you for the fast fixes! Pointer: should a changelog get "too long", you can always let it end with a "link to more" (before it reaches the 500 chars) so no details are lost.

I'll now update the RFP stating Fastlane is available. In order to pick up from here, we have 2 options:

there's a new tag/release soon covering Fastlane: then we preferably wait for that to establish the first build
there's no ETA yet for that – but it's fine to pick the commit adding Fastlane (7cadd390f190ebbfa756f664514c14aa65761763) and use versionName + versionCode of the last tag/release, if that's OK

Oops, looking for those details I find they are missing, as you use dynamic versioning. That doesn't work with F-Droid's checkupdates, which needs literal values here (so no variables or function calls). I see we could fetch versionCode from here – but where to find versionName? If there's no such place yet, maybe it could be added as VERSION_NAME= in the same file? Or ideally, "plain values" could be used in build.gradle?

IzzySoft commented 2 years ago

PS: as an interim, check this link in about an hour – which is when the next sync of my repo should be through. I've meanwhile added your app there (updates will become available within 24h of your tagging a new release and attaching the proper APK to it), as it still will take a little until F-Droid can provide it (the last gap closed, it still needs to have its metadata created, the build succeed and a final review being made). Once it's at F-Droid, I usually remove the app from my repo after an overlap of about 10 days, unless explicitly requested otherwise.

camillemonchicourt commented 2 years ago

OK thanks for the interim solution. No release scheduled very soon but one planned in the coming weeks.

Can't help about version code and version name. Maybe the main developper @sgrimault has an idea of what could be done.

Thanks.

sgrimault commented 2 years ago

Hello @IzzySoft,

The name of the application version is defined by the version variable declared in the build.gradle. This variable is modified by hand at each release of the application. The version code is modified automatically at each build of the application. You can find it in the version.properties file.

About the min required version of this app, the build.gradle is outdated (API 21 -> Android 5.0). Please consider that the min required Android version is Android 8 (API 26). The latest versions of this application have lost compatibility with older versions of Android.

IzzySoft commented 2 years ago

The version code is modified automatically at each build of the application.

That won't work with F-Droid then. We need to know both values in advance, else it fails. Our checkupdates compares the versionCode of our latest build against the versionCode declared on your end for the latest tag and only starts the build process if there's a newer version available at your end. If that can only be found out after having built the version, we'd need to check out and build thousands of repos each day – which not only would be a total waste of resources, but also beyond our capabilities.

So I hope I misunderstood what you were meaning. Let me set up some assumptions and please tell me if they're correct:

I check out code from a given tag
I pick versionName from build.gradle
I pick versionCode from version.properties
Now I start a build from the checked-out tag
the resulting APK file shows the very same versionName and versionCode I evaluated before the build

If that's the case, we can get checkupdates working:

UpdateCheckData: occtax/version.properties|VERSION_CODE=(\d+)|occtax/build.gradle|version\s=\s"(.+?)"

But if the build itself modifies those values, we cannot.

sgrimault commented 2 years ago

OK, I understand that. I need to modify the build process slightly to manually set the version code as for the version name.

IzzySoft commented 2 years ago

Thanks! If you let me know then, I'll again transfer that over. IIRC that was the last open item and we then can perform the next steps.

DonovanMaillard commented 2 years ago

Thank you,

@sgrimault when do you think you will can do this changes, and do you need a new specific command for those modifications ?

sgrimault commented 2 years ago

I plan to review this the next week. @DonovanMaillard, no need to make an order for that :)

sgrimault commented 2 years ago

@camillemonchicourt, @DonovanMaillard, I would suggest you to follow Fastlane/F-Droid directory/file structure as follow :

/ root
├─ fastlane
│  └─ android
│     └─ metadata
│        ├─ en-US
│        ├─ fr-FR
│        ├─ ...
├─ ...
├─ occtax
├─ ...

The metadata directory can evoke several things, whereas placing it in the fastlane directory seems more meaningful to me :blush:

sgrimault commented 2 years ago

@IzzySoft, The master branch has been updated to reflect these changes: The version code is no longer calculated automatically and is set manually in the build file occtax/build.gradle.

IzzySoft commented 2 years ago

@sgrimault that's great news, thanks! Any ETA for the next tag – or should we start with the corresponding/latest commit, i.e. 8619326169ac9cb108886ae145963cd6c3b2c0bb then?

IzzySoft commented 2 years ago

PS: I've set up metadata if someone wants to give the merge request at fdroiddata a chance.

camillemonchicourt commented 2 years ago

OK @sgrimault, several directory organisation were possible and I thought that the metadata one was the most generic and understandable. But if it can bring some confusion, yes we could change it for a more specific one (fastlane/android/metadat)

IzzySoft commented 2 years ago

@camillemonchicourt both structures are identical; the one you've chosen is just skipping 2 directory levels (take fastlane and remove the leading /fastlane and the middle /android level, and you get what you have) and mostly represents what F-Droid uses internally. Fastlane is more wide-spread and used in many places, so one could indeed say it's more "common".

camillemonchicourt commented 2 years ago

I did the PR to reorganize the Fastlane folders structure - https://github.com/PnX-SI/gn_mobile_occtax/pull/173

sgrimault commented 2 years ago

@IzzySoft, Is that possible for you to wait a little for the incoming release (in a few days I think) ?

IzzySoft commented 2 years ago

@sgrimault will you send us a heads-up when it is ready from your side?

sgrimault commented 2 years ago

Hello @IzzySoft,

I have published a release candidate last week and I'm waiting a validation from @DonovanMaillard or @camillemonchicourt to build and publish an official version.

sgrimault commented 2 years ago

Hello @IzzySoft,

The last release has been published. You can find the details here : https://github.com/PnX-SI/gn_mobile_occtax/releases/tag/2.3.0. Sorry again for the delay, time on our side to validate everything before releasing this version.

IzzySoft commented 2 years ago

Thanks! I've updated the RFP accordingly and created the corresponding merge request. Please check once the pipelines are through and assist me should there be any issues with the build, as I'm not a developer and thus might not be able to "fix things".

IzzySoft commented 2 years ago

OK, we got the build succeeding. But I'm afraid noone at F-Droid can test the APK for functionality. Could you please pick the APK from the end of this comment, sign it and see if it works as intended? We could then do the remaining review tasks – but that would do no good if the build produced something "broken" :wink: Thanks!

sgrimault commented 2 years ago

@IzzySoft,

adb install fr.geonature.occtax2_3070.apk 
Performing Streamed Install
adb: failed to install fr.geonature.occtax2_3070.apk: Failure [INSTALL_PARSE_FAILED_NO_CERTIFICATES: Failed to collect certificates from /data/app/vmdl320419390.tmp/base.apk: Attempt to get length of null array]

Seems like you don't sign the apk.

IzzySoft commented 2 years ago

Read again: that's why I wrote you need to sign it :smile: That's just the unsigned APK our CI build produced. Signing follows in a separate step (manually, on an air-gapped server, for security reasons – that way the keys should be safe against leaving our system) after the "real build" on our build servers.

sgrimault commented 2 years ago

Sorry :) This is the signed version of your APK: fr.geonature.occtax.zip I used the same keystore for the "official" releases published directly from project's github and to keep compatibility between versions published from F-Droid repo and those published from github. So a version of the application installed from F-Droid can also be updated directly via the GeoNature instance.

I Think you should use the same keystore to maintain this compatibility. @IzzySoft, I can send you this keystore and let you configure the sign part when building the APK from your CI.

I made some tests from this version, I configured the GeoNature server (https://demo.geonature.fr) and perform synchronization data successfully. @camillemonchicourt or @DonovanMaillard if you have time to install and perform some tests with this version :) Thank you

IzzySoft commented 2 years ago

@sgrimault Thanks! For us it was the last paragraph ("tested successfully, works as expected") which was needed – not the signed APK of our CI be made available (which will be done by us once the MR was merged – I've marked it for review now and left a comment that functionality has been confirmed by you).

Oh: And we use our own keystore, thanks. That means, for each app a dedicated keystore is created automatically and used during its lifetime. That keystore is kept on the air-gapped server I mentioned. When needed, we also can set up a "shared keystore" (if two apps of the same developer need to be signed by the same key for certain interactions), but that's rather the exception.

PS: I'm not yet closing the issue as inclusion is not yet through. If no more issues arise (I doubt any will, but let's wait for the review), you can close it once the app showed up in the catalog, should I forget to do so.

licaon-kter commented 2 years ago

@sgrimault it's nice that you signed it with your key, but F-Droid will sign it with its own when building/hosting.

If the app can be build reproducible F-Droid can include your APK signed by you instead, but I'm not sure if this app can be build like that.

Regarding the map feature, this depends on a certain service (eg. mapbox, openstreetmap) and the user can choose or how does this work?

sgrimault commented 2 years ago

This application can update itself from the configured GeoNature instance. It can check if there is a more recent version or not at startup. If the user can also install the application from F-Droid, it is necessary to keep the possibility that it can be updated directly via the GeoNature instance and for that, it is necessary to use the same configuration when signing the APK. Currently the application is distributed via releases published on the project's github. Then the administrators of the GeoNature instances configure the availability of the application for automatic update.

IzzySoft commented 2 years ago

If the user can also install the application from F-Droid, it is necessary to keep the possibility that it can be updated directly via the GeoNature instance and for that, it is necessary to use the same configuration when signing the APK.

That is only possible when using reproducible builds – else, update would fail due to signature mismatch. So I basically see 2 possibilities:

ensure the app can be built reproducible
have a special build-flavor without that update functionality (as it would only cause frustration)

If none of these are possible, the update check should be made configurable so the user can turn it off – and it should only update the app itself when the signature matches and the user decided such. That might require NonFreeNet – but so might some of the servers contacted for other reasons (OSM & Wikimedia are fine, but I'm not sure about Mapbox, Thunder & Co). Basically, you could also have it check the signature first and skip the check if it's not yours.

sgrimault commented 2 years ago

I agree with you and in my opinion the easiest way is to be able to activate or not the automatic update of the application via GeoNature by configuration. At the moment, this point implies to slightly revise the configuration on the GeoNature side and to revise the update of the configuration on the application side. Another point might be to abandon the way of distributing and updating the application via GeoNature in favour of F-Droid ? @DonovanMaillard , @camillemonchicourt ?

DonovanMaillard commented 2 years ago

@sgrimault think it is complicated to abandon updates driven by geonature and to let this job to f-droid due to compatibilities between geonature versions and occtax-mobile versions. We can't Know who use which version of geonature and in the worst cases a user can connect to several geonature servers... if we would adopt a such update process, we Will have to warranty a greatest retro-compatibility across geonature and mobile versions. Administrators will no have longer control of which mobile version they want to purpose to their users according to their geonature release...

Tell me if I make mistakes of if there are others capabilities...

sgrimault commented 2 years ago

Oh yes, right... I forgot that there can be incompatibilities between versions of GeoNature. This makes it more complicated because there is no strong correspondence between the versions of GeoNature and Occtax. If the application is made available to all users on the F-Droid store, then it is imperative that the application "knows" which version(s) of GeoNature it can run on.

DonovanMaillard commented 2 years ago

Yes. For more context element for @IzzySoft , at the origin occtax-mobile and geonature have been developped by national parks. Even if it is now used by more and more people, we continue to use it in a context of "I am an organism, I deploy my own GeoNature instance, and I administrate my agents devices fleat directly from my geonature instance".

In some cases, it can be volunteers from an associations, but we can't provide how many times a year the organism update his geonature on server side. GeoNature's API is moving across versions and it can broke Occtax-mobile data synchronization with the server.

@sgrimault , could we imagine having 2 versions code (one for F-droid and one for github ?) and the app check the both ? Or if we reverse the problem, is it possible GeoNature configuration give a min and max compatible versions, and "block" updates from f-droid if the new version is not compatible with GeoNature ?

IzzySoft commented 2 years ago

Thanks for the details! That looks more complicated than I imagined. You cannot "block" updates from F-Droid (as you cannot block updates from Play – or keep a user from adb install -r newver.apk), you can only block your own update checker inside the app. If "the store" offers an update of your app and the user decides to install it, it will be installed. That's the way things work with stores and repositories.

But that problem already exists now, as you pointed out:

in the worst cases a user can connect to several geonature servers

So what if one server has it's "max supported version" below the currently installed, while another has its "min supported" above? The only things you can do about that is having instances updated quickly – and showing a warning in the app if one of the instances configured has its max at the current version (a la "dear user, if you update now that instance becomes inaccessible").

camillemonchicourt commented 2 years ago

Yes, the particular point with Occtax-mobile and GeoNature is that each organisation has its own GeoNature instance on its own server and manage its updates process and planning.

When publishing Occtax-mobile on F-droid I had thought to this point.

I think that we can go this way to start with 2 different keystores. Those who will install Occtax-mobile from F-droid will only update with F-droid. Those who will install Occtax-mobile directly from a GeoNature server will update it from there.

Actually we can't choose to install Occtax-mobile only with F-droid, but we'll see how it goes in the future.

In the future, we will think about the versions compatibility. GeoNature server could expose its version and Occtax-mobile could check it to update or not.

IzzySoft commented 2 years ago

That sounds like a good approach to it (combined with having the update check disabled in the F-Droid version). The app description should make this clear, though, so users get to know before installing and configuring.

Remains the question on the other servers contacted by the app: are they all falling under a libre license, or are any of them not (so we need to raise the NonFreeNet anti-feature at F-Droid)?

camillemonchicourt commented 2 years ago

All GeoNature instances are based on GeoNature application that we develop and maintain and is under libre license.

sgrimault commented 2 years ago

To summarize:

Added a new dedicated flavor for distribution on the F-Droid store. This flavor will allow to activate or not the update of the application directly via GeoNature.
As we don't have control over the update of the application, it is important to be able to determine a table of correspondence between the versions of the application and GeoNature. The user will then be notified in case of incompatibility. This verification will be done at the time of the GeoNature server configuration and at each application startup.

PnX-SI / gn_mobile_occtax

F-Droid inclusion #134