F-Droid - Githubissues

linsui commented 4 years ago

Would you please to publish this great app in F-Droid? This will help this app arrive more users caring about privacy and safety. Thanks!

Gedsh commented 4 years ago

I have a few problems with the F-Droid. There are f-droid requirements that InviZible does not meet:

The software should not download additional executable binary files (e.g. non-free addons, auto-updates, etc).

When including donation information, the relevant donation links (e.g. Bitcoin/ PayPal/Flattr/etc) must also be available either:

In a README or similar file in the projects source code.

On the applications main website.

If the software is hosted on GitLab, then it is sufficient that the person requesting donation info to be added to the fdroiddata repository is the same user account which manages the application source code.

Packages built by F-Droid are signed by F-Droid, so all apps in the official F-Droid repo are signed by F-Droid keys. Only APKs built by F-Droid are included in the official F-Droid repo.

So I have to disable DNSCrypt, Tor, Purple I2P updates. Disable own InviZible auto-update. Cut information about donate.

In addition, InviZible has some signature protection, which displays a warning only in case of a sign mismatch, to inform users that the application was not build with the author. It should be disabled too.

Publishing app in f-droid require disabling many features. Maybe later...

linsui commented 4 years ago

Thanks for your reply. @IzzySoft Could you please help? I guess this can go to your repo first.

IzzySoft commented 4 years ago

@Gedsh are you sure you got that right? I'm just reacting on the ping sent by @linsui here, so I have no deep knowledge of your app and all I do here is an educated guess:

Why do you thing you need to disable DNSCrypt etc? I guess you mean because of "binaries". Don't take that too literally, or we'd have to remove all web browsers (they download images! And other stuff!). What is meant here, put into general terms, is "no sneaky download of addons you run in the background without the user's consent". Updating DNSCrypt keys etc. should be fine – especially if it happens "on the user's advice" (aka configuration/setup).

Yes, the signature protection would need to be disabled for the F-Droid build. This should be possible using e.g. a build flavor. And what you definitely need to take care for is increasing versionCode with each release: up to now, you've left that at 1. This makes updates impossible (and is the reason I cannot yet follow @linsui's suggestion to take the APK into my repo: fdroidserver would complain with each new version that it's a duplicate (versionCode? is how Android tells versions apart).

Gedsh commented 4 years ago

are you sure you got that right?

No :)

Why do you thing you need to disable DNSCrypt etc?

Do not disable completely, just disable automatic updates.

Updating DNSCrypt keys etc. should be fine

InviZible usually replaces DNSCrypt, Tor, and I2P executables when performing automatic updates. These are binary files. But only with user confirmation. Is this acceptable for f-droid?

Yes, the signature protection would need to be disabled for the F-Droid build. This should be possible using e.g. a build flavor.

I use build flavor already. But signature protection is a little complex. Anyway it can be disabled. It is not a big problem.

you definitely need to take care for is increasing versionCode with each release

OK. I will start.

Thanks so much for the tips!

IzzySoft commented 4 years ago

Do not disable completely, just disable automatic updates.

Do not disable completely, just disable automatic updates by default – i.e. the user must toggle them on manually once, for example. The mechanism can be present.

But only with user confirmation.

Then it should IMHO be fine – as long as the confirmation is "honest" (i.e. it's clear to the user what (s)he approves). If that were not OK, we'd have to kick our own client (downloads and installs apps, man!) as well as similar ones like Yalp Store and Aurora Store/Droid (they also install apps).

But signature protection is a little complex.

If you can disable it for the F-Droid flavor, that would be sufficient. If later you find a better solution, that can always be updated.

versionCode: As soon as you have implemented that, please let me know. I could then immediately pick your current APK (which still has it on 1) and put that into my repo. When you later publish the update, versionCode will be increased and all is fine. Adding it to my repo is quick – adding to the official repo will take at least a couple of days (usually 1..2 weeks, sometimes even longer). Once it reached the official repo then, I can remove it from mine again (to avoid confusion caused by the signature mismatch).

Gedsh commented 4 years ago

just disable automatic updates by default

Understood.

versionCode: As soon as you have implemented that, please let me know.

Already implemented for next beta. It's simple.

If you can disable it for the F-Droid flavor, that would be sufficient.

I am looking for the easiest way to do this. It will take some time.

Gedsh commented 4 years ago

Once it reached the official repo then, I can remove it from mine again (to avoid confusion caused by the signature mismatch).

I think the best way to completely disable all auto updates for the f-droid version. So f-droid version will not have this functionality.

IzzySoft commented 4 years ago

@Gedsh so the next release will have versionCode increased? Then I could pick the APK now and add it to my repo.

I think the best way to completely disable all auto updates for the f-droid version.

That would mean "crippling" your software beyond what's needed IMHO (even the F-Droid app, with the help of its "privileged extension", allows automatic update of apps). But of course, it's your decision.

Gedsh commented 4 years ago

so the next release will have versionCode increased?

Yes.

Then I could pick the APK now and add it to my repo.

You can. Thank you.

IzzySoft commented 4 years ago

Thanks, done – should show up here in about 20h (with the next sync).

May I suggest to establish Fastlane file structure in your repo here, to provide e.g. screenshots? F-Droid would pick up that automatically then (and I could pick the screenshots manually for now).

Gedsh commented 4 years ago

May I suggest to establish Fastlane file structure in your repo here, to provide e.g. screenshots?

OK. But a bit later.

should show up here in about 20h

Thanks. You have a very useful repo.

IzzySoft commented 4 years ago

Thanks! Please ping me when Fastlane is there.

Gedsh commented 4 years ago

Please ping me when Fastlane is there.

OK

Gedsh commented 4 years ago

Thanks! Please ping me when Fastlane is there.

Hello. I have prepared Fastlane. Thanks.

IzzySoft commented 4 years ago

Thanks! Added the screenshots to the app's listing in my repo. Did you plan to go for the main F-Droid repo? I've just checked and couldn't find any RFP (Request For Packaging) or merge request. Don't get me wrong, I'm totally fine having your app in my repo, so I'm just asking :wink:

IzzySoft commented 4 years ago

Hehe, nice way to answer that question. Thanks!

Gedsh commented 4 years ago

Thanks to you too.

Gedsh commented 4 years ago

Hi, @IzzySoft I'm just curious to know. What is the problem with f-droid build? Maybe I need to fulfill some additional requirements?

IzzySoft commented 4 years ago

Looks like our packagers didn't yet find time to process it. As I've just pinged them a week ago, I'm hesitant to do so again right now. Let's give them at least another week, shall we? If they miss something they will tell. As there was nothing told, they simply haven't yet picked up.

Gedsh commented 4 years ago

Let's give them at least another week, shall we?

OK. No problems. Thank you.

IzzySoft commented 4 years ago

@Gedsh just wondering: the per-release changelogs in your fastlane seem to have the wrong names. E.g. the APK of the last version has versionCode 3078 – so the corresponding file should be 3078.txt. That does not exist. Instead, there is 100078.txt. Moreover, according to build.gradle the versionCode should have been 78 (so as this does not match, we cannot enable auto-update in F-Droid). What do I miss there? Or did you miss something?

Quite confusing. So before I ping our packagers and ask where this git stuck, we'd better clarify this before :wink:

Gedsh commented 4 years ago

I have armv7a and arm64 builds. They must have different version codes. For armv7a versionCode = 2, for arm64 versionCode = 3. build.gradle uses a script to generate version codes 2078 and 3078 at compile time:

applicationVariants.all { variant ->

    def abiVersion = variant.mergedFlavor.versionCode

    variant.outputs.each { output ->
        output.versionCodeOverride = abiVersion * 1000 + defaultConfig.versionCode
    }
}

we cannot enable auto-update in F-Droid

So what should I do with armv7a and arm64 versions?

we'd better clarify this before

I am not familiar with the F-Droid. You say, I do.

IzzySoft commented 4 years ago

Well, I'm neither packager nor Android dev – so at this I must pass. @licaon-kter @rudloff could one of you give advice? How do we deal with multi-architecture?

Rudloff commented 4 years ago

AFAIK, we don't support auto-update for multi-architecture apps, so the build entries have to be added manually.

Gedsh commented 4 years ago

build entries have to be added manually

Fine. Thank you. But what exactly should I do? I use gradle build flavors as build entries.

Rudloff commented 4 years ago

I am talking about build entries in the F-Droid metadata file for your app. If we can not enable auto-update for this app, someone (you or an F-Droid contributor) will have to submit a merge request to update the metadata for each new release.

JonnyTech commented 4 years ago

@IzzySoft I only see the arm64 version on your repo, do you plan to host the arm32 too or just wait until it hits the official f-droid repo?

Gedsh commented 4 years ago

If we can not enable auto-update for this app, someone (you or an F-Droid contributor) will have to submit a merge request to update the metadata for each new release.

Thank you for the advice. It will take a time.

IzzySoft commented 4 years ago

@JonnyTech that's not the intended way; I usually pick the 32bit version as that also runs on 64bit. Replaced, and fixed the config so it ignores the arm64 in the future. Should be reflected with the next sync tomorrow. Thanks for pointing out!

licaon-kter commented 4 years ago

FYI https://gitlab.com/fdroid/fdroiddata/-/merge_requests/7305#note_402386818

6543 commented 4 years ago

@Gedsh it would be realy nice to know how you build the whole stuff in tordnscrypt/libs/

Gedsh commented 4 years ago

it would be realy nice to know how you build the whole stuff in tordnscrypt/libs/

Copying dnscrypt here https://github.com/DNSCrypt/dnscrypt-proxy/releases Copying tor here https://github.com/guardianproject/gpmaven/tree/master/org/torproject/tor-android-binary

Compiling from source on my local machine: https://github.com/PurpleI2P/i2pd https://github.com/Yawning/obfs4 https://github.com/keroserene/snowflake

Gedsh commented 3 years ago

@IzzySoft I started posting the stable version with the beta on Github, so maybe some fixes will be needed for your f-droid repository. I just don't know how best to contact you. Thanks for maintaining your f-droid repo!

IzzySoft commented 3 years ago

Oh, this place is fine (or any separate issue you want to open for it – either here with a ping to me so I get "invited", or at my Repo's GitLab repo (oops) :smile:

OK, so what needs to be fixed/adjusted? I currently pick the .*amr64.*\.apk from its corresponding release, adding 98000 to its versionCode to fetch the matching changelog. No restriction as to the tag names, but ignoring pre-releases. So…

if you want me to only pick stables for now, I can either match on the tag name (you currently use the tag naming pattern v<versionName>-(beta|stable), so I could eg match that against v%v-stable) – and similarly switch to betas once stables are on "F-Droid proper"
you could tick the "this is a pre-release" check-box when creating a beta release, so those would be ignored automatically
you could just continue as you do now: it seems your betas are all useing v1. while releases use v5.; once my updater picked the first v5, it won't ever pick any v[0-4].* again.

Just let me know which path you like, and I see how I can adjust to it.

licaon-kter commented 3 years ago

@Gedsh is an official inclusion still far?

IzzySoft commented 3 years ago

@licaon-kter once it's "officially included" (looking at the backlog in RFP I guess that won't be this week), I'd switch to serving betas in mine if @Gedsh wants me to (and propose to use a slightly altered package name for those, eg. an appended .(beta|testing) to easily keep them apart). That way stables (for the end-users) would be available at F-Droid, while the testing/beta ones are not held back (my updater is a bit faster as I don't have a build process to run etc – and for testing purposes this should be fully fine).

PS: the first v5 will be pulled automatically in about half an hour, according to the current setup.

Gedsh commented 3 years ago

@IzzySoft I think the beta is more interesting for your repo as the stable release comes out about once a month. But it's up to you. Better to use the tag (stable | beta). Thanks!

IzzySoft commented 3 years ago

@Gedsh that's what I was suggesting for when it's on F-Droid: Let F-Droid use the *-stable tags and me switch to the *-beta ones. To keep the two apart, append .beta to the beta's package name (i.e. make that pan.alexander.tordnscrypt.beta) and " (beta)" to the "display name" (e.g. "InviZible Pro beta" or "InviZible Pro (beta)"). That way…

users can easily distinguish which one to install
descriptions are not getting "mixed" as different package names make sure F-Droid handles them as separate apps
users even can install both in parallel (e.g. to verify whether a reported problem was solved with a given beta)

Ideally add a "beta" to the icon as well – so if someone has both installed it's easy to see which is which.

For now we simply leave things as they are. The package name you will use for the stable at F-Droid will remain unchanged, so no need to hurry now. The moment the app appears in F-Droid's index you start letting the beta having the .beta suffix, let me know, and I'll switch to that.

How does that sound to you?

IzzySoft commented 3 years ago

@Gedsh Oh, I see you've already changed the setup. But what doesn't seem to make sense: why is the versionCode of the 1. releases higher than that of the 5. releases – and why do the 5.* releases carry on with the versionCode the other package name had? As package names differ, you could even let both packages share the same versionCode.

May I suggest to switch the betas back to versionCode 2xxx? Unless I've missed the reason for this. To carry on, I now have to switch to the beta tags explicitly. But once a beta with 3xxx was published, switching that back to 2xxx would cause confusion – so please let me know first which it should be. Especially as fetching the corresponding change log would mean I had to change calculation of its versionCode: your fastlane's changelogs/*.txt don't match any. With the 2xxx VCs I had to add 98000 to match them – with 3xxx I had to make that 97000 (I hope for F-Droid they will simply match)…

Apart from that, I get the idea: I shall just carry on with the very same package name in my repo as it was before. I like that :smiley:

Gedsh commented 3 years ago

@IzzySoft I cannot change the package name as it will require the user to do a clean install, which is not desirable.

I do not want to change beta version code as it will break built-in auto update. The version code is 2 for armv7a and 3 for arm64, paired with the real version code. For the latest beta -> armv7a, version - '2'(armv7a) + '115'(version code) = 2115 -> InviZible Pro beta 1.1.5 (armv7a); arm64 version - '3'(arm64) + '115'(version code) = 3115 -> InviZible Pro beta 1.1.5 (arm64).

Stable version has the version codes as a corresponding beta. Stable version 5.0.0 has codes 2115 and 3115 for armv7a and arm64 respectively. The name of the stable release is increased based on minor or major changes in the code or functionality.

Beta and stable versions already have different package names and icons, so they can be installed in parallel. Beta is pan.alexander.tordnscrypt, stable is pan.alexander.tordnscrypt.stable.

IzzySoft commented 3 years ago

I cannot change the package name as it will require the user to do a clean install, which is not desirable.

Full ack!

The version code is 2 for armv7a and 3 for arm64

Yuck! I need new glasses (or clean the current ones). Yes, yes, that's what my config says: ignore the arm64, not pick it… My bad, apologies!

version - '2'(armv7a) + '115'(version code) = 2115 -> InviZible Pro beta 1.1.5

Ah! Thanks, now I get that part :smile:

OK, all fine then – sorry for the noise and confusion! I've only adjusted the config now by fixing it to

AutoUpdateMode: Version v%v-beta

and manually triggered an update – which was successfully fetched (so config change was verified). Now the index build is through, confirming changelogs are fetched fine as well :+1:

So for your YAML at F-Droid:

AutoUpdateMode: Version v%v-stable

and each repo will stick to its own tag.

Beta and stable versions already have different package names and icons, so they can be installed in parallel.

Wonderful! Now we must only hope users are not confused by betas being seemingly that far lower than releases. But as that's only "cosmetics" with no technical effects, it can be adjusted any time if needed.

So: you've planned that excellently – just my brains were a bit sluggy to grasp it :smile:

IzzySoft commented 3 years ago

PS: if you compare your app's description in my repo with the one you have here in Fastlane… would you consider a "merger"? Basically the formatting from mine (I could make a PR for that), I'm just not sure about the content. Currently I simply keep mine, but it might be better to hand that back to you so it doesn't get outdated.

Also not sure if there should be different descriptions for stable and beta. I don't think so as the two won't be that far apart. And if you adjust it for a new beta, my updater would fetch it with that beta and F-Droid would simply stick to the one it fetched with the last stable, until the next stable is released, so a single one should fit.

Gedsh commented 3 years ago

@IzzySoft Yes, there is some confusion with the armv7a and arm64 versions.

The f-droid version currently has the package name as a beta - pan.alexander.tordnscrypt. Should I change it?

Gedsh commented 3 years ago

@IzzySoft

if you compare your app's description in my repo with the one you have here in Fastlane… would you consider a "merger"?

I think this is a good idea. This simple description is better for the new users.

IzzySoft commented 3 years ago

Oops? I got pan.alexander.tordnscrypt.stable for the ARMv7 APK at the stable tag, which should be fine.

package: name='pan.alexander.tordnscrypt.stable' versionCode='2115' versionName='5.0.0' compileSdkVersion='30' compileSdkVersionCodename='11'

Ah, correspondingly the YAML file must be named pan.alexander.tordnscrypt.stable.yml, yes.

I'm no packager, so I have to delegate the question concerning multi-arch to @licaon-kter and @Rudloff (I forgot if those will be handled as separate APKs or as single multi-arch APK).

Gedsh commented 3 years ago

I got pan.alexander.tordnscrypt.stable for the ARMv7 APK at the stable tag, which should be fine.

Got it. I will change the package name to pan.alexander.tordnscrypt.stable

I forgot if those will be handled as separate APKs or as single multi-arch APK

It's impossible to make a multi-arch APK without changing the source code. So it could be separate APKs or just armv7a as it works on both armv7a and arm64 processors.

IzzySoft commented 3 years ago

This simple description is better for the new users.

Now I just need to make sure if you meant mine or yours :smile:

So shall I prepare a PR based on mine, and you decide what of yours should be added (and maybe what of mine removed) after the merge, based on the diff?

I will change the package name to pan.alexander.tordnscrypt.stable

According to the APK at tag v5.0.0, that's already done. But make sure to name the YAML file the same as you name the package, else the build will fail.

or just armv7a as it works on both armv7a and arm64 processors.

Yeah, that's why I picked that for my repo actually. OK, let's leave that for the packagers, it's not really my area of proficience. I'm the one for the metadata (and even there I have to ask from time to time).

Gedsh commented 3 years ago

So shall I prepare a PR based on mine, and you decide what of yours should be added (and maybe what of mine removed) after the merge, based on the diff?

Of course, if you want to.

According to the APK at tag v5.0.0, that's already done. But make sure to name the YAML file the same as you name the package, else the build will fail.

InviZible uses Gradle to build. It has "flavors" for different versions. I think f-droid uses gradle too, so it should use "fdroid flavor" specially designed for f-droid.

IzzySoft commented 3 years ago

Of course, if you want to.

OK, will do (just not today, it's past midnight already).

it should use "fdroid flavor" specially designed for f-droid.

Yes, please! That would be awesome. Thanks!

Gedsh commented 3 years ago

OK, will do (just not today, it's past midnight already).

Sure! Whenever you want.

Yes, please! That would be awesome. Thanks!

It was done a long time ago. I just changed the package and version name. ` productFlavors {

    fdroid{
        applicationId "pan.alexander.tordnscrypt.stable"
        versionName "5.0.0"
        dimension = 'version'
        resValue 'string', 'package_name', applicationId
    }

    armv7a{
        dimension = 'processor'
        resValue 'string', 'appProcVersion', 'armv7a'
        versionCode = 2

        ndk {
            abiFilters 'armeabi-v7a'
        }
    }

    arm64{
        dimension = 'processor'
        resValue 'string', 'appProcVersion', 'arm64'
        versionCode = 3

        ndk {
            abiFilters 'arm64-v8a'
        }
    }

}`

Gedsh / InviZible

F-Droid #5