Publish to F-droid

[lachmanfrantisek]

Hello,

thank all of you for such a nice app. Would it be possible to publish talk also to the f-droid repository like other nextcloud apps for android?

Thanks in advance!

[testbird]

Ok, I see two aspects to this then.

(One) Having a central notification source (daemon) on the device. This makes very much sense, because otherwise multiple apps would have to implement a part that stays permanently in the foreground, draining the battery. But GCM is increasingly locking apps into their central and closed system, so we need an alternative.

(Two) Relying on a common push protocol ("on the wire"). This is not a necessity, a notification daemon could support multiple protocols, but having some common protocol options could make sense to allow interoperability. Some important features for a FOSS solution would be federation, availability, and wide-spread usage (existing users, and bullet proof implementations). Mature contenders are XMPP, IMAP,...? Projects may choose to set up their own central servers, or let the users choose or setup their own servers, but they would not have to. Many users already have at least one IMAP email server that they trust, and they are free to change servers/providers anytime.

I can't say why google chose to force GCM on android instead of relying on a standard messaging protocol like XMPP, or IMAP push. But email certainly has the most users and fortunately IMAP-push is an adequate solution for mobile clients. If they would have made the right decision, in the interest of the users, we wouldn't be in this situation now.

It actually boils down to whether one would like that Nextcloud-talk, Telegram, Deltachat, Signal, Kontalk, ChatSecure, and all the other nice Messaging and Email-Applications would get a way to communicate with each other... due to force, or to choosing a free and open source protocol?

[testbird]

Well, to get forward, people will need to reach out now, consult, and step forward with areas of knowledge, ideas, support, availability, funding etc.:

Implementing FLOSS push message service (daemon app): On android, it should create a permanent (hidden) notification to stay in foreground, and it provides

• [ ] inter-process communication with client apps
• [ ] network communication with support for:
  • [ ] autocrypt enabled IMAP-push email (using deltachat-core lib, with a filter for push spec compliant messages)
  • [ ] XMPP ?
  • [ ] ...other push server protocols?

Implementing android app support:

• [ ] client app library for inter-process communication with the push message deamon

Implementing server messaging support:

• [x] (php)sendmail with autocrypt helper (muacrypt or pyac https://autocrypt.org/dev-status.html)
• [x] own IMAP push server (https://en.wikipedia.org/wiki/Comparison_of_mail_servers)
• [ ] ...other push server protocols?

(@hpk42 @r10s @Ampli-fier)

[thermatk]

@testbird what am I missing and how is it even related to IMAP finally? To implement push emails one needs an open connection, for which there is an extensions for IMAP, but what's that have to do with the daemon? Is there an open source implementation?

In my opinion, priority would be API compatibility of the client library with GCM, changing just the package name in the best case and keeping other things equal. What has to change in the API is only the additional required setting with custom server domain/ip.

[X-Ryl669]

@thermatk What you describe is called MicroG. This has downside because: 1) GCM's application is a system application, you can't replace it without being root 2) The application is using cryptography to prevent being replaced -kindof- so you must be root to add your own keys 3) Not all application developers want to manage one or more servers for their users, yet GCM requires a server to work. This means that you have to delegate some of your privacy to a server (typically, the server needs to decrypt the message to figure who to notify) since it's the 24/7 running server that's sending message to GCM to wake your device.

I've proposed a solution for this issue in K9 without decryption, but it's specific to IMAP's IDLE implementation, and can not work everywhere.

[Bubu]

What about moving this discussion elsewhere? It doesn't seem particularly related to this issue.

I'd propose creating a new gitlab.com project (because we want to use free software, which github is not) giving everyone interested a wiki-like space for coordinating the efforts.

Additionally something for live discussion s would be nice. Not sure what we could all agree on. I can create an irc channel in lack of another common denominator. Other suggestions?

Would people be on-board to move this discussion somewhere else?

On February 15, 2018 4:30:20 PM GMT+01:00, X-Ryl669 notifications@github.com wrote:

@thermatk What you describe is called MicroG. This has downside because: 1) GCM's application is a system application, you can't replace it without being root 2) The application is using cryptography to prevent being replaced -kindof- so you must be root to add your own keys 3) Not all application developers want to manage one or more servers for their users, yet GCM requires a server to work. This means that you have to delegate some of your privacy to a server (typically, the server needs to decrypt the message to figure who to notify) since it's the 24/7 running server that's sending message to GCM to wake your device.

I've proposed a solution for this issue in K9 without decryption, but it's specific to IMAP's IDLE implementation, and can not work everywhere.

-- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/nextcloud/talk-android/issues/58#issuecomment-365962287

-- Sent from my Android device with K-9 Mail. Please excuse my brevity.

[testbird]

IMAP, but what's that have to do with the daemon? Is there an open source implementation?

IMAP is an open and wide-spread "on-the-wire" protocol for message delivery. I think 95% yes there is an implementation for what's needed for IMAP, the https://github.com/deltachat/deltachat-core lib is implementing the handling of general, autocrypted, and spam exempt, Email-chat spec compliant messages by listening on a connection to an arbitrary, regular email account (with IMAP push), without disturbing the regular email communication of the account with high frequency chat messaging. I think it would be desirable to not just rely on authenticated email chat messages for the notifications, but to have a slightly adapted spec that further separates the notifications from the general chat messages, and reduces the data overhead after the initial hand-shake. As the deltachat experience shows, there are quite a lot of things to solve to get messaging right, so the idea is to build on that.

The most prominent advantages that I see of using IMAP email is that the infrastructure is already there, and as a federated system, it does not enforce any particular organizational structure, while already being very accessible from virtually every programming language and platform. No need for app developers to run servers, no need for central servers, users may have their own servers.

[testbird]

moving this discussion elsewhere?

Time to spread the word, definitely, meet people who want this and are able to commit to this. Then start a gitlab project, or somebody is determined to take the lead and start a project.

https://stackoverflow.com/questions/20329403/android-push-notification-without-using-gcm

[iteco]

@testbird Isn't solution with IMAP leaving communication details between servers for a plain view (except encrypted message content)? If so, it looks that IMAP based solution is great for email apps (like K9) as a gradual step to improve things. But for the end-to-end encrypted new generation message apps (Telegram, Signal, NextCloud talk) that seems like a step backwards.

However, common FOSS notification server would be a great thing!

[testbird]

@iteco I really don't have an in depth answer concerning the implications for the newest generation apps.

Concerning IMAP, it should of course be secured IMAPS (non gcm servers). Think that it depends whether you communicate over your own server, and that one may distinguish between email as a universal way to communicate and just doing push notifications (for some unrelated updates etc.). This seems already quite useful for me for the general case.

To further this to the top level, one would have to use own servers, maybe there is an advantage to also implement another protocol (the checkmarks with ? above), but I don't know enough to tell.

[testbird]

@iteco Maybe you thought about something like the FOSS notification service on the mobile device, or your IMAPS server, to use something like a tor protocol. Though, just as the alternatives listed at https://stackoverflow.com/questions/20329403/android-push-notification-without-using-gcm this will require mobile and web app writers to support a new protocol, instead of just sending a (https://autocrypt.org compliant) email message.

[mario]

You should be discussing how to create a service that will not die, not the protocol :) Since the first is obviously harder and on some phones an impossible problem to solve.

[testbird]

Yes, what much can we do about such presuming phones, other than working on alternatives? Nothing.

@mario Your discussion is actually brilliant and right on point, you close the case with "impossible problem to solve", period.

[mario]

We need new phones that will get 50% of the market - done :)

[mario]

@Bubu there seems to be an alternative to client side libs?

https://github.com/microg/android_packages_apps_GmsCore

Edit: looks like I'm wrong and it doesn't work properly with Firebase :(

[mario]

@Bubu do you think you could mail me at mario@nextcloud.com? I'd like some input on F-Droid support :)

[Bubu]

As nobody objected to gitlab I created a group and a wiki project there: https://gitlab.com/foss-push/planning/

You should be able to request access to the corresponding group in gitlab.

I also created #foss-push on freenode... all welcome to at least get some clarity about goals, requirements, options for this written down.

Let's see if this goes anywhere.

@mario Sorry for hijacking your issue tracker ;-)

[mario]

With some help from @Bubu, I've done lots of work on F-Droid support (without push of course), so we might still see this on F-Droid for 1.1 too :)

[testbird]

Thanks, Mario, looking forward to the F-Droid premiere!

(PS: @thermatk better explanations concerning push protocol and IMAP seem to be covered at https://gitlab.com/foss-push/planning/ now.)

[Bubu]

I've a preliminary metadata file here: https://gitlab.com/Bubu/fdroiddata/commit/3e5144287ebc004c9edf92100552f1b7ad31afe2

What's missing is a release containing the necessary changes for this to work. And @mario could you add descrition, summary and screenshots to your repo? See here https://f-droid.org/en/docs/All_About_Descriptions_Graphics_and_Screenshots/

[mario]

Hm, I thought I've answered this on IRC, but just in case .... ^_^

• screenshots, description and summary will be added
• beta2 is scheduled in a week or so
• as for tags, all unstable releases are either marked as unstable (via github flag (can you track this?)) or I'll have a "beta" in the tag name
• are capitalized flavour names a big issue for you?

cc @Bubu ^_^

[Bubu]

Hm, I thought I've answered this on IRC, but just in case .... ^_^

matrix.org's irc bridge was having trouble the last few days apparently, I never got your reply :-/

as for tags, all unstable releases are either marked as unstable (via github flag (can you track this?)) or I'll have a "beta" in the tag name

We can't track the github flag, but beta in tag name we can deal with. We'll figure something out.

are capitalized flavour names a big issue for you?

No, it just requires to manually set the output path, no big deal. Just an edge case that didn't come up before.

[mario]

@Bubu can you check shots and descs now?

[Bubu]

Screenshots and description don't show up as is. I have not much experience with this myself it might again be related to the flavour thing.

@tobiasKaminsky you implemented flavour specific fastlane metadata for fdroid, right? Could you have a look at this?

[tobiasKaminsky]

@Bubu I took your build receipt and enhanced it a bit: https://gitlab.com/fdroid/fdroiddata/merge_requests/2968

Currently it will not compile, due to a build error.

[mario]

@tobiasKaminsky you should probably make sure to only build non-beta versions.

Also what's the problem with building?

[tobiasKaminsky]

Auto update is set to stable version. We can manually add beta versions, so that an user can select and install them.

[tobiasKaminsky]

Also what's the problem with building?

Latest beta cannot be build with ./gradlew assembleGeneric

/home/tobi/projekt/nextcloud/fdroid/fdroiddata/build/com.nextcloud.talk2/app/src/main/java/com/nextcloud/talk/application/NextcloudTalkApplication.java:40: error: cannot find symbol
import com.nextcloud.talk.utils.ClosedInterfaceImpl;
                               ^
  symbol:   class ClosedInterfaceImpl
  location: package com.nextcloud.talk.utils

According to drone, this seems to be fixed on master. Therefore I removed beta build on fdroid.

[tobiasKaminsky]

@Bubu https://gitlab.com/fdroid/fdroiddata/merge_requests/2968 is now ready for review & merge :tada:

[tobiasKaminsky]

@mario can we have a stable release, which supports "generic" flavor? Current stable 1.0.14 still have google deps and therefore cannot be published on fdroid.

[mario]

@tobiasKaminsky 19.03. for a stable release.

[mario]

cc @jospoortvliet if 19.03. suits you? That is, assuming nothing bad happens :P

[Bubu]

The fdroid keysharing issue has been solved. How do we proceed? add beta4 already? wait for release?

[mario]

@Bubu awesome! I'll tag a stable release either tomorrow or on saturday. Is that ok?

[tobiasKaminsky]

Just ping me @mario, then I can do the froid release and @bubu can merge it :tada:

[Bubu]

@tobiasKaminsky Way ahead of you! ;-)

https://gitlab.com/fdroid/fdroiddata/merge_requests/2968#note_63451053

[Bubu]

@mario feel free to close this or leave it open until the release is actually live on F-Droid.

[mario]

Will close, special thanks to @Bubu , @tobiasKaminsky and everyone else who participated in this issue!

[tobiasKaminsky]

@Bubu https://gitlab.com/fdroid/fdroiddata/merge_requests/2968/diffs#note_63634442 I fear that we need to change to "gradle=generic" .

[tobiasKaminsky]

It is now in Fdroid available :tada: As I manually installed it at some point, and also installed gplay version at some point, I fear this is due to my messed up android, but does anybody have an idea how to fix this? With root I deleted everything related to nextcloud talk and even wiped all fdroid data.

[lachlan-00]

The apk package is signed when built. Fdroid and Google apks have different signatures.

When you install a google play or custom signed version you need to uninstall the apk first then install the fdroid version.

[tobiasKaminsky]

Sorry, I guess I was a bit unclear, no talk app is installed anymore. I verified this via:

OnePlus3:/ # cmd package list packages | grep -i nextcloud                                                                                                                                                                     
package:com.nextcloud.client
package:com.nextcloud.android.beta

[Bubu]

The error message says something about left over app data. You could do a find / -name talk or something maybe?

Anyway, I just installed and account importing from nextcloud worked! Very nice!

On March 20, 2018 12:10:11 PM GMT+01:00, Tobias Kaminsky notifications@github.com wrote:

Sorry, I guess I was a bit unclear, no talk app is installed anymore. I verified this via:

OnePlus3:/ # cmd package list packages | grep -i nextcloud             

package:com.nextcloud.client
package:com.nextcloud.android.beta

-- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: https://github.com/nextcloud/talk-android/issues/58#issuecomment-374559474

[mario]

@Bubu yay! I'm so happy that we waited! :)

[tobiasKaminsky]

@Bubu

With root I deleted everything related to nextcloud talk

I did exactly this, find / -name talk...

[DaCryptor]

After Googles "stunt" regarding Gapps and CTS (possible long-term lock-out for Custom ROM users in the future) it seems to be smart to find a solution to replace Google Cloud Messaging (GCM) soon.

[ghost]

And even today not everyone wants to install MicroG, just to get push notifications for one single app. But if I understand marios post correctly, there already is a plan to offer an alternative.

[mario]

@DaCryptor can you link me somewhere to read more on this?

[mario]

@DudleyDursley well, f-droid flavour is already out, I don't know when I'll get to thinking about this.

[testbird]

What kind of push notification method or other signaling is used in the f-droid version?

[mario]

@testbird absolutely none. I might implement polling mechanism eventually, but this will most probably kill the battery :-/