Publish to F-droid

[lachmanfrantisek]

Hello,

thank all of you for such a nice app. Would it be possible to publish talk also to the f-droid repository like other nextcloud apps for android?

Thanks in advance!

[mario]

Hopefully in the future, yes - but the issue is you won't be able to receive push notifications unless you have MicroG, and I hear it's relatively hard to install.

[lachmanfrantisek]

Thank you for a quick answer.

I am not sure about the MicroG -- it can be probably installed directly via apk? (Or use play only for this.)

Would be nice to install at least talk via f-droid.

[mario]

MicroG is a set of services that are open source replacements for the Google Play services on "regular" devices.

https://microg.org/

You can download an APK of the Talk app directly until I figure out a proper way to support F-droid. https://github.com/nextcloud/talk-android/releases

[lachmanfrantisek]

OK, sorry that I messed the MicroG and GlooglePlayServices...

I have a google-play, but would like to use it as little as possible...;-)

Thank you!

[mario]

Understood. Feel free to install the APK directly for now and let me know how you like the app!

[lachmanfrantisek]

Still waiting for the server update and preparing on the phone side..;-)

[ghost]

To publish to F-Droid, there would need to be a seperate FOSS flavor without GCM... And I don't think it's acceptable to be dependent on Google services for a project such as Nextcloud in the long run. Anyway, really cool app, I would love it to be completely independent from proprietary alternatives like Telegram :)

[mario]

@Matttter I've done the separation for the Nextcloud Android app, so I know. There's no way to do push notification without Google services, at least not on the server side. On the client side I hear it's possible to use MicroG but I've never tried it.

Maybe I will in the future.

[hex-m]

We had the discussions of GCM-dependency for years until it was fixed in Signal. It's true that you don't have push notifications without gapps/microg but that just means the device has to contact the server regularly and thus use more battery - right?

[mario]

This is not a philosophical, but a technical issue. This is a talk application - once I get a call, I want to get a notification right away, not in 5 or 10 minutes. Now, technically:

• the only way to make sure I can check the server in an interval is alarm manager

• how often would you query the server? 0.5 seconds? 1 second? 5 seconds? 10?

• to make sure such frequent polling works, you would:

  • have to disable battery saving
    • and still, some phones/vendors would stop the process after you do this for a certain number of times
    • the battery would be depleted in half an hour to an hour max

• Signal implements a service which is more than likely to die on quite a few phones, even when you ignore battery optimizations and other things. Sure, you can restart automatically, unless the watchdog services dies too :)

So if you ask me is this possible: definitely. Is it possible in a way where you wouldn't yell at me after a while for killing your phone? Absolutely not.

That being said, there will be an implementation for F-Droid eventually. I just don't know when I'll get to it, but am more than available to help you contribute if you decide to do so! :)

[kertase]

hasn’t push been around since way longer than google play? So it should be possible to use for example an http push protocol instead of google play?

[tobiasKaminsky]

Until you find the time @mario to get push implemented on f-droid in a proper way, why not do it the same as in the nextcloud app: If detected that push is not supported/working show a snackbar saying that the user will not receive a notification on an incoming call. But the user can still use the app to call people, to join a conference call.

I think that people that are using a complete google free device are willing to accept some drawbacks. (At least I am doing ;-))

[ghost]

Yes, if it's not possible or practical to implement a notification service, then just having the app available on F-Droid, even without the notifications, would be great. Like @tobiasKaminsky proposed, a snackbar or a dialog on first launch or something of the sort that tells users what's up would then make sure users who do really need the notifications could resort to the regular app

[ghost]

@mario

There's no way to do push notification without Google services

why is that? what is so exceptional about GCM that it can't be replicated by another party? is there any documentation where i can read up on this? how does it work, shouldn't it just be a server sending notifications to a client and that's it? how does Google prevent the enormous battery drain and what prevents anyone else from doing the same?

[mario]

It's a long story. Like I said - you can use MicroG to receive notifications sent by Google WITHOUT having Google services installed, but it's a bit involved. Alternate ways include things what signals does, or what we'll eventually do - service, IGNORE BATTERY OPTIMIZATIONS, and do http get every now and then (quite often, 10 seconds?).

I'd say GCM is magical in a way it can interact with the system due to having the privilege of being a Google product, so system will never kill, as opposed to even foregrounded services (or even apps that ignore battery optiomizations) which are less likely to get killed, but still do get killed.

You can download an APKs from here if you don't have Play Store: https://github.com/nextcloud/talk-android/releases

Sure, they have Google Play compile code baked in, but I think it shouldn't crash - the only thing is you won't be receiving notifications.

Like I said before, there will be an F-Droid version, I just don't want to make promises when since I want to get it right, as much is possible in this situation.

[kertase]

Instead of get could you not use use long pooling or maybe webpush api could be implemented in nextcloud and could be used instead ?

[mario]

Since Nextcloud has to be accessible on lots of installations, websockets or such is not an option (and you'd still need a service!). And long polling still requires a service that would do it.

[ghost]

@mario

so the only actual advantages that GCM has over other notification services is that it's very optimized to not drain much battery, and that its service doesn't have to restart because the system will never kill it?

then maybe the best way to do it without requiring multiple different app versions would be:

if GCM/microG is installed on the system then the app uses it

if GCM/microG is not installed but the system is rooted the app asks the user for permission to run the service as root so it won't be killed

if GCM/microG is not installed and the system is not rooted, the app tells the user that the notification experience may not be perfect

to make sure that not every nextcloud suite app needs root maybe the main app could implement this and handle the notification service of the talk app and any other app that needs it, if GCM/microG isn't present.

also the user should be able to set a custom refresh rate for the notifications, so everyone can decide for themselves how much speed vs battery they want.

btw since google plans to switch from GCM to FCM, will any of this change or will it basically stay the same?

[mario]

GCM == FCM mostly. It's the same thing.

[testbird]

An elegant way for nextcloud to avoid depending on google push servers (or whatever current cover name) could be to leverage on the IMAP push. On phones without GCM it is proven to work (https://delta.chat) and it is using even less battery then having GCM installed.

All the NC server would have to do is to send a special email (so they can also be easily filtered), and the NC client could watch the IMAP server for such messages.

(For XMPP there might also exist a working implementation, but almost all users may already have an email account.)

[testbird]

Actually, there is a delta.chat issue open to support initiating video chat. So maybe get in contact to work out an "email signaling format" that could let delta chat or other MUAs open a NC video chat. Then you don't even have to implement and run another IMAP watcher on the phone. https://github.com/deltachat/deltachat-core/issues/101

[testbird]

@mario is there maybe already some manual way to invoke the nextcloud app from another app? For example, can it answer a call if the caller sends a link in an email message, and the user clicks on the link.

[testbird]

Some "email push" message standard brain storming (possibly based on the autocrypt, and email chat standard) at https://github.com/deltachat/deltachat-core/issues/101#issuecomment-364890221

[testbird]

The paging / notification works very fast with deltachat. So if initiating a call with Nextcloud talk could send an email message with a link, then answering/receiving a call may be made as easy as clicking the link in the message when it arrives in the mobile email messenger (deltachat, conforming to autocrypt and other stanards https://delta.chat/en/standards), and have the messenger app open the link (call) with the nextcloud talk app.

[schemar]

I am assuming you do not want to look at push implementations where you need to set up a different server as everything should be part of the Nextcloud server, right? Maybe it is time for Nextcloud to create an open PHP project for push notifications ¯_(ツ)_/¯

[mario]

Well ... afaik, you cannot do a websockets as part of PHP, so it would need to be a separate server, sure. And this would obviously increase the barrier to entry as not everyone will be able to host it. It would also need an app to nextcloud to make it work, or a patch to an existing one. So not that easy :)

[testbird]

That's why I proposed email. By now it has wide-spread push functionality, and virtually everybody has email, can get or set up email (even through self-hosting projects).

Establishing an Email-push-notification standard that defines push messages in a way that allows to auto-sort/filter them into a separate folder by sieve server filters and/or the IMAP push subscribed client (i.e. an Email-notify standard similar to the Email-chat standard https://delta.chat/en/spec) would realize an alternative that does not require to set up additional new servers. Push capable client libs that allow easy encryption through back-and-forth handshake are available (e.g. https://github.com/deltachat/deltachat-core), and seem easily expandable to support notifications in addition to chat messages.

[testbird]

Note that an email push notification standard could reduce the transfered data quite much. Only for the initial autocrypt initialization handshake it would have to use full email messages with a body and public keys attached. The actual push messages might not have to contain much more than a signature as "payload" (possibly in the subject). IMAP push then informs the client that a new message is waiting, and the client may optionally load only the subject for verification?

[mario]

The Telegram-FOSS issue is correct. You cannot guarantee a persistent service, and IMAP push cannot wake the device.

[mario]

There's MicroG like I mentioned before, and that's the best solution, even if it's still subject to background limitations etc etc etc.

[testbird]

Does push necessarily need to wake the device? Maybe ringing and letting the user wake the device is enough?

[mario]

You need to wake the device to ring :)

On Wed, 14 Feb 2018 at 09:27, testbird notifications@github.com wrote:

Does push necessarily need to wake the device? Maybe ringing and letting the user wake the device is enough?

— You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub https://github.com/nextcloud/talk-android/issues/58#issuecomment-365530340, or mute the thread https://github.com/notifications/unsubscribe-auth/AAAWsn4s1XyPCQpC8cPFdTFfZm470zApks5tUpjjgaJpZM4RZfK7 .

[testbird]

Ok, then deltachat-core lib is already able to wake the device. It holds open a permanent notification that it is waiting for messages. To avoid that every app need its own watcher and hold an own notification open, deltachat-core client lib could maybe be packaged separately and provide a shared service that forwards messages to other apps.

[schemar]

One last try, even though I understand that push is practically not feasible :disappointed: Ratchet is a PHP WebSocket server implementation: https://github.com/ratchetphp/Ratchet

[X-Ryl669]

The issue with MicroG is that it's replacing/emulating GCM. In order to do so, it's replacing Google's private key in the ROM by its own keys for the GCM's service, so you must be root to install it. There is no MicroG without being root and you can't have both GApps and MicroG, so said differently, it's not a solution for the majority of users.

There is no solution to this issue, Google locked us all to its system for push notification.

The only solution that might work, it to build a common service/proxy that's using GCM to trigger fake notifications with some ciphered content that's telling which intent/application to wake up. That way Google will not know what is the notification about, yet your software can still be opensource and not dependent on Google's closed source (and accepted in F-Droid).

However, the "common service/proxy" that's relying on GCM can not be integrated in F-Droid (since it depends on GCM) but if it's open-source, and available on the play store, it might still be acceptable.

So basically, in this utopia: 1) Users having GApps will have to download the common service/proxy from the Play store and download the other software from F-Droid 2) Users not having GApps but MicroG can download the common service/proxy from the common service/proxy website (or maybe F-Droid ?) 3) Users without GApps nor MicroG don't have GCM anyway so they can not get push notification on recent Android versions/phones (unless they deal with battery optimizations, but this proved to fail anyway).

[thermatk]

A better idea would to move to foreground with a persistent notification(as notifications can be hidden per channel in Oreo, it's not annoying). Yet it would be extremely ineffective to do this in every app. Implementing one common gate for push notifications that users can self-host and installing one foss helper app that would talk to it would be the best. And I don't know a project better suitable to be at the center of such new system than Nextcloud, the self-hosted everything else 👍

That would also be the solution for our https://github.com/Telegram-FOSS-Team/Telegram-FOSS/issues/172 Continuing to dream, the client library API can be made as close as possible to GCM, while sadly I think on the server it would take more effort to incorporate it. The more reasons having a good common standard, instead of a myriad of solutions for every app.

[testbird]

Thanks for clarifying @X-Ryl669. This emphasizes the need for a free, federated and commonly maintained push messaging alternative. Not only for android devices, but also for UBports, Librem 5, plasma-mobile, custom roms etc.

Users without GApps nor MicroG don't have GCM anyway so they can not get push notification on recent Android versions/phones (unless they deal with battery optimizations, but this proved to fail anyway).

Experience showed notifications work very well with https://github.com/deltachat/deltachat-core which builds on IMAP push (federated email system).

[testbird]

And I don't know a project better suitable to be at the center of such new system than Nextcloud, the self-hosted everything else

Then one would only be able to reach Nextcloud users, instead of all email accounts. (=> includes IMAP-push email servers)

[X-Ryl669]

deltachat use a foreground service so you get a permanent notification (you currently can't mask because any other notification would be masked too). So no, deltachat solution is not good either. I don't want to clutter my notification area with many filtering rules, it's not intuitive and, IMHO, it's a hack that Google will fix in a later version anyway.

[X-Ryl669]

@thermatk Can you be more specific about the "hidden by channel" in Oreo ? How do you do that ?

[thermatk]

@X-Ryl669 Oreo introduced Notification channels, so even if the app implements it like having two channels "Persistent push channel" and "Everything else", one would be able to hide and mute just the persistent notification. Didn't see an app needing or doing it yet, but should work. Yet channels are not important if we make one tiny unified push receiver with one open socket connection and one notification.

[testbird]

channels are not important if we make one tiny unified push receiver with one open socket connection and one notification.

Well, the android notification channels won't be important to hide the permanant notifications of multiple apps, but the "unified push receiver" would still need to be able to have something like channels to know to which app a message needs to be forwarded. (https://github.com/deltachat/deltachat-core currently implements "chats" and "group chats" (based on the From: and autocrypt).

[thermatk]

This has nothing to do with notification channels, just with the implementation of client app, respective lib to be added to apps and their inter-process communication

[testbird]

Right, we may better refer to that as something like inter-process receiver apps. (Current https://github.com/deltachat/deltachat-core frontends could become inter-process receiver apps as well, instead of packaging the lib.)

[Bubu]

Wow, this issue exploded...

Here is my 2c, why microG shouldn't be the solution:

• the needed libraries client-side are still proprietary, so no inclusion into f-droid for everything that requires it.
• We are still relying on the centralized google infrastructure which is bad for privacy concerns. Even if sending just empty pushes through that it still allows google to analyze all the communication metadata. (who communicates with whom and when.)

Otherwise I geneally agree with @thermatk here

[X-Ryl669]

@thermatk It has. Because the only way to avoid GCM is to have a process that's not killed by the system and the only way to have a persistent process is to have it run a foreground service. Because foreground service requires a persistent notification, it's a pain for users (why did google invented this?). Because it's "process" wide, unless you use notification channels, you can't really use this technic to bypass GCM (since hiding the foreground notification means no notification displayed whatsoever to the user). With notification channel and selective hiding of the "persistent foreground service" notification, it might be acceptable from a user point of view.

[testbird]

With notification channel and selective hiding of the "persistent foreground service" notification, it might be acceptable from a user point of view.

Thats how https://github.com/deltachat/deltachat-core currently implements it (as a lib, not yet a "unified push receiver" service), and it's even acceptable on Android =< 7 where I see one persistent notification "connected to name@email.server, waiting for messages ..." (and way better than not getting any push messages).

[X-Ryl669]

True. But, before Oreo, you can only hide the whole application's notification, one can not hide the "connected to ..., waiting for messages..." notification, else she would loose all notifications. If you have 20 applications like this, then it'll clutter your notification panel.

If, starting with Oreo, you can hide only the "foreground notification" but keep the other (that are sparse by nature) then it's ok from an ergonomic point of view.

[testbird]

before Oreo, you can only hide the whole application's notification

Wouldn't that actually be ok for a "unified FLOSS push service"? I mean if https://github.com/deltachat/deltachat-core would be converted from a stand-alone lib into a shared service app, that distributes messages to other apps, it's permanent notification to always stay in the foreground could be hidden by default. The inter-process receiver apps would then not need neither to each implement and run own push servers and clients, nor need to have a separate hidden notification channel to hide a permanent notification and always staying in the foreground. They would just need to use a "deltanotify-clientlib" to subscribe to notifications from the "deltanotify-core".

[X-Ryl669]

The issue is that not all FLOSS app use IMAP underneath (that's a huge dependency). So, only the "foreground service" in deltachat is interoperable. Instead, it could be interesting to have a common "Notification app" that being triggered by whatever FLOSS application running foreground services (and thus, hidden by user) to display something nicely and, on click, display the FLOSS application that triggered it.

Said differently, since all FLOSS app will have their own "push" protocol, it's hard to merge/unify them. However, in all cases, when something is received on the pushed channel, it's always the same behavior: The user gets notified (somehow). So, I believe the common part should be here, in the notification.

Or (the other solution): Each FLOSS implement a kind of server plugin running on a GCM-like server, and we have a common FLOSS notification application that monitors what's talking to it and wakes up the final application (talk-android, deltachat, k9, whatever).

This is good on paper, but it's a lot of work (and it's both server work and client work and system administration work). Not sure all app developer will want to delocalize their "push" protocol server side and manage it.