Push notifications

[paolo-caroni]

What about use unified notification ?https://unifiedpush.org/

[2743ab2]

What about use unified notification ?

Strongly against! FOSS has gone from one unified google push notification service to come to another?! What for? So that now another service knows what software I use? I do not like it. My use of Telegram, it's only mine and Telegram! In addition, the Telegram FOSS background service does not cause any problems with device autonomy. In the absence of Google services, my Android 9 smartphone works confidently for at least 5 days on a single charge. If you have problems with this, then it is logical for you not to use FOSS, but to pay attention to the official Telegram client, whose notifications are based on a unified push service from Google.

On the contrary, I would add a line to manifest.xml to request to ignore battery optimization "android.permission.REQUEST_IGNORE_BATTERY_OPTIMIZATIONS" so that all sorts of Chinese shells do not kill the background process.

[vfosnar]

Well, you can always self-host your notification server. My opinion is there should be at least an option for using the unified push system because as I said some people host their own instances and I don't see any problem there.

[2743ab2]

My opinion is there should be at least an option for using the unified push system because as I said some people host their own instances and I don't see any problem there.

On your part, there is an unreasoned exchange of opinion, a wish. You do not answer the main question - "What for?". What "FOSS client" problem do you want to solve by adding a unified push service? Why should it be exactly https://unifiedpush.org/ and not something else like https://openpush.io ? Who will guarantee that these services will continue to exist and work tomorrow?, and not disappear like hundreds of xmpp-servers (maintained by both individuals and groups of enthusiasts)? If you have a clear vision for the implementation, you can develop your own "fork" individually or together with like-minded people.

[AlbanBedel]

@2743ab2 not having a framework for push notifications in the FOSS android ecosystem is definitely a long standing pain point. Sure if you only use a single messenger, and it implement its background sync correctly, you probably don't see a problem. But as you add more apps it start to crumble because all these background processes keep constantly waking your phone and all together they do consume quiet some power.

Now if you take a look at what unifiedphush is about you will see that it is basically an API, it doesn't mandate how the push system is to be implemented. So it neither lock you with a vendor, nor with a specific push transport technology.

I don't think anybody is asking to drop the current background sync implementation and switch to UP only, just to have optional support.

[astein248]

I'll bump this request - I'm trying to de-google and having a bunch of messengers syncing in the background is neither pleasant nor good for battery. I just set up my own ntfy server to handle unifiedpush notifications (as well as other push notifications from my own alerts), and it was very simple to do.

[white-gecko]

Here is some information on how the matrix client element does it: https://github.com/vector-im/element-android/blob/16ca26569f99df230888f78a8e2529c5e385ddda/docs/unifiedpush.md

[BigmenPixel0]

Any news?

[neutralinsomniac]

I am all for this, however, I believe in order to do this properly, you would need something on the server side to actually publish notifications to the client(s). There is a zero percent chance that Telegram themselves will do it, so that leaves someone to write a middle-layer service that would run on a server somewhere, logged into your Telegram account, that will handle pushing notifications to your other client(s) when you get a new message. It's not as easy as setting this up on something like Matrix, where they control the server implementation and have a system in place for setting up arbitrary notification systems already.

So the tldr; this would require a server component to be written in addition to client changes, which is probably out-of-scope for this project.

[ildar]

I agree. UnifiedPush has no chance to work on Telegram. Yet FOSS implementation of FCM would help a lot. There are some in the hood.

[danog]

Telegram allows sending push notifications to arbitrary HTTPS endpoints using the web push protocol: https://core.telegram.org/api/push-updates (type=10).

I might give a go at writing a web-push=>unifiedpush proxy this weekend, which should solve this issue.

[danog]

Turns out no proxy is needed, Telegram's implementation is lax enough and doesn't check the return code of the push endpoint, for example pushing to ntfy works OK even if the return code is 200 and no Link header is present. Unifiedpush can be directly integrated in the app, no proxy required.

[martin-braun]

@2743ab2 Sorry if I sound harsh, but it seems you have not spend a dime looking into https://unifiedpush.org/, have you? This is not a "service", it's an open-source library that can be implemented in any app and it allows the user to decide themselves what push server to pick (self-hosted like ntfy or NextCloud or FCM etc.) by installing the proper push client on his/her device for it. The user could even select an open-source FCM push client (which could also be embedded, since it's not official proprietary stuff), but this makes no sense here, because people could just get the official Telegram app for that, if they really want Google to be involved.

Take a look:

The green parts are installed by the user, the app registers itself through his/her push client with the push server (own infrastructure) and passes the received endpoint to the Push Server, afaik. When the App Server wants to push the message, it just sends it to the endpoint, which points to your Push Server (or a proprietary Push Server like FCM, if the user has chosen to).

There is just a thing to be mind of: The "App-Server" would be the Telegram server in this case, which won't happen. So, there has to be something in between. Since we don't want to introduce another to-be-trusted dependency that our messages have to go through, the "App Server" needs to be a self-hosted gateway as well, so it really only makes sense to consider for self-hosting.

The benefit is huge though. For all apps that implement Unified Push you only have to have one Push Client and its notifications run through your infrastructure, your battery will thank you without relying on any other dependency.

[danog]

The only things Telegram-FOSS has to do to use unified push are:

1) Get a push endpoint from the library 2) Generate ECDH key material according to RFC-8291 3) Call account.registerDevice with the generated public key and the received endpoint 4) Decrypt received notifications first with ECDH, then with MTProto 5) Handle decrypted notifications.

[martin-braun]

@danog I wasn't talking to you, I was talking to @2743ab2, but it's nice to get some more insight about what actually need to be done on Telegram-FOSS by someone who is even deeper into the specs, I scratched the surface so far.

I actually missed your last comment. Wow this is good news, but kinda unsupported. A proxy would make this stable and prevent things from breaking for long if they decide to change something. But it's great to know that this would work without a proxy.

[spazziale]

Any update?

[joshcangit]

Not sure if this is in the Todo list. Hope someday Telegram FOSS uses UnifiedPush.

[moradi-morteza]

Telegram allows sending push notifications to arbitrary HTTPS endpoints using the web push protocol: https://core.telegram.org/api/push-updates (type=10).

I might give a go at writing a web-push=>unifiedpush proxy this weekend, which should solve this issue.

hi, i want to send notification to my own telegram clone with fcm, but i can not recognize what should be the json that i send, can you help me? i can send remote message with data payload from fcm api but telegram need its own data payload

[c33s]

unified push is quite nice, element (f-droid) has the option to use unified push as notification system. they use https://ntfy.sh/ as default server but ntfy can easily be self hosted. fluffy chat also uses unified push. works quite nice.

[SolAstri]

is anyone working on implementing this? or am i free to fork and make a PR?

[ildar]

no doubt, yes, you can

[quqkuk]

is anyone working on implementing this? or am i free to fork and make a PR?

I have done some work to push to Telegram (https://github.com/quqkuk/Telegram-webpush) (that could be merged here too), currently it's not the best code (it uses Java's security API instead of using BoringSSL and there's not a lot of error handling), but most of the RFCs are implemented

[drizzt]

hi, if you want to test it I backported the patches from @quqkuk on Telegram-FOSS.

https://github.com/drizzt/Telegram-FOSS/tree/UnifiedPush

[gcvl]

hi, if you want to test it I backported the patches from @quqkuk on Telegram-FOSS. I rebased it with the last Telegram version, I removed the unneeded FOSS patches (all the patches that "fixes" standard telegram push, since my fork is ONLY to be used with UnifiedPush) and I updated it to be built on SDK 34:

https://github.com/drizzt/Telegram-FOSS-UnifiedPush

Thank you so much for this, I'm trying it right now! What are the recommended settings for the "keep alive service" and the "background connection" toggles? I already excluded it from battery optimizations, actually I disabled Doze altogether.

EDIT: I installed the UnifiedPush FCM Distributor (since I'm on microG but here no TG client is delivering notifications except Telegram X) and was able to register TG FOSS, let's see how it goes!

EDIT 2: I switched from the FCM Distributor to ntfy which seems to work flawlessly so far. Very good job, keep this beating!

[spazziale]

hi, if you want to test it I backported the patches from @quqkuk on Telegram-FOSS. I rebased it with the last Telegram version, I removed the unneeded FOSS patches (all the patches that "fixes" standard telegram push, since my fork is ONLY to be used with UnifiedPush) and I updated it to be built on SDK 34:

https://github.com/drizzt/Telegram-FOSS-UnifiedPush

I'm testing it and everything is working right now. Do you want to merge it upstream or keep a separate fork?

[gcvl]

hi, if you want to test it I backported the patches from @quqkuk on Telegram-FOSS. I rebased it with the last Telegram version, I removed the unneeded FOSS patches (all the patches that "fixes" standard telegram push, since my fork is ONLY to be used with UnifiedPush) and I updated it to be built on SDK 34:

https://github.com/drizzt/Telegram-FOSS-UnifiedPush

Moreover, are you willing to maintain both this fork and Mercurygram? Thanks!

[drizzt]

hi all, I cleaned up the patches, added a settings and proposed them as PR #715.

In the future, I'll only push stuff inside Mercurygram, that will include the features I use, and I'll open PR here with bug fixes for UnifiedPush.

[gcvl]

hi all, I cleaned up the patches, added a settings and proposed them as PR #715.

In the future, I'll only push stuff inside Mercurygram, that will include the features I use, and I'll open PR here with bug fixes for UnifiedPush.

That's great! Thank you for all this: given my long-standing problems with Telegram notifications, it's exactly what I've been helplessly looking for since ages!

[Gabr-F]

@quqkuk

I have done some work to push to Telegram (https://github.com/quqkuk/Telegram-webpush) (that could be merged here too), currently it's not the best code (it uses Java's security API instead of using BoringSSL and there's not a lot of error handling), but most of the RFCs are implemented

Hey, great work, you're going to try to have it added to Telegram itself? I didn't see a pull request there yet

[quqkuk]

I've been monitoring this issue in the last few days, thank you for the attention, but there are bad news I'll get to in a bit.

@Gabr-F said: Hey, great work, you're going to try to have it added to Telegram itself? I didn't see a pull request there yet

Yes, that was my goal - Instead of making a patch based upon Telegram-FOSS I wanted to base it on Telegram, hoping that even if it wouldn't get merged, applying it to this codebase would still be feasible without too much hassle.

@marc0x1 said: I'm testing it and everything is working right now.

Are you sure? I checked @drizzt's repo (thank you for the work btw) and it still has the Push Service Notification to keep the connection to Telegram's servers open, which means that notifications might be coming from that connection instead of the UnifiedPush distributor

I'm saying this because my patches never worked even for me - At some point I doubted my own RFC8188 message parser and since I had accepted that I would've had to rewrite the whole thing to follow the "guidelines" I had guessed while writing and debugging, I pulled Google's Tink which implements Message Encryption for Web Push (it's the same library that the UnifiedPush example uses) and it too threw an error about the record size field bearing an invalid value, which means that Telegram doesn't seem to return a valid encrypted WebPush payload (I'm ruling out it being a WebPush-through-UnifiedPush problem, as the UnifiedPush example treats the message received from the distributor as an encrypted WebPush payload that follows the RFC8291 spec). Due to other complications I ended up pausing development, but I intend on fixing this in the following weeks (I assume there must be a way, since Telegram Web does receive a payload it can read). I'm sorry I haven't communicated this well enough and I've let my repo die without mentioning anything (I intended to have some notice written in the README, but alas, I must've gotten lost while testing with Tink)

[drizzt]

@quqkuk In my tests I disabled push notification in options and I'm using a self-hosted ntfy. I see that Telegram app is not running and, as soon as I receive the POST from Telegram Server (sometimes it takes some minutes) to the correct ntfy topic, I receive the notification

[quqkuk]

@quqkuk In my tests I disabled push notification in options and I'm using a self-hosted ntfy. I see that Telegram app is not running and, as soon as I receive the POST from Telegram Server (sometimes it takes some minutes) to the correct ntfy topic, I receive the notification

Well, that's good news! I'll improve the code if I can, as it's clearly something that should not go in production and was there just to test things, but that will let me rest more easily

[drizzt]

@quqkuk ugh, you were right, it seems it's not parsing the message correctly (wrong size as you told me) and I'm investigating about that. I didn't notice that before, since I get the notifications since processRemoteMessage makes Telegram reconnects to the server, get the updates and return in standby. For this reason, I changed my branch, and my PR, to use a simpler implementation of UnifiedPush handler that only does that (by using Simple push instead of Web Push).

Hopefully it'll possible to fix your Web Push implementation some days, but in the meanwhile the simple one works

[gcvl]

Hopefully it'll possible to fix your Web Push implementation some days, but in the meanwhile the simple one works

JFYI Before this change it was perfectly working here, not anymore now. I had to re-enable the keepalive service (not the background connection, but still) to receive notifications and am not even sure they come consistently yet.

EDIT: it seems that re-enabling the keepalive did the trick but I'll be testing it throughout the day...

[quqkuk]

I think I know what's going on I suspected this, I've run a tcpdump and now I know for certain - Telegram's servers use an old draft of RFC8188, which passes the key and the salt as HTTP Headers instead of at the beginning of the payload. I've just sent this in as a bug report but it also means that implementing UnifiedPush requires a rewrite proxy (a Gateway, in UnifiedPush lingo) to decrypt the message before it reaches the Push Provider

[drizzt]

@quqkuk I think it'll require a proxy to be put BEFORE nfty (a proxy that directly receive Telegram messages) and that converts it to the real RFC 8291, since headers are stripped by nfty server and not forwarded

[quqkuk]

@quqkuk I think it'll require a proxy to be put BEFORE nfty (a proxy that directly receive Telegram messages) and that converts it to the real RFC 8291, since headers are stripped by nfty server and not forwarded

Yes that's what I wrote, or at least what I meant

[gcvl]

Hopefully it'll possible to fix your Web Push implementation some days, but in the meanwhile the simple one works

JFYI Before this change it was perfectly working here, not anymore now. I had to re-enable the keepalive service (not the background connection, but still) to receive notifications and am not even sure they come consistently yet.

EDIT: it seems that re-enabling the keepalive did the trick but I'll be testing it throughout the day...

Ok, this morning's build was working reliably with the keepalive service enabled; this afternoon's build is working with all disabled, don't know whether it's coincidental or you changed something! 👍

[drizzt]

@quqkuk ok, I created a simple proxy that appends the headers to the payload (using the same method as https://github.com/UnifiedPush/common-proxies/blob/1567aa9ea7fc6aef80162180f4e54208cd8784f9/gateway/webpush.go#L33-L42), so now we only need to change your java function in order to get the keys from the base64url headers and use it by using the old method in draft-ietf-httpbis-encryption-encoding-03. I have my proxy on top on ntfy at https://ntfy.belloworld.it, if you want to use it to implement the remaining stuff.

Hint: if you want to install the proxy on your setup, be sure to put the hostname inside /etc/hosts (in my case I have 116.203.117.243 ntfy.belloworld.it) so you don't need to do lots of dns queries

[gcvl]

From an user perspective: since right now the simple implementation is working like a charm, won't this new reworked one overcomplicate things?

[drizzt]

@gcvl yes, I changed something in the UnifiedPush handler (threads, postInitApplication, etc). if you can confirm it works also after 30+ minutes from the previous notification I'll remove the draft from the PR so they could merge it

[gcvl]

@gcvl yes, I changed something in the UnifiedPush handler (threads, postInitApplication, etc). if you can confirm it works also after 30+ minutes from the previous notification I'll remove the draft from the PR so they could merge it

Yes, I currently have it working, I tested with variable timespans, from some minutes to over 30 (screen off) and it got notifications near instantly every single time.

I'll tell you my setup though: I'm on Lineage + microG, Doze entirely disabled, Mercurygram unrestricted from battery optimizations with keepalive and TG background connection both disabled, ntfy connected to ntfy.sh and registered to FCM through microG (surprisingly I never ever managed to make the official TG receive notifications through microG, on all my degoogled phones it's the only app that just wouldn't reliably alert me no matter what I tried to do and all its other forks were no better).

I could do some tests with Doze fully enabled and battery optimization ON for MG or if you wish to remain on the safe side you can toggle ON the keepalive service as default so that it places just the persistent notification to stay active and running.

[gcvl]

Doze ON and battery optimization ON: the app fell asleep and didn't notify. Making other tests but I guess that we need either battery optimization OFF or the persistent notification (keepalive ON).

[drizzt]

Doze ON and battery optimization ON: the app fell asleep and didn't notify. Making other tests but I guess that we need either battery optimization OFF or the persistent notification (keepalive ON).

https://github.com/drizzt/Mercurygram#notes

Thank you for reporting, I hope it's clear enough and it's what you said

[gcvl]

Doze ON and battery optimization ON: the app fell asleep and didn't notify. Making other tests but I guess that we need either battery optimization OFF or the persistent notification (keepalive ON).

https://github.com/drizzt/Mercurygram#notes

Thank you for reporting, I hope it's clear enough and it's what you said

Yes, they're needed and very clear hints! Meanwhile Doze ON and battery optimization OFF: OK. Doing some more tests anyway.

[drizzt]

Doze ON and battery optimization ON: the app fell asleep and didn't notify. Making other tests but I guess that we need either battery optimization OFF or the persistent notification (keepalive ON).

https://github.com/drizzt/Mercurygram#notes Thank you for reporting, I hope it's clear enough and it's what you said

Yes, they're needed and very clear hints! Meanwhile Doze ON and battery optimization OFF: OK. Doing some more tests anyway.

if you can try the last apk I pushed some minutes ago. it should not change anything, but who knows 😵‍💫

[gcvl]

Doze ON and battery optimization ON: the app fell asleep and didn't notify. Making other tests but I guess that we need either battery optimization OFF or the persistent notification (keepalive ON).

https://github.com/drizzt/Mercurygram#notes Thank you for reporting, I hope it's clear enough and it's what you said

Yes, they're needed and very clear hints! Meanwhile Doze ON and battery optimization OFF: OK. Doing some more tests anyway.

if you can try the last apk I pushed some minutes ago. it should not change anything, but who knows 😵‍💫

Sure! Updated, I'm doing some tests and letting you know.

[gcvl]

Doze ON and battery optimization ON: the app fell asleep and didn't notify. Making other tests but I guess that we need either battery optimization OFF or the persistent notification (keepalive ON).

https://github.com/drizzt/Mercurygram#notes Thank you for reporting, I hope it's clear enough and it's what you said

Yes, they're needed and very clear hints! Meanwhile Doze ON and battery optimization OFF: OK. Doing some more tests anyway.

if you can try the last apk I pushed some minutes ago. it should not change anything, but who knows 😵‍💫

It's great! It works just like the earlier one (with battery optimizations OFF) 😊🫡

[gcvl]

Ok, I happily switched to Mercurygram from Telegram X, I'm sold 😊 It's your project, but as long as it works like this I'd recommend to just keep it simple and usable for everyone; ottimo lavoro, continua così! 👍

[quqkuk]

I'm waiting for a reply on UnifiedPush/common-proxies#64. That will hopefully give us a straightforward way to setup a proxy that bridges between WebPush Draft 4 "aesgcm" format payloads and UnifiedPush

[karmanyaahm]

I'll preface this by saying that I love Mercurygram and @quqkuk I love your PR. But on the UP side we have previously discussed the merits of sync on push vs a webpush proxy: https://github.com/UnifiedPush/wishlist/issues/15#issuecomment-1560678223

We're looking at this from the perspective of simple self hosters, not just UP enthusiasts, i.e. people that might be using their own NextPush or Prosody UP plugins.

Sync on Push: they don't need to host common proxies (or a similar gateway doesn't have to be built into all the distributors), and they don't need to use a public gateway (privacy).

With a WebPush proxy: Maybe one or two fewer requests to the server? The phone still has to wake up and talk to the server. Additionally, we endorse and support an outdated specification which is (imho) a bad thing.

@quqkuk I'm not totally familiar with the Telegram API, are there any advantages here in particular of decryption instead of sync on push?

@p1gp1g what do you think?