VAPID support - Githubissues

pimeys commented 7 years ago

I'm already having the signer part done. What I still don't know is that even when Google says you can use the fcm endpoint without an API key if using VAPID signing, will the request content still be in JSON format or will it work like the standard where content is just http-ece encrypted data.

Anyways, we should get the support merged to master soon.

Ogeon commented 6 years ago

Hi! I'm working on a project where I need VAPID support (seems to be pretty much mandatory now). I tried using the vapid branch, but I'm getting "incompatible objects" when it tries to serialize the public key with POINT_CONVERSION_UNCOMPRESSED. I haven't yet tried to change it to compressed, which serializes without problem, but I may end up doing that next time I get the chance to work on it.

It's really nice to have a ready made solution for it, since it's not too easy to get everything right. 👍 Took me too many hours to even generate a valid public key that the browser would accept, and I haven't checked if the ones that the vapid branch want are valid yet.

Either way, I'm wondering if you intent to keep working on it? Just asking so I know what I can expect. 🙂

pimeys commented 6 years ago

Short answer: The situation is that we don't need VAPID in our SDK right now, so I cannot put company time to this ticket. I need to find a slot later and do it in the evenings. Would be happy to do that though, and happy to get some help!

Long answer: I've seriously been trying to find information about web push and how it works. I've read all the RFC drafts and implemented the different versions, tried to get any of them to work with Firefox and Chrome. Also did the same with VAPID, I remember vaguely that this patch here worked like a charm with Firefox but then, how I understood but it's not really strictly written anywhere, you can skip using Firebase API key with a VAPID token. The latter I never could get to work, and it would make our own architecture much nicer if we didn't need to ask FCM API keys from our customers. None of the other web push libraries I've found has either recent RFC draft implemented or their code is very hard to follow and doesn't follow nice guidelines.

Ogeon commented 6 years ago

I see. I could give you a hand whenever I can get some time for it. I did manage to create a subscription with only a public key from the web-push JavaScript module, so I guess there could be some information to gather from there, but I haven't tried to send a push message. I could maybe start digging around there some day.

Den mån 22 jan. 2018 09:32Julius de Bruijn notifications@github.com skrev:

Short answer: The situation is that we don't need VAPID in our SDK right now, so I cannot put company time to this ticket. I need to find a slot later and do it in the evenings. Would be happy to do that though, and happy to get some help!

Long answer: I've seriously been trying to find information about web push and how it works. I've read all the RFC drafts and implemented the different versions, tried to get any of them to work with Firefox and Chrome. Also did the same with VAPID, I remember vaguely that this patch here worked like a charm with Firefox but then, how I understood but it's not really strictly written anywhere, you can skip using Firebase API key with a VAPID token. The latter I never could get to work, and it would make our own architecture much nicer if we didn't need to ask FCM API keys from our customers.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/pimeys/rust-web-push/issues/2#issuecomment-359355688, or mute the thread https://github.com/notifications/unsubscribe-auth/AANvzXNLIjEBzcTj24eRxHl19F4rdp4uks5tNEergaJpZM4QSUg0 .

pimeys commented 6 years ago

Yeah, and if you find any recent blog posts, RFC's or tutorials, they would be really handy. Firefox at least is a bit easier, but Google seems to act the web push doesn't exist at all, and whatever I found from them is already so old the information has no value.

What I wanted to do here is to have at least one web push library with clean code, an easy interface and a way to support different drafts, because web push has been a moving target since beginning. What's funny is that here we implement the draft 3 of the RFC's. The latest is something like draft 7 or draft 8, but I never could get any of them to work with either Firefox or Chrome. And there is almost no articles about these newer drafts, so my guess is the whole technology is not a priority to either one of the browsers.

Ogeon commented 6 years ago

Yeah, I'll see what I can find. This one seems to be pretty up to date, but I haven't read the protocol part: https://developers.google.com/web/fundamentals/push-notifications/

I could follow it without any major problems.

Den mån 22 jan. 2018 09:46Julius de Bruijn notifications@github.com skrev:

Yeah, and if you find any recent blog posts, RFC's or tutorials, they would be really handy. Firefox at least is a bit easier, but Google seems to act the web push doesn't exist at all, and whatever I found from them is already so old the information has no value.

What I wanted to do here is to have at least one web push library with clean code, an easy interface and a way to support different drafts, because web push has been a moving target since beginning. What's funny is that here we implement the draft 3 of the RFC's. The latest is something like draft 7 or draft 8, but I never could get any of them to work with either Firefox or Chrome. And there is almost no articles about these newer drafts, so my guess is the whole technology is not a priority to either one of the browsers.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/pimeys/rust-web-push/issues/2#issuecomment-359358496, or mute the thread https://github.com/notifications/unsubscribe-auth/AANvzdjYgqmnLCRt486zJjBaGVq52Llsks5tNEsCgaJpZM4QSUg0 .

pimeys commented 6 years ago

Cool, so we finally have some documentation. As I remember correctly, I could get this branch to work with Firefox, but not with Chrome. I might need to try again. The deal should be, that IF we provide a VAPID key, then the push API should be the same for both browsers. And that could free us from the silly limitations, such as the smaller payload limit for Chrome and the requirement for FCM keys.

Ogeon commented 6 years ago

Yes, it seems like they are finally converging into some kind of common standard. According to the parts I have read, it should be the same for both now. The problem I had with it is that it has conveniently left out some details for the key pair. It may be hidden in the protocol description, though.

Den mån 22 jan. 2018 10:34Julius de Bruijn notifications@github.com skrev:

Cool, so we finally have some documentation. As I remember correctly, I could get this branch to work with Firefox, but not with Chrome. I might need to try again. The deal should be, that IF we provide a VAPID key, then the push API should be the same for both browsers. And that could free us from the silly limitations, such as the smaller payload limit for Chrome and the requirement for FCM keys.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/pimeys/rust-web-push/issues/2#issuecomment-359369532, or mute the thread https://github.com/notifications/unsubscribe-auth/AANvzQ_yqpvIjd5wQXUofGYuTSFztoeWks5tNFYtgaJpZM4QSUg0 .

pimeys commented 6 years ago

Ok, so I did a quick test where we are with this branch.

1) It compiles, yay! 2) I use the push codelab as my client side project. Change the main.js to your application public key. 3) Store the given subscription info to a JSON file. 4) Test sending with the example vapid keys, I just pushed a name change so please do a pull first:

./target/debug/examples/simple_send --vapid_key ./examples/vapid_key1.pem -f examples/mozilla.json -p "Test"
pub: BBl8r1l9YxYQ9i60PWX36ofzmcAcWx_EPeJImQHXXwRyDfu3H1HICoPmRGNEGffO0y5bUowRp9cw2Sz2hBnZxuE
OK

./target/debug/examples/simple_send --vapid_key ./examples/vapid_key2.pem -f examples/m
ozilla.json -p "Test"
pub: BP8ni2I3MoBiGABCc35OAEJ2AONEo6-fLO__Hbu2syiTfc5sUQdGqpHAi_Y_i52ZMy6_gvrtIcVdtE9dGM5sMow
ERROR: Unauthorized

So this seems to work on Firefox, but not on Chrome. I tested chrome the last time on November, so things might've changed.

Ogeon commented 6 years ago

Then it's probably not too far off, I guess. But I wonder where my openssl error came from... I'll have to investigate it later.

pimeys commented 6 years ago

Hey @Ogeon if you're still following this, I installed Chromium and did some tests. The best I can get out from FCM is an HTML error page, so I needed to hack the library a bit to see what they reply:

"<HTML>\n<HEAD>\n<TITLE>UnauthorizedRegistration</TITLE>\n</HEAD>\n<BODY BGCOLOR=\"#FFFFFF\" TEXT=\"#000000\">\n<H1>UnauthorizedRegistration</H1>\n<H2>Error 400</H2>\n</BODY>\n</HTML>\n"

Googling a bit I'm not the only one with the same problem: https://github.com/web-push-libs/web-push-php/issues/63

Ogeon commented 6 years ago

That's strange, especially since it's a fresh install, and the linked chromium bug seems to be fixed (unless it's back again). I'll see if I can get any further today.

anaskhan96 commented 6 years ago

Is the vapid branch working? From what I gather, it seems to be working on Firefox which is a good start. Could there be some documentation provided in the branch's README?

Ogeon commented 6 years ago

I haven't checked in a while (took a break from the project where I'm using it to focus on other things), but it didn't work too well when I did. The information online did not make it easy to figure out why, but that may have changed by now.

pimeys commented 6 years ago

The VAPID branch works with Firefox, but for some reason I can't get those requests through to Chrome. How it SHOULD work is that you just use the whole subscription uri as-is, adding a VAPID header and content such as you do with Mozilla's AutoPush. But I just can't get that to work. What I suspect here is Google doesn't want us to use VAPID, but instead they want us to create FCM credentials for them to track.

We still don't need VAPID support here, but if anybody wants to have a shot with this, please go ahead.

pimeys commented 6 years ago

The funny thing is how Google's troubleshooting tells you to try the requests with Firefox to get better error responses. Of course with Firefox all of this works...

https://developers.google.com/web/fundamentals/push-notifications/common-issues-and-reporting-bugs

pimeys commented 6 years ago

Been debugging the whole day, trying different base64 encodings, trying to change the separator from the crypto headers from ; to ;, updated openssl to 0.10 branch and my computer clock is synced with ntp. Firefox just works, chrome gives

"<HTML>\n<HEAD>\n<TITLE>UnauthorizedRegistration</TITLE>\n</HEAD>\n<BODY BGCOLOR=\"#FFFFFF\" TEXT=\"#000000\">\n<H1>UnauthorizedRegistration</H1>\n<H2>Error 400</H2>\n</BODY>\n</HTML>\n"

There seems to be lots of cry around the internet for this error, read many discussions and solved nothing. The library now works without VAPID, you can also take this branch and set the vapid key for requests going to Firefox. Chrome needs an FCM account to work for now. There's some small difference in the encryption what FCM doesn't tolerate. I'll buy a beer for the person who finds it.

pimeys commented 6 years ago

@anaskhan96 The docs are here :) https://pimeys.github.io/rust-web-push/v0.2.2/web_push/index.html

anaskhan96 commented 6 years ago

@pimeys 400 Unauthorized Registration is something I've encountered before while working with VAPID in Go, this discussion in which I participated in seems to resolve one such instance. If you follow the link, you'll find the issue had originated between different base64 encodings (for p256dh and auth) in the client and server side. base64 in the client one and base64url in the server, and vice versa.

However, if it is related to FCM, other web push packages would have different encryption methods for the two browsers. For example, webpush-go uses the same method and works on both browsers.

This one is probably off by a long shot but the error might be in the generation of the VAPID keys too. I wrote a small function a while ago which used to work for me. However the error code given for incompatible VAPID keys is 401, but this might be worth checking too.

I'll take a look at the branch when I find the time :)

pimeys commented 6 years ago

Spent the whole day debugging this, and found that the Crypto-Key header missed the p256ecdsa parameter, which should be the VAPID public key. Of course that didn't work in the end, but spent some time refactoring the system. Now if you run the simple_send example with a chrome subscription info, you get some prints:

> ./target/debug/examples/simple_send --vapid_key ./examples/vapid_key1.pem -f ./examples/chrome.json -p "moi"

JwtClaims { aud: "https://fcm.googleapis.com", exp: 1521698346 }
pub: BBl8r1l9YxYQ9i60PWX36ofzmcAcWx/EPeJImQHXXwRyDfu3H1HICoPmRGNEGffO0y5bUowRp9cw2Sz2hBnZxuE
[("Authorization", "vapid t=eyJhbGciOiJFUzI1NiIsInR5cCI6IkpXVCJ9.eyJhdWQiOiJodHRwczovL2ZjbS5nb29nbGVhcGlzLmNvbSIsImV4cCI6MTUyMTY5ODM0Nn0.x0MPBYuyMw9JYYF+sWpr4i7eNKtZzOgJtcqoGe//uDW7INzay12lYBlPZj7yJwj2aH1PJLQvFf1refXBwle/3A,k=BBl8r1l9YxYQ9i60PWX36ofzmcAcWx/EPeJImQHXXwRyDfu3H1HICoPmRGNEGffO0y5bUowRp9cw2Sz2hBnZxuE"), ("Crypto-Key", "keyid=p256dh; dh=BOb9fCSZBLV_mkrZtZdYw0C4GhVW2oznUvsJDpZ2d2-b8SVMeNIVuI4RYYoivONf529LOMqVwQqvDKJsUQzeFq0; p256ecdsa=BBl8r1l9YxYQ9i60PWX36ofzmcAcWx/EPeJImQHXXwRyDfu3H1HICoPmRGNEGffO0y5bUowRp9cw2Sz2hBnZxuE"), ("Encryption", "keyid=p256dh; salt=2zK7r-AKUymPQXW7VQFb6w")]
Request { method: Post, uri: "https://fcm.googleapis.com/fcm/send/dxJMrMAC-x0:APA91bEc6tbXc17tjlys3usScdaB1IBdGW4SQAh_KaRrEaBfS89wSdgwXaEpEAYLU-EnPWMu84IIdiurizl1-Okprn1HGfpOT1A3GBl3avRdwq3pqU4SQD0NACJpGFUjDNIMznOz5R8c", version: Http11,
remote_addr: None, headers: {"Content-Encoding": "aesgcm", "Content-Length": "3816", "Authorization": "vapid t=eyJhbGciOiJFUzI1NiIsInR5cCI6IkpXVCJ9.eyJhdWQiOiJodHRwczovL2ZjbS5nb29nbGVhcGlzLmNvbSIsImV4cCI6MTUyMTY5ODM0Nn0.x0MPBYuyMw9JYYF+sWpr4i7eNKtZzOgJtcqoGe//uDW7INzay12lYBlPZj7yJwj2aH1PJLQvFf1refXBwle/3A,k=BBl8r1l9YxYQ9i60PWX36ofzmcAcWx/EPeJImQHXXwRyDfu3H1HICoPmRGNEGffO0y5bUowRp9cw2Sz2hBnZxuE", "Crypto-Key": "keyid=p256dh; dh=BOb9fCSZBLV_mkrZtZdYw0C4GhVW2oznUvsJDpZ2d2-b8SVMeNIVuI4RYYoivONf529LOMqVwQqvDKJsUQzeFq0; p256ecdsa=BBl8r1l9YxYQ9i60PWX36ofzmcAcWx/EPeJImQHXXwRyDfu3H1HICoPmRGNEGffO0y5bUowRp9cw2Sz2hBnZxuE", "Encryption": "keyid=p256dh; salt=2zK7r-AKUymPQXW7VQFb6w"} }
Err(BadRequest(Some("<HTML>\n<HEAD>\n<TITLE>InvalidParameters</TITLE>\n</HEAD>\n<BODY BGCOLOR=\"#FFFFFF\" TEXT=\"#000000\">\n<H1>InvalidParameters</H1>\n<H2>Error 400</H2>\n</BODY>\n</HTML>\n")))
ERROR: BadRequest(Some("<HTML>\n<HEAD>\n<TITLE>InvalidParameters</TITLE>\n</HEAD>\n<BODY BGCOLOR=\"#FFFFFF\" TEXT=\"#000000\">\n<H1>InvalidParameters</H1>\n<H2>Error 400</H2>\n</BODY>\n</HTML>\n"))

The private key used in the call is:

And the public key in our javascript test app:

const applicationServerPublicKey = 'BBl8r1l9YxYQ9i60PWX36ofzmcAcWx_EPeJImQHXXwRyDfu3H1HICoPmRGNEGffO0y5bUowRp9cw2Sz2hBnZxuE';

Tomorrow I hopefully have still some time to continue debugging.

pimeys commented 6 years ago

Status update: so I was basically refactoring the VAPID api, allowing custom claims to be set with a nice interface. Now when I try it with Firefox, it just works. If I go with chrome, I get InvalidParameters http400. This doesn't make any sense. The payload is identical with Firefox, the crypto is the same, headers are set. All the documentation tell different things how this should work, the Authorization header should either start with vapid then including the signature, or how Google says it, with WebPush including the signature. vapid works for Mozilla, WebPush works for none.

The old GCM api wanted to have the payload as json, including registration_ids and raw_data as base64 encoded. Is this still the case with FCM+VAPID? This is how it's now set up, and the exp and aud are set automatically here.

Sometimes I really think web is horribly broken...

anaskhan96 commented 6 years ago

I think there might have been a critical update on Chrome or FCM's side. Maybe their release notes might be of some help. I can't think of any other reason after all this as to why it's not working.

pimeys commented 6 years ago

Ok everybody, this finally works and is ready for wider beta testing. Been spending time here and having "fun" with the 10 different RFC's and the python/go/php implementations trying to find hints what might be wrong. In the end, the last stretch was that VAPID with aesgcm should have only the signature in the Authorization header, it should say WebPush instead of vapid and the base64 encoding should be URL_SAFE_NO_PAD, not STANDARD_NO_PAD.

The next step for this repo is to support aes128gcm, which I'll add eventually.

Now how to test:

Get the https://github.com/GoogleChromeLabs/web-push-codelab/tree/master/completed/08-push-subscription-change and run a web server in the directory, such as python3 -m http.server
Edit scripts/main.js and change the applicationServerPublicKey to BBl8r1l9YxYQ9i60PWX36ofzmcAcWx_EPeJImQHXXwRyDfu3H1HICoPmRGNEGffO0y5bUo wRp9cw2Sz2hBnZxuE.
Store this private key to a file such as vapid_key.pem:

Open the web push client page, subscribe with the button, copy the subscription info to a file such as subscription_info.json
Compile the example client: cargo build --examples
Send yourself a test push: ./target/debug/examples/simple_send --vapid_key vapid_key.pem -f subscription_info.json -p "Test" -t 60

Next things: documentation, private key generation helper or docs how to do it with OpenSSL and moving forward to aes128gcm.

Ogeon commented 6 years ago

Awesome! I'll see if I get the chance to try it soon.

pimeys commented 6 years ago

Yeah, I'm writing tests, docs and doing a bit of refactoring so the API doesn't change that much when we get the working aes128gcm. Read the simple_send.rs to understand how the vapid part works, that's not documented yet.

pimeys commented 6 years ago

I consider this done, merged to master.

Ogeon commented 6 years ago

Thank you for this! :tada: I didn't get enough time to try it before you were done, with Easter activities and such in the way, but now I want to continue with my project! :slightly_smiling_face: Can't wait for the weekend

pimeys commented 6 years ago

The docs are now here https://docs.rs/web-push/0.3.0/web_push/

pimeys commented 6 years ago

0.3 also out :)

Ogeon commented 6 years ago

Just dropping by to say that I got it working in my project! :+1:

pimeys / rust-web-push

VAPID support #2