Request did not validate Invalid bearer token: Auth expired

cancanf3 commented 6 years ago

Hello,

Disclaimer: Please bear with me. It is my first time implementing my own web push service on a REST API. I will do my best to specify my problem and explain my implementation's logic.

I am using Python 3.6, Django 2.0, pywebpush 1.6.0 and vapid 1.3.0

I generated my new keys on Saturday and they worked fine the next day. But Now I am trying to push a message and I am getting the following error today (two days after I generated them).

I have read that the max amount of time for the vapid_claims to expire is 24 hours. Does this mean that I have to generate a VAPID keypair every 24 hours? if yes, Any recommendations to do it on run time?

Error when subscribing with Firefox:

 Push failed: <Response [401]>: {"code": 401, "errno":                                                                                                                                                                    
 109, "error": "Unauthorized", "more_info": "http://autopush.readthedocs.io/en/latest/http.html#error- codes"                                                                                                                                                                   
 , "message": "Request did not validate Invalid bearer token: Auth expired"}

Error when suscribing with Chrome:

 Push failed: <Response [400]>                                                                                                                                                                                               <HTML>
 <HEAD>
  <TITLE>UnauthorizedRegist </TITLE>
 </HEAD>
 <BODY BGCOLOR="#FFFFFF" TEXT="#000000">
  <H1>UnauthorizedRegistration</H1>
  <H2>Error 400</H2>
 </BODY>
</HTML>

This is the code I am using:

    @api_view(["POST",])
    def pushMessage(request, format=None):
        for user in PushClient.objects.all():
            try:
                webpush(subscription_info={"endpoint":user.endpoint, "expirationTime": user.expirationTime,
                "keys" : {"p256dh" : user.p256dh, "auth": user.auth}},data=request.data,
                vapid_private_key=VAPID_PRIVATE_KEY,vapid_claims=VAPID_CLAIMS)
            except Exception as e: 
                logger.error(e)
                user.delete()
                continue

        return Response(status=status.HTTP_200_OK)

This are my VAPID_CLAIMS. I used py-vapid 1.3.0 to generate my keys. Then I used toString64 using node to encode them.

VAPID_PRIVATE_KEY = 'TSH+PiQph1DlfRJjPS/j4yehCAldPy20/ZyCkIjTamU='
VAPID_PUBLIC_KEY = 'BG+xOxh5lIEgFb/3M+1ROVvHQFJnCWuZD+DRELiwTb4nHGOFLPMtDH99/cFeG/sdUnoDMe278S+cUWTizMGjeHU='

VAPID_CLAIMS = {
"sub": "mailto:****@orlandocitysc.com"
}

I don't know if this would help but here it is. This is the subscription one of my client has registered. it is one of the objects I got after I do PushClient.objects.all().

{
        "id": 129,
        "endpoint": "https://fcm.googleapis.com/fcm/send/dEuBF4q-YZk:APA91bGK3VrHwz0M_2TMJUdYEhCGfnmGRwo5e1XOHqUA0bQShbIodU6uVAGfWEDj3Lcn1Wtd3NtbTzREl_VruWXE-jbCVv9jLlgSxzZj4TnNZzmXujyvRF3_RiIH1UUNArGSbbRdsgfk",
        "expirationTime": null,
        "p256dh": "BHtmNQqwqQXpYcLB3W6nd1snkhYYj7qFVGy7D40U58tqr9jsSsLMEPGLeLckhhZzmvTFPw31eI6TIvYLLWVct-M=",
        "auth": "n8-gO34skDelvWtE8CO_9Q=="
    }

jrconlin commented 6 years ago

It might help to take a few steps back and understand what VAPID is supposed to be. Webpush, by design, is pretty anonymous. The folks running the Webpush servers don't have to know who the people sending a push notification are, nor do they have to know who gets that notification. There's a lot of good reasons for that (Yay! Privacy!), but it does make things a bit hard if, say, someone creates a subscription provider that's doing something bad. Ideally, the push service should be able to reach out to that provider and say "We noticed that you're doing X, it'd be better if you did Y. Thanks!"

That's kinda where VAPID steps in. VAPID has, really, three mandatory fields. A field to say who should get the header (the "aud"ience), who's sending the subscription update (the "sub"scriber) and how long this, particular VAPID certificate should live (the "ttl" or time to live). The cert is then signed using a key pair so that nobody in between can mess with things and sent off to the push service provider. As an added benefit, the user agent (the browser) can include the VAPID public key in the subscription request for a new push endpoint, and that endpoint then gets locked to the VAPID key pair. This means that you have to use the same VAPID key pair to sign future subscription updates, or else the Push Service rejects them.

So, ok, that was pretty long and complicated, so let me put it a slightly different way.

The VAPID key pair you generate are like a lock and key. When you set up your subscription, you can provide the Push Service with a copy of the lock you have. This makes sure that, say "badguy@evilonastick.com" doesn't send in updates pretending to be you.

When you send an update (and kinda where the lock and key thing breaks down) you provide a tag that says "Use this lock. Here's my email if you need to reach me, oh, and this tag is going to be valid until tomorrow". That last bit is the "ttl". That's so that the push service doesn't hang on to really old tags and tries to contact you via an email you haven't had since everyone was talking about how cool iPods were.

What that also means is that if you toss your VAPID key pair, you're effectively tossing your lock and key set, and all the subscription URLs you had before that used that lock will no longer work.

So, what's probably happening is that you're accidentally tossing the lock and key when all you really wanted was to tell the server, "Hey, make sure you keep a fresh record of how to contact me."

I'll note that Google makes VAPID a lot less "voluntary", but the same concepts still apply.

So the guidance I'd give are: 1) Make a VAPID key pair, and KEEP IT SAFE. Understand that while you can create a new key pair at any time, you'll have to get everyone to resubscribe, and that's going to be a pain. 2) The TTL is an indicator to the Push Server how long that they can cache your email address. If you're feeling paranoid, set the value low to prevent potential replay attacks. If you don't want to sign every single outbound subscription and instead save a bit of CPU, set it higher and reuse the VAPID header. 3) Use an email address that is valid. You can also include additional information that may help Ops folks figure out what's going wrong at 3AM, when these sorts of problems happen.

Does that help?

cancanf3 commented 6 years ago

Thanks for the great explanation of VAPID. I was a little bit confused about it. But now it is clear.

I wanted to provide you with my key pair to check if they were in the correct format and if the process I did to create them was a proper one. I will generate them again to keep my secret key secure.

However, I am still kinda lost with the responses I am getting from Chrome and Firefox's push services. I made sure the client was receiving the correct public key from the server and the client properly subscribe to the push notification.

jrconlin commented 6 years ago

Well, the 401:109 error you got from mozilla is probably due to the VAPID cert TTL expiring. Normally, the webpush function sets this to the current time + 24 hours for you, but you can always specify it yourself as something like:

vapid_claims = { 
   "sub": "mailto:foo@example.com",
   "ttl": int(time.time() + 86400),
}

(One other silly thing you can check is your server's time. It's possible that it may be wrong and sending a bad time, or it could be sending a local time instead of UTC, or a bunch of other things.)

cancanf3 commented 6 years ago

Yes the server was setup with local time. I switched back to UTC. The new keypair has worked so far.

However, I had some problems generating the key pair. I got a lot of "cannot deserialize key" but I ended up upgrading to py-vapid 1.31 and it worked. ( I was using 1.3.0)

I also setup the TTL to 6 hours. I saw one client (I was testing on) was presenting errors after 6 - 8 hours after being registered (my backend deletes any client from the DB that presents errors to send a push notification to its end). Following your explanation, The user agent subscribes with the current vapid (lock) of the moment, meaning that if the vapid has expired, the client must update its subscription? Correct me if I am wrong.

*I was not able to get more information from the error after 6-8 hours because that client was using Chrome, I have now subscribed a Mozilla client to get better details of the error.

Thank you for your help! I have found this technology quite interesting. I am hoping to get a better understanding to get the best of it.

jrconlin commented 6 years ago

Again, the TTL expiration is for the specific VAPID cert header, not your key. You should hold on to your VAPID key pair for as long as you want your subscription endpoints to work.

If you set the TTL for, say, one hour from now, you can reuse that same VAPID header on all updates until that TTL. The max you can set a TTL for is 24 hours.

cancanf3 commented 6 years ago

I meant the client, the person subscribing to the push service. Keys remain unchanged. But is it normal that after a couple of hours I start receiving this?:

{
  "code": 401, "errno": 109, 
  "error": "Unauthorized", 
  "more_info":  "http://autopush.readthedocs.io/en/latest/http.html#error-codes", 
  "message": "Request did not validate Invalid Authorization Header"
}

and I have to go to the client and re-subscribe again.

jrconlin commented 6 years ago

No, normally you should not see that unless there's something off with the Authorization header crypto. The link is still valid, and no re-registration is required. (only a 404 or 410 would require a re-registration)

If you resubmit with a new VAPID cert (same keys, just a new VAPID header) it should work. I've seen some failures due to library incompatibilities.

cancanf3 commented 6 years ago

What version of the libraries would you recommend? I am using pywebpush 1.6.0 and vapid 1.3.1

jrconlin commented 6 years ago

This is where things get horrible. It's not those libraries. It's the ones that they may be talking to. In particular the openssl libraries, or the math libraries that they may be reliant on, or the transcoding libraries that convert the byte array. There's also the problem that EC256 has particular points on it's curve that are invalid, but some implementations of libraries don't check for those points while others do.

To be honest, treating that error as "Reply hazy, try again", as well as some level of caching working headers is probably the sanest approach.

cancanf3 commented 6 years ago

Sounds good. I will try that. Thank you

web-push-libs / vapid

Request did not validate Invalid bearer token: Auth expired #66