search

web-push-libs / web-push

Web Push library for Node.js
Other
3.2k stars 306 forks source link

403 (Forbidden) FCM Errors #457

Open cbratschi opened 5 years ago

cbratschi commented 5 years ago

NOTE: Please test in a least two browsers (i.e. Chrome and Firefox). This helps with diagnosing problems quicker.

Setup

Please provide the following details, the more info you can provide the better.

Operating System: Linux Node Version: 11.x web-push Version: 3.3.4

Please select any browsers that you are experiencing problems with:

Please list the browsers you are have tested this, including the version of the browser (i.e. Chrome Beta, Firefox Beta etc).

Problem

Please explain what behaviour you are seeing. Some of our messages sent to the FCM endpoint (https://fcm.googleapis.com/fcm/send/*) fail with 403 (Forbidden). Because we are not removing such failed token from our database the amount keeps increasing. Currently about 2% of all our web push tokens are affected, more than 1000 tokens. These are all VAPID tokens.

Does anyone else have similar issues? Do you discard such failing tokens?

Is this documented anywhere by Google?

Expected

Please explain what you expected to happen

Features Used

Example / Reproduce Case

Please provide a code sample that reproduces the issue. If there is a repository that reproduces the issue please put the link here.

Other

Please put any remaining notes here.

marco-c commented 5 years ago

I haven't heard of this problem before. I think you'll have to file a bug for FCM.

rohit-gohri commented 5 years ago

Is this documented anywhere by Google?

https://firebase.google.com/docs/reference/fcm/rest/v1/ErrorCode

cbratschi commented 5 years ago

I informed Google but they could not analyze my failing tokens. It seems all affected tokens are from April to June 2019. To be confirmed soon.

cbratschi commented 5 years ago

Yes, only new tokens are affected.

cbratschi commented 5 years ago

pywebpush made some VAPID exp adjustments: https://github.com/web-push-libs/pywebpush/issues/100

They had the same problem.

marco-c commented 5 years ago

@cbratschi @renegadeHomie can you share the code you are using to send the push notifications with web-push? I can check if it could be the same problem as pywebpush, but I doubt it (as far as I can see, we regenerate the expiration on every call to sendNotification).

cbratschi commented 5 years ago

Hi @marco-c,

We are using an Express service i.e. the module is loaded once and stays in memory. About 10 web pushes are sent a day to about 50k subscribers. We are using 4 child processes to send the pushes as fast as possible.

Here is the central part of the push code:

    const payload = typeof opts.payload === 'string' ? opts.payload:JSON.stringify(opts.payload);
    const optionsVapid = {
        //VAPID
        vapidDetails: {
            subject: opts.vapid.subject,
            publicKey: opts.vapid.publicKey,
            privateKey: opts.vapid.privateKey
        },

        //GCM (or FCM without VAPID)
        gcmAPIKey: opts.gcm,

        //others
        TTL: 0
    };
    const optionsNoVapid = {
        vapidDetails: null,
        gcmAPIKey: optionsVapid.gcmAPIKey,
        TTL: optionsVapid.TTL
    };

The non-VAPID version is used for old GCM tokens migrated to FCM.

Push loop where options points to the array above (array instance is static):

        const subscription = {
            endpoint,
            keys: {
                p256dh: item[2],
                auth: item[3]
            }
        };

        const push = webpush.sendNotification(subscription, payload, options);

        push.then(() => {
           ...
        }, err => {
          ...
        });

Hope this helps.

marco-c commented 5 years ago

I'm assuming the failing ones are all VAPID ones like you said in the first comment, correct?

Could you try running https://github.com/web-push-libs/web-push/blob/c8bb8b406819f19d6f0c0452322469a792ef494b/src/vapid-helper.js#L128 and seeing if the expiration is correct? Is your system clock set wrongly by any chance?

cbratschi commented 5 years ago

Yes, these are all VAPID ones, since April 2019. Older tokens seem to work fine. But not all new tokens are affected.

Will check the server time now.

cbratschi commented 5 years ago

The server time is fine.

cbratschi commented 5 years ago

The Java library has issues too: https://github.com/web-push-libs/webpush-java/issues/57

In my opinion this is a Firebase bug but Firebase support does not want to investigate yet.

wolfgang42 commented 5 years ago

For whatever it's worth, I was getting this same 403 Forbidden error and it turned out to be because I was decoding my VAPID keys the wrong way on the client (the library I used didn't understand base64url) so they didn't match the ones on the server. I doubt that this is what's causing your problem but this issue was the top search hit for the error message so hopefully I can help someone else.

dseljan commented 4 years ago

I can confirm sending push notifications doesn't work for FCM using VAPID. Getting 403 each time. I'm on latest Chrome version. I've just tried to send a notification using subscription in Mozilla, everything works as expected.

EDIT: I have managed to send a notification to Chrome! It turns out, if you send an empty payload, you get a status forbidden for some reason.

berberon commented 4 years ago

had the same problem, needed to add "https://" before fcm.googleapis.com/fcm/send and it solved the problem

cbratschi commented 4 years ago

had the same problem, needed to add "https://" before fcm.googleapis.com/fcm/send and it solved the problem

In our cases the URLs are all HTTPS but fail with 403. Currently 2k of about 74k pushes are failing. The number is now more or less stable. We plan to remove those 2k tokens as expired ones soon. There is no other solution right now.

aranoe commented 4 years ago

I have the same problem, getting 403 Forbidden. I switched back to an older one and it works again.

marco-c commented 4 years ago

I wonder if this is the same as #532.

zhelingwang commented 4 years ago

I met this problem . I think the reason why you get a 403 Forbidden is that cache file used by serviceWorker does not update in time.

marco-c commented 2 years ago

Are you still seeing this issue?

cbratschi commented 2 years ago

We consistently get some 403 errors. These values are from our latest sent batch of push notifications:

Host errors:

Format: ::