RouterException: Server error: INVALID_ARGUMENT: Request contains an invalid argument.

[jrconlin]

Staring since around Mar. 16

https://sentry.prod.mozaws.net/operations/autopush-prod/issues/6072684/

RouterException: Server error: INVALID_ARGUMENT: Request contains an invalid argument.
  File "twisted/internet/defer.py", line 654, in _runCallbacks
    current.result = callback(current.result, *args, **kw)
  File "autopush/router/fcmv1client.py", line 129, in error
    raise RouterException("Server error: {}".format(failure.value))

RouterException: Server error: INVALID_ARGUMENT: Request contains an invalid argument.

[jrconlin]

Not really seeing where the error might be from the docs

[AzureMarker]

The response should contain more information about what was invalid: https://firebase.google.com/docs/reference/fcm/rest/v1/ErrorCode#ENUM_VALUES.INVALID_ARGUMENT

[jrconlin]

Right, but the question is whether or not the error handler for treq allows the response body to be read. I think so, but I'd have to play around a bit to make sure.

[Adrianotiger]

I have the same error message, when I try to send a push notification to a mobile device from a php-script. It works on Firefox for desktop, but not Firefox on mobile (it works on Edge and Chrome), I get this message:

_Server error: INVALIDARGUMENT: Request contains an invalid argument.

"endpoint": https://updates.push.services.mozilla.com/wpush/v1/gAAAAABfgDHFI3uKHq48tJLjFHD1Osft2o_4YSJgHeswI1XAPVODFc-ZYlqj2wm... "code": 500, "errno": 999, "error": "Internal Server Error", "more_info": "http://autopush.readthedocs.io/en/latest/http.html#error-codes"

How should I know what is wrong, if the answer is a errno999?

[AzureMarker]

@Adrianotiger The cause of this error was due to message size limitations. Autopush was allowing larger message sizes than what Firebase would allow. Check the size of your messages, though that error is generic enough that you may have a different problem. The fix for this specific issue is targeted for release 1.58.2: https://github.com/mozilla-services/autopush/pull/1437

[jrconlin]

Thanks Mark!

Yeah, the error is fairly generic and we're going to be fixing that (hopefully) soon. I'll add in that since messages are going over a bridge we don't control (in this case, Firebase) we lose some of the message to processing and conversion. This isn't a problem for chrome, since they own the bridge they're sending the data over so no reformatting is required. Normally, the server should kick back a message stating that the message is too big for a specific platform, so there may be something else at play.

You can see what version of software is running on our servers with: curl https://updates.push.services.mozilla.com/__version__

On Fri, Oct 9, 2020 at 3:24 PM Mark Drobnak notifications@github.com wrote:

@Adrianotiger https://github.com/Adrianotiger The cause of this error was due to message size limitations. Autopush was allowing larger message sizes than what Firebase would allow. Check the size of your messages, though that error is generic enough that you may have a different problem. The fix for this specific issue is targeted for release 1.58.2: #1437 https://github.com/mozilla-services/autopush/pull/1437

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/mozilla-services/autopush/issues/1373#issuecomment-706422771, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAIXK2OHHMYD4WKSMJ25EDSJ6ES5ANCNFSM4LXAVMLQ .

[Adrianotiger]

Thank you. Finally I was able to implement it on Firefox mobile, using aes128gcm as encryption and setting autopad to false. Edge mobile still receive an empty payload, but probably it is a browser bug.

[tublitzed]

Re-opening as this isn't fixed in this release.

[jrconlin]

Seeing a spike in these. Really need to add some additional info to the sentry error including: VAPID sub, body length, and a breakdown of google.rpc.BadRequest

[pjenvey]

The spike began on May 1st (suspicious).

and a breakdown of google.rpc.BadRequest

AFAICT from their general error docs and the firebase code docs for INVALID_ARGUMENT "An extension of type google.rpc.BadRequest is returned to specify which field was invalid." which would be the details error field.

Many of the errors seen during the spike have no details field (we emit "INVALID_ARGUMENT: Request contains an invalid argument. No Details").

There's also some "The registration token is not a valid FCM registration token" and CancelledError (likely timeouts) errors.