Append BadGateway(502) error in retryNetworkAndHTTPErrors

castor-rtzr commented 2 years ago

[REQUIRED] Step 2: Describe your environment

Library version: latest
Firebase Product: cloud messaging

[REQUIRED] Step 3: Describe the problem

Steps to reproduce:

Recently I received BadGateway(502) error often when I send push messages using fcmClient.

http error status: 502; reason: server responded with an unknown error; response: <!DOCTYPE html>
<html lang=en>
  <meta charset=utf-8>
  <meta name=viewport content="initial-scale=1, minimum-scale=1, width=device-width">
  <title>Error 502 (Server Error)!!1</title>
  <style>
    *{margin:0;padding:0}html,code{font:15px/22px arial,sans-serif}html{background:#fff;color:#222;padding:15px}body{margin:7% auto 0;max-width:390px;min-height:180px;padding:30px 0 15px}* > body{background:url(//www.google.com/images/errors/robot.png) 100% 5px no-repeat;padding-right:205px}p{margin:11px 0 22px;overflow:hidden}ins{color:#777;text-decoration:none}a img{border:0}@media screen and (max-width:772px){body{background:none;margin-top:0;max-width:none;padding-right:0}}#logo{background:url(//www.google.com/images/branding/googlelogo/1x/googlelogo_color_150x54dp.png) no-repeat;margin-left:-5px}@media only screen and (min-resolution:192dpi){#logo{background:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) no-repeat 0% 0%/100% 100%;-moz-border-image:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) 0}}@media only screen and (-webkit-min-device-pixel-ratio:2){#logo{background:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) no-repeat;-webkit-background-size:100% 100%}}#logo{display:inline-block;height:54px;width:150px}
  </style>
  <a href=//www.google.com/><span id=logo aria-label=Google></span></a>
  <p><b>502.</b> <ins>That’s an error.</ins>
  <p>The server encountered a temporary error and could not complete your request.<p>Please try again in 30 seconds.  <ins>That’s all we know.</ins>

The error message says <p>Please try again in 30 seconds. <ins>That’s all we know.</ins>

In my opinion BadGatway(502) error need to be retry.

Relevant Code:

https://github.com/firebase/firebase-admin-go/blob/241c29308e3ea64fff9edc8e848bffba9e1ec07a/internal/http_client.go#L82

Want to change.

CheckForRetry: retryNetworkAndHTTPErrors(
    http.StatusServiceUnavailable,
    http.StatusBadGateway,
),

google-oss-bot commented 2 years ago

I found a few problems with this issue:

I couldn't figure out how to label this issue, so I've labeled it for a human to triage. Hang tight.
This issue does not seem to follow the issue template. Make sure you provide all the required information.

hiranya911 commented 2 years ago

502 is not safe for retries. See https://cloud.google.com/apis/design/errors#retrying_errors

@chong-shao do you know what the above error is about?

chong-shao commented 2 years ago

We saw a similar error reported previously https://github.com/firebase/firebase-admin-python/issues/493 and the decision there is not to add code in Admin SDK to retry on this. From the previous discussion, customer can add retry logic on their end for this.

502 errors seems to be transient and rare, and is not related to the FCM backend.

castor-rtzr commented 2 years ago

Thanks for the detailed reply. I will close this issue.

firebase / firebase-admin-go