search

mozilla / normandy

Firefox recipe server
https://wiki.mozilla.org/Firefox/Recipe_Server
Mozilla Public License 2.0
54 stars 46 forks source link

Normandy produces a 500 error communicating with Remote Settings (sometimes) #1810

Open mostlygeek opened 5 years ago

mostlygeek commented 5 years ago

see: https://sentry.prod.mozaws.net/operations/normandy-prod/issues/5401918/?query=is:unresolved

The error is: KeyError 'Unable to update a record, need an id.'

I tried it twice and it failed. Rehan tried it a minute or so later and it worked. Hmm.

glasserc commented 5 years ago

Remote Settings may have been down in the interval that you tried because of the megaphone migration to GCP.

mostlygeek commented 5 years ago

OK. That would explain it ... we had to ship a hotfix so this wasn't schedule.

leplatrem commented 5 years ago

I can confirm it was a connection error with Megaphone https://sentry.prod.mozaws.net/operations/kinto-prod-1/issues/5401917/ happened between Mar 21, 2019 10:31:47 PM UTC and Mar 21, 2019 10:33:59 PM UTC

leplatrem commented 5 years ago

we had to ship a hotfix so this wasn't schedule

What was this hot fix?

mostlygeek commented 5 years ago

https://bugzilla.mozilla.org/show_bug.cgi?id=1536453

shell1 commented 5 years ago

noisey / unhelpful failure from Normandy when it failed to talk to remote settings. more helpful failures possible