web-platform-tests / wpt.fyi

web-platform-tests dashboard
https://wpt.fyi/
Other
186 stars 88 forks source link

/api/shas issues hundreds of Datastore reads and takes >5s when not cached #1299

Open Hexcles opened 5 years ago

Hexcles commented 5 years ago

I happened to notice this while triaging the recent 500s returned by our APIs (mostly for my recently launched screenshot API).

The following log searching query should show you them:

resource.type="gae_app"
resource.labels.module_id="default"
logName="projects/wptdashboard-staging/logs/appengine.googleapis.com%2Frequest_log"
protoPayload.status=500
"/api/shas"

I correlated these slow 500s with traces and found that they had 500+ datastore.next calls. Presumably they eventually timed out.

Hexcles commented 5 years ago

cc @lukebjerring