search

mozilla-services / location-leaderboard

A leaderboard service for Mozilla Geolocation Stumbling
Mozilla Public License 2.0
4 stars 5 forks source link

Uploads to the leaderboard are stalling still #298

Open crankycoder opened 8 years ago

crankycoder commented 8 years ago

We're seeing it on the stumbler client in https://github.com/mozilla/MozStumbler/issues/1790

jaredlockhart commented 8 years ago

@crankycoder Where are you seeing that uploads are stalling?

crankycoder commented 8 years ago

It looks like the connection is timing out during upload of data to the leaderboard.

The relevant workaround in stumbler is : https://github.com/mozilla/MozStumbler/pull/1786

On Tue, May 10, 2016 at 2:03 PM, Jared Kerim notifications@github.com wrote:

@crankycoder https://github.com/crankycoder Where are you seeing that uploads are stalling?

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub https://github.com/mozilla-services/location-leaderboard/issues/298#issuecomment-218240275

jaredlockhart commented 8 years ago

@crankycoder the server logs show a maximum response time of 400ms for the data submission API end point on production. I can't find any logs showing timeouts, do you have any additional information?

crankycoder commented 8 years ago

The last crash I can see happened at 2016-May-10 at 12:43 PM UTC time. All I can see is that the socket timed out. I've got a hard 5 second timeout for any sockets set in the HTTP client inside the stumbler.

Can you see anything around that time?

jaredlockhart commented 8 years ago

@crankycoder No the longest I see in datadog and kibana is 600ms from today. Is it possible that the device is having wireless connectivity degradation and timing out at the signal level? I don't see any indication the server is failing to respond to clients, the timing looks consistent and I don't see any non 201 responses.