search

webcompat / webcompat.com

Source code for webcompat.com
https://webcompat.com
360 stars 191 forks source link

Investigate a timeout when delivering webhook #3575

Open ksy36 opened 3 years ago

ksy36 commented 3 years ago

There is a timeout error in https://github.com/webcompat/web-bugs-private/issues/24214 when delivering a webhook with "action": "opened" :

We couldn’t deliver this payload: timed out

The comment with public issue url was added and a classification request was sent. I think the timeout is happening at classification result retrieval step.

When I redilivered the webhook, the classification was performed successfully and label was added

Screen Shot 2021-05-20 at 12 58 47 PM
ksy36 commented 3 years ago

https://github.com/webcompat/web-bugs-private/issues/24222 https://github.com/webcompat/web-bugs-private/issues/24217

Two more issues that should have been labelled as invalid, but didn't

karlcow commented 3 years ago

We discussed a bit with @ksy36 about the options.

$ grep "classification failed" webcompat.log
2021-05-20 16:45:19,995 tracking issue 24214 classification failed (Couldn't classify issue 24214 in 12 seconds, aborting)
2021-05-20 17:45:46,286 tracking issue 24217 classification failed (Couldn't classify issue 24217 in 12 seconds, aborting)
2021-05-20 19:32:25,267 tracking issue 24222 classification failed (Couldn't classify issue 24222 in 12 seconds, aborting)
2021-05-20 22:29:22,275 tracking issue 24229 classification failed (Couldn't classify issue 24229 in 12 seconds, aborting)
2021-05-20 23:03:49,235 tracking issue 24232 classification failed (Couldn't classify issue 24232 in 12 seconds, aborting)
2021-05-20 23:05:28,528 tracking issue 24233 classification failed (Couldn't classify issue 24233 in 12 seconds, aborting)
2021-05-20 23:57:11,941 tracking issue 24234 classification failed (Couldn't classify issue 24234 in 12 seconds, aborting)
2021-05-21 00:31:38,522 tracking issue 24238 classification failed (Couldn't classify issue 24238 in 12 seconds, aborting)
2021-05-21 01:11:01,866 tracking issue 24240 classification failed (Couldn't classify issue 24240 in 12 seconds, aborting)
ksy36 commented 3 years ago

Let's try to measure an optimal time for waiting before retrying to call HTTP service by adding a timer at the beginning of this function https://github.com/webcompat/webcompat.com/blob/master/webcompat/webhooks/ml.py#L29