search

web-platform-tests / wpt

Test suites for Web platform specs — including WHATWG, W3C, and others
https://web-platform-tests.org/
Other
5.03k stars 3.12k forks source link

Stress test wptserve #10898

Open Hexcles opened 6 years ago

Hexcles commented 6 years ago

In light of the tricky race condition we encountered a while ago (#10667), I'm thinking maybe we should at least stress test wptserve.

It's hard to write unit tests to verify multi-threading and catch race conditions, but stress testing should help to some extent. The reason why #10667 wasn't discovered was because of the relatively low load on both Travis and Mozilla testing infra (both of them don't run a lot of browsers in parallel in one shard), but it was later discovered in Chromium CI which has a much higher load. I imagine some stress testing (e.g. using ab on a bunch of files with server-side features) would have caught the 500 problem (of course, the more subtle bugs that don't throw exceptions can't be caught as easily).

cc @jgraham @gsnedders @foolip

foolip commented 6 years ago

I think this is a great idea. We could hit some subset of files in the manifest where we know what the response is supposed to be over and over in parallel, and just see if it breaks. Not sure how to do that in Python though :)