search

Skallwar / suckit

Suck the InTernet
Apache License 2.0
741 stars 39 forks source link

Improve speed regression testing #107

Open CohenArthur opened 3 years ago

CohenArthur commented 3 years ago
Skallwar commented 3 years ago

Self hosted server to avoid network delays

Fixed by #108

pinkforest commented 3 years ago

@CohenArthur @Skallwar any specific reason for bringing python and http.server ?

httpmock is rust too and runs either standalone or via API in sync and async context if you wish to decide adopt async later and include it easily within tests.

httpmock runs on hyper (like reqwest) and also features delay setting and I've been pondering contributing there for more network behaviour scenarios.

nginx also would provide sendfile and other optimisations if standalone is still required and it's portable and fast.

nginx also is event based reactor (+prefork) which means no messing with threads while you serve static content but simulating (without breaking portability) delay scenarios and other network blips is harder with limit_req, limit_req_zone, rate=20r/s

Also instead of cloning the linux repo there could be "a seed" setting that readily generates into repeatable data for various test cases as there can be different performance scenarios for tons of small files, large files, error handling etc.

If you are keen I could push a PR

CohenArthur commented 3 years ago

@CohenArthur @Skallwar any specific reason for bringing python and http.server ?

Not really :D It was simple, and this is what the first testing benchmark was originally written in. It would scrape http://books.toscrape.com/ and time it, which was bad for a number of reasons. I am definitely not against functional testing/benchmarking being done in Rust, and I don't think @Skallwar is either.

Also instead of cloning the linux repo there could be "a seed" setting that readily generates into repeatable data for various test cases as there can be different performance scenarios for tons of small files, large files, error handling etc.

The test clones the linux repository in order to benefit from a big folder with known data. We checkout the v5.9 branch so that every test runs against the same content. Seeding it would be interesting, but maybe not for this speedtest specifically.

If you are keen I could push a PR

That would be absolutely fantastic and we'd love it! I believe that we are using some form of HTTP mocking in the tests already, so this would definitely fit. Thanks a lot!