JamesHutchison / pytest-hot-reloading

A hot reloading pytest daemon, implemented as a plugin
MIT License
89 stars 2 forks source link

Improve start-up time (client) #14

Closed JamesHutchison closed 1 year ago

JamesHutchison commented 1 year ago

There appears to be an overhead to running things caused by pytest. This overhead seems to occur prior to running the plugin. If we can jump straight to running the plugin, running tests will be very quick.

JamesHutchison commented 1 year ago

Out of scope for this is improving the start-up time incurred by using jurigged. In tests, start-up could be 50% slower with jurigged. It would be better if it was about the same as normal.

JamesHutchison commented 1 year ago

17 will cover the jurigged overhead

JamesHutchison commented 1 year ago

Profiling this, I'm actually seeing the time sink seems to be this:

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    1.602    1.602    1.602    1.602 {method 'recv_into' of '_socket.socket' objects}

When I profile the opposite end, the test is much faster, so perhaps there's a bit of dead time in the communication that could be reduced

JamesHutchison commented 1 year ago

This branch experiments with substituting the built in XMLRPC library with Robyn. The performance is nearly identical so its not worth making the switch for that reason alone.

https://github.com/JamesHutchison/pytest-hot-reloading/tree/robyn-experiment

Without debugging, it looks like it's getting :

I know its possible to get tests to run in a split second (close to that server time), because that is how fast it ran on my hackathon implementation (which used files instead of a web server). I distinctly remember the spinner not being able to complete a full rotation. That said, things are pretty fast so this might be splitting hairs. Of course, the daemon is frequently ran with the debugger, and these times balloon out to where the time savings are more significant and meaningful. Using the debugger seems to add less than half a second.