search

faster-cpython / ideas

1.68k stars 48 forks source link

Cached module execution state and start-up time improvement #599

Open JamesHutchison opened 1 year ago

JamesHutchison commented 1 year ago

Kind of an open idea - I see a few challenges so I just wanted to float the general idea at first before really doing too much digging. I'm assuming this has been floated before, but I didn't happen to see an existing issue.

When I was trying to streamline a client for pytest hot reloading, I noticed a substantial amount of time was spent doing stat. This is also an issue that's come up with Brython - effectively, a bunch of the start-up time is spent traversing directories, and its all done in serial fashion. With remote file systems and the docker filesystem layer, the time cost is extrapolated. I'd imagine that single file python executables would benefit as well with a solution that's built here. Being able to blaze through imports can substantially improve start-up time. Start-up time is import for running tests quickly and also for CLI applications (imagine pressing tab to autocomplete from bash and having to wait a second or so to see the results). I'd imagine there may also be benefits in serverless applications.

I'm wondering if we can do some sort of caching where the module execution state is cached. Perhaps even multiple modules are cached. Some challenges I foresee is that the import mechanisms are very complex and many things are executed at runtime and shouldn't be cached. Many modules also have side-effects. Perhaps someone has already explored this approach?

oraluben commented 1 year ago

There are tests like https://github.com/faster-cpython/ideas/issues/500 to reduce IO during module import with zipped-pyc files. It can significant reduce import time on windows against Windows Defender IIRC. We also developed a memory map one to do similar things, which can reduce 30% startup time on 3.12 thanks to the immortal object proposal.

All the similar approaches I know do not cache "module state", but bytecode or similar things. The main reason is that a module is basically a dict that is generated from the execution of a bunch of bytecodes and can be really unstable. Instead, bytecode is more deterministic for caching.