search

tbenthompson / cppimport

Import C++ files directly from Python!
MIT License
1.18k stars 66 forks source link

Possibility of hot-reloading? #39

Open alipunjani opened 4 years ago

alipunjani commented 4 years ago

It there some way to make it possible to hot-reload changes to a .cpp file in a running python process (e.g. jupyter notebook)?

In particular, for development of high performance numerical computing code, it would be very useful to be able to have a jupyter notebook where data and other memory structures are loaded and set up, and then be able to use cppimport to import a .cpp file that is being developed for some heavy compute implementation. Right now, calling importlib.reload on a cppimport compiled/imported module does seem to trigger a re-compile (if the .cpp file has changed), but the new module is not actually loaded into the python process.

This is probably a python limitation, but there are (hacky) workarounds like this: https://stackoverflow.com/questions/8295555/how-to-reload-a-python3-c-extension-module/8295590#8295590 I wonder if there would be a way to enable an optional "development" mode for cppimport where modules are loaded with these tricks that allow reloading of binary extensions?

It would even be great to get some pointers about how to go about implementing this so I can make a PR.

tbenthompson commented 4 years ago

Great idea! I've also run into situations where hot-reloading would be nice. I have some thoughts on how it might be feasible. When I get around to it, I'll think about this more and write my thoughts here.