search

hmenyus / node-calls-python

Call Python from NodeJS directly in-process without spawning processes
MIT License
252 stars 26 forks source link

How to unload any imported python module? #43

Closed sliterok closed 1 year ago

hmenyus commented 1 year ago

what do you mean "unload"? I think there is no standard way for this in python. Do you mean reloading?

sliterok commented 1 year ago

Unload as in killing the child process, but I figured out since it isn't spawned in the traditional sense, that will require some work to be done in the bindings. I'm trying to do some ML stuff with this package, but I want to use different models for different purposes, so it makes sense to allow unloading of unused python modules. It may be done through tensorflow/torch interface, but last time I tried it on google colab it didn't work at all.

hmenyus commented 1 year ago

There is no way to unload a module in python officially (not a single function on the python c api). There are tricks and hacks which may or may not work.

Honestly managing the allocated resources (including memory, files, filelocks, etc.) of your python code is the responsibility of your python code and not this addon. If you want to "unload" different models you have to rely on reference counting and GC of (c)python. For more details: https://towardsdatascience.com/memory-management-and-garbage-collection-in-python-c1cb51d1612c

The only responsibility of the addon is to free up python objects created by import/importSync/create/createSync. But that is done once GC of NodeJS runs and cleans up unreferenced handlers returned by those funcs.