Reloading imported modules

[josephjclark]

Hi, I'm loving node-calls-python! I'm using it in bun.js and it works great.

I'm struggling a bit to reload my python imports.

If I py.import('script.py', true) it'll reload script.py script each time, no problem.

But anything that script.py imports WON'T get reloaded.

In my case I don't just want to reload that one module, I want to reload my whole environment.

What I really want is:

• In production, I want to use the same python environment to run my scripts, with no reimporting. That's why I came here!
• In dev, I want to re-load my whole python environment every time, so that I don't need to restart my server manually. I appreciate that this might leak memory but in a dev environment I'm not so worried.

I suppose I could just shell out to python for this, but it means my dev and prod environments are going to be very different, which is nightmare for testing :thinking:

I've run a hacky little test and creating a new Interpreter() isn't enough to give me a pristine new python environment. I feel like I want to be able do Intepreter.destroy() and new Interpreter() to create my own threads?

[hmenyus]

I have added two minor things that may help you: https://github.com/hmenyus/node-calls-python?tab=readme-ov-file#development-mode You will need 1.9.0 to try out these.

[josephjclark]

Amazing, thank you! I'll take a look and post some feedback:)

On Wed, 1 May 2024, 22:01 Menyhért Hegedűs, @.***> wrote:

I have added two minor things that may help you:

https://github.com/hmenyus/node-calls-python?tab=readme-ov-file#development-mode You will need 1.9.0 to try out these.

— Reply to this email directly, view it on GitHub https://github.com/hmenyus/node-calls-python/issues/78#issuecomment-2089134265, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABVZZXKOX72UTSLGUC3XS7LZAFJ4DAVCNFSM6AAAAABG2UTB7WVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAOBZGEZTIMRWGU . You are receiving this because you authored the thread.Message ID: @.***>

[josephjclark]

Hi, bad news I'm afraid.

First off I'm seeing this in 1.9.0: https://github.com/hmenyus/node-calls-python/issues/81

The two APIs look great, reimport in particular seems to be what I want. But I have two issues! 1) It's still not working for indirect dependencies - it only reloads the exact module I call 2) After calling reimport a couple of times I get a core dump exception.

">> reimporting" is my crappy log output, called immediately before calling py.reimport.

I'm just running a dummy python module here which console.logs and returns a dict. Nothing heavy.

In my use-case I have a few python scripts, but I access them through a single entrypoint entry.py:

This is basically what I'm running:

# entry.py
import echo

def main(args):
   return echo(args

Re-importing entry doesn't help me - I need to re-import echo.py. Ideally actually that would work on a whole folder, rather than a specific file, but I can workaround that

[josephjclark]

I should note as well that I've got a workaround to all this now (I've setup nodemon to restart my whole node process when a python file changes, it's fine).

I would prefer to be able to stably and safely reimport the python each time (a la py.reimport), but at this point I don't actually need it.

[hmenyus]

Crash fixed in 1.9.1. I have a test and dependent modules are reloaded properly

[josephjclark]

This works great, thank you!

No crashes, and I can pass a folder rather than a file into py.reimport