Binary Python modules are not re-imported

[vlad0337187]

Hello.

After you stop game engine and start it again, all text Python modules are re-loaded. So if I made some changes to code, new actual code will be launched.

But this doesn't apply to binary Python modules, they aren't reloaded. Even if you compiled a new version, you need to close and re-open Blender itself to see actual changes.

Only if you import that module from some python code, you can make imp.reload(binary_module_name), but if you specified binary module name in Python controller directly - you can't reload it until Blender restart.

Here video with steps to reproduce: https://youtu.be/EYFQExAyjLE

(I wrote binary module in Nim there, but same applies if you'll write it in Cython, for example)

Hope, it can be fixed easily =)

System: Linux Mint 19 x64 Cinnamon UPBGE: 0.2.4

Best regards, Vladislav

[Masterblop]

I'm not an expert, but I believe it works properly.

[paul-marechal]

Interesting issue! Although not sure if that is an issue caused by the way Python internalize modules...

[BluePrintRandom]

I wonder if we can unload the module using PY?

[BluePrintRandom]

import sys
del sys.modules['ModuleName']

also see -

in Python 3, reload was moved to the imp module. In 3.4, imp was deprecated in favor of importlib, and reload was added to the latter. When targeting 3 or later, either reference the appropriate module when calling reload or import it.

[paul-marechal]

Maybe the changes would happen somewhere around this method, although I wouldn't know how to make the BGE reload the module if it already exists...

Am on Windows so I have no idea how to reproduce.

---

Edit: https://docs.python.org/3/c-api/import.html#c.PyImport_GetModule So maybe just checking that before importing, and if not nullptr then reload it I guess?

@panzergame would that make sense?

[lordloki]

If you launch blenderplayer works?

[paul-marechal]

@lordloki given that it will be a new process at each launch, I would assume that it will work (because it will reload the Python interpreter each time)

[vlad0337187]

@lordloki , @marechal-p , just checked with standalone player. All works as expected =)

[vlad0337187]

Hm.. Seems there are problems with reloading binary modules in Python itself. I'm not sure that approach from PR would work. https://bugs.python.org/issue1144263

Made such function to perform "force" reloading: https://pastebin.com/tfquvGd0 But It works on linux, I didn't tested it on Windows. Not sure it could be easily re-writen to C and be merged with UPBGE core code.

[paul-marechal]

@vlad1777d This way can you point to a Python script that would delegate the call from the BGE to your native .so module ?

I was thinking that in scripts importing native modules, we could have a require function (similar to the way JS works in a node env) that would basically do the correct import/reload?

from loader import require
native = require('some_native_module')

[vlad0337187]

@marechal-p , yes, I think so.

[panzergame]

@marechal-p I saw that sys.meta_path is used to find modules. Did you try this already ?

[paul-marechal]

@panzergame never tried, but sys.meta_path just holds a list of finders for modules. What was your idea?

[panzergame]

I though about registering a finder at the beginning of the list which always failed but register the import calls. But i'm absolutely sure that's a terrible way.