Why 2 instances SharedInterpreter cannot be executed in the same thread?

[Daniel-Alievsky]

I see that your SharedInterpreter and SubInterpreter instances cannot be created simultaneously in the same Java thread - it is blocked by "threadUsed" tracker. I must close one interpreter, before I'll be able to use another (in the same thread). Could you tell, what is the reason of so serious restriction? Maybe, it can be relaxed for a case of two SharedInterpreter ?

I can explain. I'm working over an essentially large and extensible system of executing different modules, written in different languages, including (for now) Java, JavaScript, Python. How can I use JPE here?

1) I allows only 1 instance of Interpreter, namely SharedInterpreter, for all my JVM. Of course, it will work. But if two users of our system will define a function with the same name "my_important_function", and if their modules will be executed inside the same product (in different windows or in different session of the web-server), it will be a catastrophe. And this problem almost cannot be resolved practically, if we allow user to buy or download different Python solutions with from different developers. Any user must know absolutely exactly, what Python solutions are executed via JPE on his server/desktop.

2) I already know that SubInterpreter is only a partial solution, because it is incompatible with necessary native libraries and can lead to a system crash (absolutely unacceptable for a commercial system). Ok, why do not use several SharedInterpreter? It seems to be logical: we have separate namespace for users' simple Python scripts, purchased/downloaded from Internet, and, if these scripts are correct and do not try to modify global variables like sys.path, all will work.

And now I see that... we cannot use several SharedInterpreter in the same thread until their closing. But an important idea of efficient using external script (Python, JavaScript, anything else) is to create an interpreter ones and reuse it many times, while a user needs this script. For example, solution A of my user performs some loop 10000 times - operation A, operation B, ..., operation X, and here operation B is implemented as a Python script. Of course, we want to reuse the same SharedInterpreter for it. But then my user buys (downloads) another package Y for adding into the same loop, and package Y also uses Python script via SharedInterpreter. And we have a message "Unsafe reuse of thread..."

Why? Is it really necessary to restrict using SharedInterpreter by only 1 instance inside a single thread?

[bsteffensmeier]

Each Jep interpreter has a unique PyThreadState. The cpython internals store the PyThreadState in a thread local variable and do not permit more than one PyThreadState to be active on a thread at the same time.

When using multiple interpreters it is common to make new threads dedicated to running each Jep Interpreter and implementing custom logic to communicate with the interpreters from other threads.

If you simply need isolated global variables you can use a single Jep SharedInterpreter and call the python exec function with custom global variables. It's a little strange to call python exec from Jep exec but I think this would work:

try (Interpreter interp = new SharedInterpreter()) {
    interp.exec("taskOneGlobals = {}");
    interp.exec("exec('import taskOne', taskOneGlobals");
    interp.exec("exec('taskOne.doStuff()', taskOneGlobals");
    interp.exec("taskTwoGlobals = {}");
    interp.exec("exec('import taskTwo', taskTwoGlobals");
    interp.exec("exec('taskTwo.doStuff()', taskTwoGlobals");
    interp.exec("exec('taskOne.doMoreStuff()', taskOneGlobals");
}

[Daniel-Alievsky]

I understand your explanations, thank you. But if CPython has some troubles with separate name spaces inside a single Python, it should not become a problem for Java users of Jep. For example, JavaScript is usually oriented to a single thread, but we have no any problems to create any number of ScriptEngine and should not think about threads at all.

What do we have right now? Very strange situation. Yes, I can create several instances of SharedInterpreter or SubInterpreter. But actually I cannot use them! The only way to do this is some "tricks" with creating separate threads. But modern practices of multi-threading in Java avoid direct thread operation; usually we use such abstractions as ExecutorService or ForkJoinPool. Usually I actually don't know anything, will be my tasks executed in separate or in a single thread! This decision is made by Java API to provide the best performance, and Java API provides guarantees that I will not have "forgotten" or "hung" threads.

I believe you should do something with this, maybe in collaboration with developers of CPython. If it is necessary to resort to some tricks so that your interpreter can be used in a natural way for Java - creating an interpreter object, using, creating another object, using, closing them when it is necessary or by garbage collector - then you, not your users like me should implement this tricks inside your library. For example, you could create separate Java threads yourselves inside your object and hide this from your end user. While such approach, your team, not your users, would be responsible for correct work of JEP, for correct stopping all unnecessary threads, for support of thread pool when it is possible, etc. I think it is a correct and good way.

On the other hand, I (like other developers) cannot spend enough time to provide absolutely correct work of JEP together with tricks like several threads, automatic creating custom namespaces with help of "exec" function etc. All that I need - provide for my users an ability to run their Python script with minimal risk and in any environment (single user, multi-user server, one or several windows with opened Python scripts etc.) So, I have no other way besides selecting the simplest solution: only 1 SharedInterpreter for 1 JVM and strict external control over namespaces (every scripts is places in a separate file inside a single tree of python scripts).

Of course, it is not so good as ability to use Python script snippets anywhere, as it is possible with JavaScript - from user's interface (simple formula "c=a+b; d=c/2", executing in some loop, say, while scanning transport network) until teaching programming (a user try to write anything, and it does not violate work of other users). Therefore I hope you will be able to resolve this problem in some future inside JEP subsystem. How do you think, is it realistic?

[Daniel-Alievsky]

It seems I've found an idea how is it possible to resolve this (and I'll try to do this in my wrapper around JEP).

You store the thread, which is executing this interpreter, and throw an exception if it already executes another non-closed interpreter. But, instead of throwing an exception, you could just close the previous instance! In most situations it is exactly behavior that the user needs. While he works with only one interpreter, even many times, all will be Ok. If he will switch between several interpreters, it will lead only to slowing down and to possible problems with namespace - if the user will create an interpreter 1, then create an interpreter 2, then will try to work with interpreter 1 (but it will be closed). But it is not a typical way of usage, and you can write about this in documentation. For example, Java iterators also cannot be used in such a manner - usually you should finish work with iterator until creating another iterator on the same collection (if iterators change the collection).

Then, you can extend this idea by supporting an internal pool of threads and performing all elementary actions (methods of interpreter) via requests to these threads. For example, if your pool will contain 16 threads, all operations will be performed quickly until a user will want to create > 16 interpreters simultaneously. Not so bad.

[Daniel-Alievsky]

By the way, I see that your method "isValidThread" is deprecated. However I really need this method, but with another semantic: I need a method which will return true/false, true if an interpreter is alive and can be used, false if not. If not, I can, for example, close it and create another one. Could you add such public method? For example:

    public void isAlive() throws JepException {
        if (this.thread != Thread.currentThread())
            return false;
        if (this.closed)
            return false;
        if (this.tstate == 0)
            return false;
        return true;
    }

[Daniel-Alievsky]

It seems that I cannot get around the problem with SubInterpreter myself by a simple wrapper... I wanted to check, is your SubInterpreter alive and good for the current thread, and, if no, to close it and to create another instance. But you "close" method contains a check of the thread. My Stare multi-functional system, where I test JEP, controls all threads itself, and sometimes it calls my function from one thread, sometimes from another. When I try to use an instance of your SubInterpreter from a new thread, I even have no ability to correctly destroy it - your method "close" will throw an exception "Unsafe close..."

Of course I can create my own thread pool with a single thread, but it seems to be too complex workaround. Hope, maybe you will implement something like this in future versions.

[Daniel-Alievsky]

But situation is even more difficult. I see that even a single SharedInterpreter, created once for JVM (singleton!), cannot be used from different threads! It seems that your library cannot be used at all without tricks like my own global pool of threads. Java API creates a lot of threads in different situation, and how can I guarantee that an instance your interpreter will be always called from the same thread?? It is non-trivial, but I see that I'm enforced to provide such a guarantee, if I don't want to recreate your interpreter every time for every little work.

[bsteffensmeier]

Jep is a small project maintained primarily by 2 people in our free time. There are endless possibilities for things we could implement to make it easier for developers to use Python from Java but we only have a very small amount of time to give to the project so the scope of Jep is deliberately quite limited. For the time I donate to Jep I try to focus on accurately exposing the existing cpython c-api so that Java developers can access Python without writing any native code.

The threading requirements of Jep are based off the threading requirements of the cpython c-api. To avoid memory leaks Jep must delete a PyThreadState before a thread ends. There is no way to detect a thread end so Jep cannot automatically handle this for you. The Closable interface is a well established way to clean up native resources which is precisely what is needed by Jep interpreters. Your proposal of closing interpreters when a new one is opened may work fine for your program but as a general solution I think it creates unnecessary ambiguity regarding whether Interpreters require closing and could easily lead developers to create inadvertent memory leaks.

Your thread pool idea is a common solution and similar solutions have been implemented in other projects that use Jep. One of the complications that has prevented such a solution from being added to Jep is that it is not clear how to call methods on a PyObject which is returned from an Interpreter on another thread. The nicest solution would be to detect and return PyObjects that can forward their calls to the appropriate threads but that essentially requires a rewrite of PyObject which is a bit daunting. It sounds like your project is not using PyObject so this complication may not be relevant to you but if we were to implement a generic solution within Jep we would want consistent handling for PyObjects returned from Interpreters.

I believe the capabilities you are looking for would best be implemented as a layer on top of Jep. Jep can provide access to the c-api along with it's warts and limitiations and another layer can build on that to provide more friendly API mechanisms for specific use cases. I suspect most projects implement such a layer to suite there needs but unfortunately I am not aware of anyone who has published their code for others to use. You seem to be currently writing such a layer yourself, I encourage you to publish your code as an open source project so that others can take advantage of it.

[Daniel-Alievsky]

One of the complications that has prevented such a solution from being added to Jep is that it is not clear how to call methods on a PyObject which is returned from an Interpreter on another thread.

I didn't understand this well. Could you give me an example? Now Jep doesn't support working from different threads at all; what possible situation do you mean?

I agree that thread pools should be implemented as an additional layer. I already implemented some version. Please analyze the attached archive; maybe I'm missing something important? It seems to be working, not depending from which threads, but, of course, we need to test in for essential time in a real complex environment. jep.zip I believe, that when all will be well tested and the code will be checked by several specialists (me, my colleagues, and I also invite you to check) - it will be the best way just to include this or equivalent package into your library.

There is an important moment (my simple test doesn't show it). If, inside a single thread, I use MORE than 1 instance of JepContextContainer and use for all instances getLocalContext, you can see that the time of every executing getLocalContext is large: it closes the previous instance and create a new one. But getGlobalContext (which returns a global interpreter, shared inside all JVM) will works quickly always. Also all works quickly if we have only one persistent container and request from it the local context. Obviously it is the most typical situation, but if it is important to support several simultaneously working local contexts (interpreters), this logic can be easily enhanced by a pool of JepSingleThreadInterpreter instances.

[Daniel-Alievsky]

I already see serious problems in my last implementation... Several local context will not work correctly while called in parallel threads. I hope to send you a link to more correct version in public repository soon.

Can you tell me, how "heavy" is every instance of Share/SubInterpreter? How much memory can it occupy in real applications, which work with Numpy, OpenCV, non-trivial Python libraries? (Of course, if the code is written correctly and it does not make special "efforts" to spend a lot of memory.) Do we speak about megabytes, hundreds of megabytes, gigabytes? I need to understand this to decide, is it possible to stay in memory, for example, up to 10 instances of ShareInterpreter until finishing all JVM, or it can lead to exhausting memory and I must control every instance of your interpreter, to close it (by special thread) after 1-2 minutes of non-using it.

[Daniel-Alievsky]

Dear sirs, I've published the preliminary results here: https://bitbucket.org/DanielAlievsky/stare-python-experiments/src/master/jep-java-tests/src/main/java/net/algart/stare/bridges/python/jep/ These are not final results, but some version that I hope will work.

Please look at net.algart.stare.bridges.python.jep.additions package. I think, when it will be completed and well-tested, it could become an extension of your library. Below are some comments.

1) JepSingleThreadInterpreter implements an idea of JEP Interpreter, that guarantedly works in a single thread. I offer to create this object instead of your Share/SubInterpreter. Yes, it requires resources also for 1 thread in the corresponding thread pool, but this thread is correctly released close method or by garbage collector, if close() was not called in time. It should safe enough. I measured ans see that an instance of Shared/SubInterpreter is not too expensive for memory, and a user can freely create 10-20 (or even more) simultaneous instances of JepSingleThreadInterpreter.

For comparison, my previous solution with the only one global interpreter was very unsafe: if one client run Python script with an infinite loop, it meant blocking of all analogous Python tasks in the same JVM, even if they are executed by other users in other sessions of the same web server.

2) JepExtendedConfig is a little extension of your JepConfig, providing two things. First, it allows to run some startup script when your Interpreter is created. For example, if an application is oriented to processing large numeric data, if can be "import numpy" - at least to check, whether this package is really installed, and to prevent further executing without this. Second, it provides a concept of "global default configurator": some function for customizing JepConfig, that will be used by default without additional efforts. (For example, my implementation calls addIncludePaths for our system-specific Python paths.)

3) JepTools should be removed, when you will have fix the bug.

4) JepContextContainer is a very simple concept: you create this class once and then can re-use the same JepSingleThreadInterpreter many times, until it will freed by freeResources() or removed by garbage collector.

Please, if possible, take a look to this code. It seems to be simple enough to be stable. But I didn't start testing in a real tasks yet, hope to do this ASAP.

[fguillotpro]

This issue is of utmost interest. Thank you for the hard work.

[Daniel-Alievsky]

Thank you :) It is necessary also for us. Please tell if you will find some problems. Next weeks I'm planning to use and enhance this code; I'll comment all changes in the GIT.

[Daniel-Alievsky]

In particular, pay attention to a new class JepCreation (added today). Its goal to help a user in a situation, when something not installed or installed incorrectly. Your module just throws something like "UnsatisfiedLinkError", and what should do the user? Especially if he is not a programmer, but a technician who installs a large system (containing JEP deelpy inside) on a client's computer? I'm a programmer, but I was enforced to ask for help here.

So, I tried to detect most popular situations and wrap Java error message into JepException with more user-friendly diagnostics. In particular, I believe that some solutions will always use some "startup" code from JepExtendedConfig - if we use JEP for extending Java by scientific numeric algorithms, it is obvious that we need "import numpy". Its import may be considered as a part of initializing, and it is a good idea to inform user about this separately (see my method "performStartupCodeForExtended").

I recommend you to embed a code like this into your own system (for example, into constructors of SharedInterpreter/SubInterpreter) in some future versions. It is very simple, and it is very probable that you will be able to provide better and more correct diagnostics than I implemented in JepCreation class.

[Daniel-Alievsky]

Dear sirs, I see that I forgot about very important things: PyObject and PyCallable. Of course, if we use JepSingleThreadInterpreter, we cannot use standard way of creating these objects like getAttr(xxx, PyCallable.class): the returned object will be executed in another thread, so, your code will throw an exception. Moreover, exception will be thrown in finally section of standard try-with-resources block.

So, I reworked this: implemented my own classes AtomicPyObject and AtomicPyCallable. Also I offer JepPerformer - a simple wrapper around JepSingleThreadInterpreter, containing ready methods for typical operations like creating new object or calling its methods.

I would very appreciate if you will look at my new code in GIT and check: https://bitbucket.org/DanielAlievsky/stare-python-experiments/src/master/ Maybe I understand something wrong?

You will find a simple test: SimpleJepForObjects; of course, I'll also test this more seriously from different threads.