Recursive callbacks cause a deadlock

[ljeub-pometry]

Hi,

I am trying to use PEMJA to build an interface between python and scala code. This is working fine for simple cases, however I run into a deadlock in the following situation:

1. Call python code from jvm
2. Python code calls a jvm method
3. The jvm method tries to call python again (deadlock)

Things I have tried to work around this issue:

• Try to create a new python interpreter in the jvm for step 3. above (with MULTI_THREAD this still deadlocks, with SUB_INTERPRETER this fails with a numpy import error.
• Try to create a new Thread in python (this causes a segfault in C [pemja_core.cpython-310-darwin.so+0x7fc8] JcpPyJObject_New+0xf8 when trying to call non-trivial jvm methods from the new python thread).

Do you have any guidance on how to work around this issue?

[HuangXingBo]

@ljeub-pometry Thanks for your usage. We don't have step 3 such use cases before, I will try your case to see where the problem is.

[HuangXingBo]

@ljeub-pometry I have figured out the cause of the error. The reason is that the PythonInterperter you use in the main thread will hold the GIL of Python during the calling process. When the calling process has not finished, you try to call to create a new PythonInterperter. During the creation process, will go to lock GIL, which leads to deadlock.

[HuangXingBo]

I'll do some experimenting and see how to meet this need

[ljeub-pometry]

Thank you for looking into it. Ideally, this situation shouldn't require more than one python thread at all, i.e., when calling a jvm method, it probably should release the GIL while python is waiting for the result, such that the interpreter is free to execute the step 3 callback above.

[HuangXingBo]

@ljeub-pometry Yes. You are right. Before calling Java Function, we need to release GIL just like numpy do. I will fix this issue asap.

[HuangXingBo]

I have created PR https://github.com/alibaba/pemja/pull/15 to fix it

[HuangXingBo]

I have merged the fix into master branch and release-0.2 branch

[ljeub-pometry]

Thank you for the quick fix! I just tested it and this does indeed fixes the deadlock!

[HuangXingBo]

@ljeub-pometry Thanks for your verification.

[HuangXingBo]

Hi @ljeub-pometry , do you plan to take use of pemja in Raphtory?

[ljeub-pometry]

We are still in the early stages of adding python support to Raphtory but it does look like pemja is the most viable option for executing python code in the backend in terms of performance.

[HuangXingBo]

Ok. If you have any needs and ideas for pemja, welcome to exchange. According to the plan, in the next version, pemja will provide support on numpy and windows