search

clj-python / libpython-clj

Python bindings for Clojure
Eclipse Public License 2.0
1.05k stars 68 forks source link

process hanging in embeded mode on exception #233

Open behrica opened 1 year ago

behrica commented 1 year ago

As a FUP of #221 which provided a new pathway to see the exception, but does not prevent the "hanging" in an other pathway.

The simplest scenario is that we have a hanging python process by using this clojure code crash.clj

(ns train
  (:require [libpython-clj2.python :refer [py.- py.] :as py]
            [libpython-clj2.python.ffi :as ffi]))

(def locked (ffi/lock-gil))
(py/import-module "not existing")

and load it in embedded mode using manual-gil via:

export JDK_JAVA_OPTIONS="-Dlibpython_clj.manual_gil=true"
python -c 'from clojurebridge import cljbridge;cljbridge.load_clojure_file(clj_file="crash.clj")'

The exception thrown by python on the last line in crash.clj is swallowed by either clojurebridge or libpython-clj and the process hangs forever.

behrica commented 1 year ago

I still have this open https://github.com/behrica/clojurebridge/issues/2

as I don't know where a potential fix needed to be.

behrica commented 1 year ago

Just to clarify.

There is an new API call which prevents the hanging seen above:

(py/with-manual-gil ...

But this API is by no mean mandatory, neither very prominently documented. So users might still use the other API, as above, and get the experience of a hanging process.

I think we should throw an exception (and terminate the JVM) as well in the case above.