python-javabridge prevents from app termination and overrides atexit

[arogozhnikov]

• I need to use javabridge in several places
• JVM can be started only once
• so I want to use a function that starts VM and kills it when app exits
• python-javabridge does not kill VM itself on exit
• so I've tried atexit hook for that

  def stop_vm():
  if vm_started[0]:
      print('killing!')
      javabridge.kill_vm()

But this code is never reached, no matter where I put

atexit.register(stop_vm)

[LeeKamentsky]

This is a chicken and egg problem I think. There are threads that are started by Javabridge and those are not terminated until javabridge.kill_vm() is run. If I remember right, the atexit function won't be called until those threads are terminated. I've always called javabridge.kill_vm() as part of normal program termination. I did try using atexit, but it did not work. Sorry about this - it's a limitation of the JVM and something I can't get around.

[thomas-villani]

In case anyone else is seeking an elegant solution to this problem, I think I've found a way. If you place the call to starting javabridge into a custom Thread subclass, and run that thread as a daemon with another class to keep track of the instance, and call the kill function on termination of the instance, then when the program execution finishes and it is only waiting on daemon threads, it will delete the instance referring to the javabridge thread, and then properly call the javabridge.kill_vm() command with atexit.

See below:

import javabridge
import logging
import bioformats
import atexit
from threading import Thread

log = logging.getLogger("JVM")
log.setLevel("DEBUG")

JAVABRIDGE_DEFAULT_LOG_LEVEL = "WARN"

class JavaBridgeException(Exception):
    pass

class _JBridgeThread(Thread):

    def run(self) -> None:
        log.debug("Starting javabridge")
        javabridge.start_vm(class_path=bioformats.JARS, max_heap_size=f"{JVM.HEAP_SIZE}G")
        rootLoggerName = javabridge.get_static_field("org/slf4j/Logger", "ROOT_LOGGER_NAME", "Ljava/lang/String;")
        rootLogger = javabridge.static_call("org/slf4j/LoggerFactory", "getLogger",
                                            "(Ljava/lang/String;)Lorg/slf4j/Logger;",
                                            rootLoggerName)
        jvm_log_level = javabridge.get_static_field("ch/qos/logback/classic/Level", JAVABRIDGE_DEFAULT_LOG_LEVEL,
                                                    "Lch/qos/logback/classic/Level;")
        javabridge.call(rootLogger, "setLevel", "(Lch/qos/logback/classic/Level;)V", jvm_log_level)

    def kill(self):
        log.debug("Killing javabridge")
        javabridge.kill_vm()

class JVM:

    _jvm_thread = None
    _jvm_instance = None

    def __init__(self):
        JVM._jvm_thread = _JBridgeThread(daemon=True)
        JVM._jvm_thread.start()

    def __del__(self):
        JVM._jvm_thread.kill()

    @staticmethod
    def start():
        if JVM._jvm_instance is None:
            JVM._jvm_instance = JVM()

    @staticmethod
    def stop():
        if JVM._jvm_instance is not None:
            del JVM._jvm_instance
            JVM._jvm_instance = None

if __name__ == "__main__":
    atexit.register(JVM.stop)
    JVM.start()

    import time
    print("something")
    time.sleep(1)
    print("something else")

Which gives on execution the following:

Starting javabridgesomething

something else
Killing javabridge

Process finished with exit code 0

[arogozhnikov]

@thomas-villani I've done it differently, but that's a very nice solution!

[posutsai]

@thomas-villani I've tried another solution. There is an argument of threading.Thread constructor in Python named daemon. If you make the spawned thread as daemon, the main thread will exit without waiting for child thread and the registered atexit hook function will be executed.

Following is a workaround with daemonic JVM thread.

import functools
import threading
import javabridge as jb
import bioformats as bf

old_init = threading.Thread.__init__
# Make JVM as daemon
threading.Thread.__init__ = functools.partialmethod(old_init, daemon=True)
jb.start_vm(class_path=bf.JARS)
threading.Thread.__init__ = old_init
atexit.register(jb.kill_vm)

However, I am not quite sure if there is any side effect. @LeeKamentsky I would like to know if you have any comment.

[LeeKamentsky]

Thanks for your suggestion @posutsai. It's worth a try, please report back if it works. I think I might have tried this in the past, but not with the daemon trick.

I wish I could add this to start_vm to make everything automatic, but I feel bad about monkey-patching the threading module - that seems very intrusive to do inside a package.

[posutsai]

Thank you @LeeKamentsky. Making JVM thread a daemon seems work perfectly and nothing goes wrong. I totally agree with your bad feeling about patching threading module. If possible, I would like to open a PR to make user able to configure JVM as daemon or not, which simply add the daemon argument in here.

def start_vm(args=None, class_path=None, max_heap_size=None, run_headless=False, as_daemon=True):
       # ....
       __start_thread = threading.Thread(target=start_thread, daemon=as_daemon)

[LeeKamentsky]

Thank you @posutsai! I'd welcome a pull request.