problem with live image display from pycromanager

[monfort314]

Hi, I'm trying the plugin for displaying the live image from https://pycro-manager.readthedocs.io/en/latest/application_notebooks/pycro_manager_imjoy_tutorial.html that uses https://gist.github.com/oeway/f59c1d1c49c94a831e5e21ba4c6111dd

Specifically, I'm trying to run the code below, but it seems that after pushing 'start live' the function get_image() is called only twice (I checked it with a global counter)

Many thanks,

`import time

import numpy as np from imjoy import api

from pycromanager import Bridge

class MyMicroscope: async def setup(self): bridge = Bridge() self._core = bridge.get_core() exposure = self._core.get_exposure() api.showMessage("MMcore loaded, exposure: " + str(exposure))

def snap_image(self):
    if self._core.is_sequence_running():
        self._core.stop_sequence_acquisition()
    self._core.snap_image()
    return self.get_image()

def get_image(self):
    # we can also check remaining with getRemainingImageCount()
    tagged_image = self._core.get_tagged_image()
    image_array = np.reshape(
        tagged_image.pix,
        newshape=[-1, tagged_image.tags["Height"], tagged_image.tags["Width"]],
    )
    image_array = (image_array / image_array.max() * 255).astype("uint8")
    return image_array

def get_device_properties(self):
    core = self._core
    devices = core.get_loaded_devices()
    devices = [devices.get(i) for i in range(devices.size())]
    device_items = []
    for device in devices:
        names = core.get_device_property_names(device)
        props = [names.get(i) for i in range(names.size())]
        property_items = []
        for prop in props:
            value = core.get_property(device, prop)
            is_read_only = core.is_property_read_only(device, prop)
            if core.has_property_limits(device, prop):
                lower = core.get_property_lower_limit(device, prop)
                upper = core.get_property_upper_limit(device, prop)
                allowed = {
                    "type": "range",
                    "min": lower,
                    "max": upper,
                    "readOnly": is_read_only,
                }
            else:
                allowed = core.get_allowed_property_values(device, prop)
                allowed = {
                    "type": "enum",
                    "options": [allowed.get(i) for i in range(allowed.size())],
                    "readOnly": is_read_only,
                }
            property_items.append(
                {"device": device, "name": prop, "value": value, "allowed": allowed}
            )
            # print('===>', device, prop, value, allowed)
        if len(property_items) > 0:
            device_items.append(
                {
                    "name": device,
                    "value": "{} properties".format(len(props)),
                    "items": property_items,
                }
            )
    return device_items

async def run(self, ctx):
    mmcore_api = {
        "_rintf": True,
        "snapImage": self.snap_image,
        "getImage": self.get_image,
        "getDeviceProperties": self.get_device_properties,
        "getCameraDevice": self._core.get_camera_device,
        "setCameraDevice": self._core.set_camera_device,
        "startContinuousSequenceAcquisition": self._core.start_continuous_sequence_acquisition,
        "stopSequenceAcquisition": self._core.stop_sequence_acquisition,
        "setExposure": self._core.set_exposure,
        "getExposure": self._core.get_exposure,
        "setProperty": self._core.set_property,
        "getProperty": self._core.get_property,
    }
    viewer = await api.createWindow(
        src="https://gist.github.com/oeway/f59c1d1c49c94a831e5e21ba4c6111dd",
        data={"mmcore": mmcore_api},
    )

api.export(MyMicroscope())`

[oeway]

Hi @Olaa4 Thanks for the report.

Could you check if there is any error in the browser console when you run it. (This is how to open it: https://balsamiq.com/support/faqs/browserconsole/)

[monfort314]

Hi @oeway ! Thanks for an immediate feedback! Yes, actually there is an error in the browser (Please see below). Strangely, anaconda console did not display any errors...

It looks like camera image buffer issue but I could not find any robust solution. Do you have any suggestions? I tried Chrome and Firefox browsers - I got the same error (In MS IE and MS Edge ImJoy plugin to fail to run) Thanks!

Traceback (most recent call last): File "C:\Users\lab-nmsb\AppData\Roaming\Python\Python36\site-packages\imjoy_rpc\rpc.py", line 391, in _call_method result = method(*args, **kwargs) File "", line 33, in get_image tagged_image = self._core.get_tagged_image() File "C:\Users\lab-nmsb\AppData\Roaming\Python\Python36\site-packages\pycromanager\core.py", line 432, in ): instance._translate_call(signatures_list, args) File "C:\Users\lab-nmsb\AppData\Roaming\Python\Python36\site-packages\pycromanager\core.py", line 554, in _translate_call return self._deserialize(self._socket.receive()) File "C:\Users\lab-nmsb\AppData\Roaming\Python\Python36\site-packages\pycromanager\core.py", line 168, in receive self._check_exception(message) File "C:\Users\lab-nmsb\AppData\Roaming\Python\Python36\site-packages\pycromanager\core.py", line 173, in _check_exception raise Exception(response["value"]) Exception: java.lang.Exception: Camera image buffer read failed. mmcorej.MMCoreJJNI.CMMCore_getImage__SWIG_1(Native Method) mmcorej.CMMCore.getImage(CMMCore.java:936) mmcorej.CMMCore.getTaggedImage(CMMCore.java:162) mmcorej.CMMCore.getTaggedImage(CMMCore.java:167) sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) java.lang.reflect.Method.invoke(Method.java:498) org.micromanager.internal.zmq.ZMQServer.runMethod(ZMQServer.java:371) org.micromanager.internal.zmq.ZMQServer.parseAndExecuteCommand(ZMQServer.java:426) org.micromanager.internal.zmq.ZMQServer.lambda$initialize$1(ZMQServer.java:111) java.util.concurrent.FutureTask.run(FutureTask.java:266) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:748)

(anonymous) @ index.js?v=20211029164638:148 Comm._callback @ comm.js:194 Comm.handle_msg @ comm.js:202 (anonymous) @ comm.js:132 Promise.then (async) CommManager.comm_msg @ comm.js:131 i @ jquery.min.js:2 Kernel._handle_iopub_message @ kernel.js:1238 Kernel._finish_ws_message @ kernel.js:1017 (anonymous) @ kernel.js:1008 Promise.then (async) Kernel._handle_ws_message @ kernel.js:1008 i @ jquery.min.js:2 rpc.js:557 Error: Traceback (most recent call last): File "", line 33, in get_image tagged_image = self._core.get_tagged_image() File "C:\Users\lab-nmsb\AppData\Roaming\Python\Python36\site-packages\pycromanager\core.py", line 432, in ): instance._translate_call(signatures_list, args) File "C:\Users\lab-nmsb\AppData\Roaming\Python\Python36\site-packages\pycromanager\core.py", line 554, in _translate_call return self._deserialize(self._socket.receive()) File "C:\Users\lab-nmsb\AppData\Roaming\Python\Python36\site-packages\pycromanager\core.py", line 168, in receive self._check_exception(message) File "C:\Users\lab-nmsb\AppData\Roaming\Python\Python36\site-packages\pycromanager\core.py", line 173, in _check_exception raise Exception(response["value"]) Exception: java.lang.Exception: Camera image buffer read failed. mmcorej.MMCoreJJNI.CMMCore_getImage__SWIG_1(Native Method) mmcorej.CMMCore.getImage(CMMCore.java:936) mmcorej.CMMCore.getTaggedImage(CMMCore.java:162) mmcorej.CMMCore.getTaggedImage(CMMCore.java:167) sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) java.lang.reflect.Method.invoke(Method.java:498) org.micromanager.internal.zmq.ZMQServer.runMethod(ZMQServer.java:371) org.micromanager.internal.zmq.ZMQServer.parseAndExecuteCommand(ZMQServer.java:426) org.micromanager.internal.zmq.ZMQServer.lambda$initialize$1(ZMQServer.java:111) java.util.concurrent.FutureTask.run(FutureTask.java:266) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:748)

at a._decode (rpc.js:768)
at a._decode (rpc.js:812)
at a._unwrap (rpc.js:844)
at rpc.js:260
at Connection._fire (index.js?v=20211029164638:76)
at index.js?v=20211029164638:153
at Comm._callback (comm.js:194)
at Comm.handle_msg (comm.js:202)
at comm.js:132

_encode @ rpc.js:557 _encode @ rpc.js:682 _wrap @ rpc.js:829 i @ rpc.js:893 Promise.catch (async) (anonymous) @ rpc.js:220 async function (async) (anonymous) @ rpc.js:202 _fire @ utils.js:863 (anonymous) @ connection.js:65 rpc.js:768 Uncaught (in promise) Error: Error: Traceback (most recent call last): File "", line 33, in get_image tagged_image = self._core.get_tagged_image() File "C:\Users\lab-nmsb\AppData\Roaming\Python\Python36\site-packages\pycromanager\core.py", line 432, in ): instance._translate_call(signatures_list, args) File "C:\Users\lab-nmsb\AppData\Roaming\Python\Python36\site-packages\pycromanager\core.py", line 554, in _translate_call return self._deserialize(self._socket.receive()) File "C:\Users\lab-nmsb\AppData\Roaming\Python\Python36\site-packages\pycromanager\core.py", line 168, in receive self._check_exception(message) File "C:\Users\lab-nmsb\AppData\Roaming\Python\Python36\site-packages\pycromanager\core.py", line 173, in _check_exception raise Exception(response["value"]) Exception: java.lang.Exception: Camera image buffer read failed. mmcorej.MMCoreJJNI.CMMCore_getImage__SWIG_1(Native Method) mmcorej.CMMCore.getImage(CMMCore.java:936) mmcorej.CMMCore.getTaggedImage(CMMCore.java:162) mmcorej.CMMCore.getTaggedImage(CMMCore.java:167) sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) java.lang.reflect.Method.invoke(Method.java:498) org.micromanager.internal.zmq.ZMQServer.runMethod(ZMQServer.java:371) org.micromanager.internal.zmq.ZMQServer.parseAndExecuteCommand(ZMQServer.java:426) org.micromanager.internal.zmq.ZMQServer.lambda$initialize$1(ZMQServer.java:111) java.util.concurrent.FutureTask.run(FutureTask.java:266) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:748)

at a._decode (rpc.js:768)
at a._decode (rpc.js:812)
at a._unwrap (rpc.js:844)
at rpc.js:260
at Connection._fire (utils.js:211)
at Connection.handleEvent (pluginIframe.js:73)

_decode @ rpc.js:768 _decode @ rpc.js:812 _unwrap @ rpc.js:844 (anonymous) @ rpc.js:260 _fire @ utils.js:211 handleEvent @ pluginIframe.js:73 async function (async) updateImage @ VM27:58 updateImage @ VM27:72 async function (async) updateImage @ VM27:58 toggleLive @ VM27:79 click @ VM28:3 t @ vue.min.js:6 Ir.t._withTask.i._withTask @ vue.min.js:6

[oeway]

Hi, thanks for the info.

Yes, it seems this is the cause of the issue: Exception: java.lang.Exception: Camera image buffer read failed..

And the error is from micro-manager itself, however, I have no idea how to address it.

I did a quick search and found this, maybe you can give it a try: https://github.com/LC-PolScope/LC-PolScope/issues/26#issuecomment-11933124

Otherwise, maybe you can ask in the micro-manager forum?