connect and control Image output stream from BK ultrasound through pyigtl

[Nicolaas94]

Hello, I am able to connect with a BK5000 ultrasound system through PlusServer. I visualize the output stream with CustusX. I have the ultrasound transducer attached to a motor with a stepwise movement. The stream of images is however continuous. This means I don't know which image corresponds to which step. I would like to connect with the BK5000 with pyigtl and control the image stream by code, parallel to the movement stream by code.

I found the pyigtl package with examples which I tried without success.

client = pyiglt.OpenIGTLinkClient(host="169.254.66.146", port=7915) message = client.wait_for_message("ProbeImage", timeout=3) print(message)

The printed output is 'None'. What could be the problem? as device_name I used "ProbeImage" as this is the name in the config files for PlusServer.

A second thing a tried was to start launching the PlusServer with a successful connection with BK5000. And then set up an internal client. Without success unfortunately:

client = pyigtl.OpenIGTLinkClient(host="127.0.0.1", port=18944) message = client.wait_for_message("ProbeImage", timeout=3) print(message)

Thanks for any help!

[ChristophG123T]

@lassoan I'm also having this issue where it says 'None' with my webcam. I've tried: -Making the host my desktop's ip address, and using port 18945. -Adding a configuration on PyCharm to the script with --address (desktop ip) and then --port 18945.

I'm trying to connect a webcam using the aruco tracker config file on port 18945.

Edit: I changed the device_name to "Image_Image" and this works. However, I'm unsure how to take the real-time image of my webcam (which is 'client') and display it on a qt application. Does ImageMessage() do this? I'll look into the package pyigtl.

[lassoan]

Most likely the device name in the messages is not exactly ProbeImage. I would recommend to connect to the server using 3D Slicer to verify that image streaming works correctly and see the actual device name.

[ChristophG123T]

Thanks! I noticed this earlier today and changed it to Image_Image. Now it works and prints a matrix along with the image size etc. However, I'm now trying to stream the image from the webcam client to a qt program. Right now, I'm sending an image "message" to the Plus server from the webcam client. However, I'm unsure how to take this message and implement it. In the following code from example_tracked_image_server.py, what does the array store, and do you know if it's compatible with qt? I think I only need the image_message for my application.

    # Send messages
    server.send_message(image_message)
    server.send_message(transform_message)
    server.send_message(string_message)

    # Print received messages
    messages = server.get_latest_messages()
    for message in messages:
        print(message.device_name)

Chris

[lassoan]

The message content is specified here: http://openigtlink.org/developers/spec

I would not recommend to implement Qt applications with background processing in Python, from scratch. It would be a lot of work, in particular incompatibility of Qt and Python multithreading it would be a lot of pain figuring it all out.

What kind of application would you like to implement? Augmented reality using ArUco tracking, surgical navigation using optical tracking, image guided application using ultrasound imaging and electromagnetic tracking,...?

[ChristophG123T]

My goal with this application in the long term is to develop a 3D carotid artery segmentation using Clarius. Right now I’m just trying to get a webcam image streaming, and then the Clarius streaming. My supervisor (Elvis Chen) wanted to get the python only IGTL working to display the image, although I might use VTK to display the image instead of qt. Would this route be better? To my understanding, Slicer uses vtk to display Plus images, and 3D models. I already have a simple qt program set up using a demo from the Clarius developers, but I want to modify it to show Plus images rather than their binaries. Our computers have trouble allowing their demo to stream through our firewall, and we want to show 3D models eventually.

By the way, we want a python only version because we want to use this on an architecture (arm64) that doesn’t support Slicer (Adam Rankin said Plus can be built on arm64 but not Slicer itself.) and also so users don’t need to download all of Slicer (~200MB iirc) for this application.