vinavfx / ComfyUI-for-Nuke

API to be able to use ComfyUI nodes within nuke, only using the ComfyUI server
GNU General Public License v3.0
193 stars 17 forks source link

Add binary data check for live preview messages. #24

Closed claussteinmassl closed 2 months ago

claussteinmassl commented 2 months ago

This is a fix for this issue: https://github.com/vinavfx/ComfyUI-for-Nuke/issues/4

The error occurs, if ComfyUI is sending live previews as jpeg raw data. In that case the incoming message is binary data and can't be decoded to json. This is just a simple fix for now. Maybe we can do something more useful with this data, like display it inside Nuke to the user.

vinavfx commented 2 months ago

I had to revert it because the progress freezes, I checked it and everything that arrives in message are bytes so with this you would be skipping everything

vinavfx commented 2 months ago

This would be good to load the image directly into the pixels of the Nuke viewer without having to create files. I am already not sure how to do it, since BlinkScript does not allow pixel entries other than through the input, a plugin would have to be made in C++ to work as a link between the bytes to be painted in the viewer.

claussteinmassl commented 2 months ago

I had to revert it because the progress freezes, I checked it and everything that arrives in message are bytes so with this you would be skipping everything

Hmm, I'm seeing a different behaviour here. For me the code worked perfectly fine and the progress was updated as expected. I printed the type for every incoming message and for me it looks like this:

` message type: <class 'str'>

message type: <class 'str'> message type: <class 'str'> message type: <class 'str'> message type: <class 'str'> message type: <class 'str'> message type: <class 'str'> message type: <class 'str'> message type: <class 'str'> message type: <class 'str'> message type: <class 'str'> message type: <class 'str'> message type: <class 'str'> message type: <class 'str'> message type: <class 'str'> message type: <class 'str'> message type: <class 'str'> message type: <class 'str'> message type: <class 'str'> message type: <class 'str'> message type: <class 'bytes'> message type: <class 'str'> message type: <class 'str'> message type: <class 'bytes'> message type: <class 'str'> message type: <class 'bytes'> message type: <class 'str'> message type: <class 'str'> message type: <class 'bytes'> message type: <class 'str'> message type: <class 'bytes'> message type: <class 'str'> message type: <class 'str'> message type: <class 'bytes'> message type: <class 'str'> message type: <class 'bytes'> message type: <class 'str'> message type: <class 'str'> message type: <class 'bytes'> message type: <class 'str'> message type: <class 'bytes'> message type: <class 'str'> message type: <class 'str'> message type: <class 'bytes'> message type: <class 'str'> message type: <class 'str'> message type: <class 'str'> message type: <class 'str'> message type: <class 'str'> message type: <class 'str'> message type: <class 'str'> message type: <class 'str'> message type: <class 'str'> message type: <class 'str'> message type: <class 'str'> message type: <class 'str'> message type: <class 'str'> message type: <class 'str'> message type: <class 'str'> message type: <class 'str'> message type: <class 'str'>`

I submitted a batch of 10 images, that's why I seeing 10x the preview image in bytes. Without my modification the JSON can't be decoded for those messages arriving as bytes and the process terminates with this error:

"ERROR:websocket:error from callback <function progress..on_message at 0x000002B5677BEF80>: 'utf-32-be' codec can't decode bytes in position 8-11: code point not in range(0x110000)"

vinavfx commented 2 months ago

Could you send me the script you used, because I use Linux and now I'll have to check it on Windows to see what happens

claussteinmassl commented 2 months ago

I'm on Mac OS actually ... the most simple example would be your LivePortrait Gizmo / the AdvancedLivePortrait Node with just driving & source images. The Expression Editor that we used in our other tests didn't show this behaviour. I will further investigate to see, when this issue comes up.