[Question] Flexx for this use-case?

[Correct-Syntax]

Hi, this project looks very promising!

I was wondering a few things about flexx and I thought I should ask some questions to save myself and potentially others a lot of time and work:

1. I have read https://pscript.readthedocs.io/en/latest/intro.html#performance about performance and https://pscript.readthedocs.io/en/latest/intro.html#support and was wondering if flexx is suitable for an image-editing application?
   • This would require a lot of speed to transfer an image between python -> js (not sure how that works with flexx).
     2. Is there support for any library that can be installed via pip, including wrappers of c-based libraries (e.g: numpy, opencv, oiio)?
     3. Can widgets be made custom or are they confined to a set widget set (like in some other similar libraries)?

Thank you for your time!

[almarklein]

if flexx is suitable for an image-editing application?

This depends a lot on the architecture of your application. If the image editing happens in JS, then yeah, it should be fine. If the image data is to be send back and forth, then ... maybe it will be fast enough, maybe not.

If it is fast enough, depending on what you want to do, you should keep in mind that your application may become more complex than it would if you'd build it with Qt. I cannot recommend it for "big" projects. Just saying.

(not sure how that works with flexx)

Websockets that send binary data, numpy arrays are supported. So it's about as fast as it can be, for a client-server architecture.

Is there support for any library that can be installed via pip, including wrappers of c-based libraries (e.g: numpy, opencv, oiio)?

If you mean can I use c-based libraries in JS, then no. Only pure Python that is compatible with PScript can be converted. Using something from a 3d party library may work, but usually some part of the code will not be compatible.

Can widgets be made custom or are they confined to a set widget set (like in some other similar libraries)?

No you can go crazy with custom widgets. Have a look at some of the examples.

[Correct-Syntax]

Thanks for your reply.

Well, my setup would be a client-server setup anyway. Python would handle the image processing side fully and all the client side needs to do is receive the numpy image and display it and be the UI. But, it sounds like flexx isn't for this kind of thing, correct?

Is it possible to use flexx just as the UI and still use python? Or is that defeating the point of flexx?

Side-question: do you happen to know of any better alternatives to where an application could be implemented in python, but the UI with web-tech (HTML, CSS, JS)? I am surprised that python doesn't have very many good options for doing this (at least, from my research).

Thanks again!

[almarklein]

Oh, you can use normal Python as well. There's also a PyWidget class, which is a widget that operates on the server side. Inside it, you can then create a common widget that operates in JS, and send actions/reactions back and forth. It's like magic! And sometimes you will be confused whether your Python code will run server-side or client-side, so beware to be consistent in the structure of your application.

Is it possible to use flexx just as the UI and still use python? Or is that defeating the point of flexx?

That would actually be the original intended way to run Flexx. It will also keep your code manageable.

Do you happen to know of any better alternatives to where an application could be implemented in python, but the UI with web-tech (HTML, CSS, JS)?

Well you could do a more classical web app with Flask or Starlette or any of the many web server frameworks. In your case, preferably one with a simple way to use websockets.

[Correct-Syntax]

Websockets that send binary data, numpy arrays are supported. So it's about as fast as it can be, for a client-server architecture.

I just a general sense, since the image data is going through localhost, how fast can it be expected to be using websockets alone? Would there be issues getting the image to update quickly on the HTML canvas (it has to send the image when a change is made to the image and update the canvas)?

Thanks!

[almarklein]

I dunno, and it will obviously also depend on the size of the images. You could try create a benchmark app that does nothing but sending images over, and see how fast it goes.

[Correct-Syntax]

You could try create a benchmark app that does nothing but sending images over, and see how fast it goes.

Indeed, I have as per your advice. :)

Using:

await websocket.send(image_np_array.tobytes())

on the server side and:

function on_message(recv_data)
{
  var recv_image_data = new Uint8Array(recv_data.data);

  var canvas_image = document.getElementById('canvas_image');
  var ctx = canvas_image.getContext('2d');
    var imageData = ctx.createImageData(1920, 1080);
    for (var i=0; i < imageData.data.length; i++){
      imageData.data[i] = recv_image_data[i];
    }
    ctx.putImageData(imageData, 0, 0);
}

on the client side.

The benchmarks with the time module on the server side was around 4sec (which may or may not be totally accurate), and the total time spent (server-side) before the await websocket.send(image_np_array.tobytes()) was about 0.8 sec. This was with a pretty high-res, 2.5MB image as a numpy array (preloaded).

So, it would seem the bottleneck is definitely in the websockets and js, client side. I am not sure if the looping on the client side is efficient, but I am not sure how else to do it.

How have you implemented this type of thing for flexx, if I may ask? Do you know if there a way to access the numpy array from the python side in js using something like a memoryview or buffer rather than sending the bytes through websockets?

[almarklein]

I wonder how much of that time is sending over the ws, and how much is the drawing.

One thing you could try, is convert the image to a PNG-encoded URI, as e.g. here: https://github.com/plotly/dash-slicer/blob/main/dash_slicer/utils.py#L38-L48. Then you have compression, although part of that advantage is undone because of the base64 encoding, but more improtantly, a PNG like this can be drawn real fast by the browser (just set it to the src attribute of an <img> element).

How have you implemented this type of thing for flexx, if I may ask?

What do you mean by "this type of thing"? The data is encoded using bsdf. Numpy arrays are basically send over by sending the bytes, as well as the shape and dtype info, so that it can be reconstructed in JS. The array will thus have a shape attribute, but is in fact a 1D array (because JS does not have nd arrays).

[Correct-Syntax]

I think I'm going to try doing a few tests with flexx and see if it would be workable.

It did seem that you didn't think it was such a good idea for this type of thing, though, correct?

[almarklein]

That depends :) Is it your intention to create an image editing app in Python? Then maybe using Qt would be easier (maybe add numba for speed). If your intention is to create something web-based, but prefer writing Python, than Flexx should be a good match.

[Correct-Syntax]

Is it your intention to create an image editing app in Python?

The intention would be to have the image processing, file IO, data storage, etc in Python, but the UI (preferably) in HTML, CSS, JS, etc. The reason being that the complex layouts and interactions we need would (probably) be much easier done in web-tech than in a toolkit like Qt or wxPython (I could be wrong on that, though....).

The main reason I looked into flexx was because of pretty-much the same reasons listed under "Motivation" in the README, plus doing the UI in flexx seems like it would be much more flexable -pun intended. :)

Then maybe using Qt would be easier (maybe add numba for speed).

Well, Qt seems to be out of the picture for this project as it would force the license to be GPL and we'd rather not have it be GPL. Also, for the reasons above.

If your intention is to create something web-based, but prefer writing Python, than Flexx should be a good match.

Hmmm....well, that's not exactly the case. It almost sounds like your explaining flexx as like python in place of js -which is pretty neat. ;)

Not sure if Pyodide or projects like that are mature and good enough for something like this.

[almarklein]

If your intention is to create something web-based, but prefer writing Python, than Flexx should be a good match.

Hmmm....well, that's not exactly the case. It almost sounds like your explaining flexx as like python in place of js -which is pretty neat. ;)

Well, I should have set write JS with a Python syntax ;)

as it would force the license to be GPL

Not with PySide.

[Correct-Syntax]

Not with PySide.

Is PySide still alive?

As you said, it doesn't seem that flexx is a good choice for this. Thanks for this discussion, even though some of it doesn't have a lot to do with flexx. ;)

[almarklein]

Is PySide still alive?

Yes, it has been "revived": https://wiki.qt.io/Qt_for_Python

[Correct-Syntax]

Yes, it has been "revived": https://wiki.qt.io/Qt_for_Python

Okay.

Thanks for answering my questions. ;) I think I was asking a bit of a "xy question" and I now see that the web UI thing is probably not the way to go in this case.