search

husker-dev / openglfx

OpenGL implementation for JavaFX
Apache License 2.0
80 stars 10 forks source link

Question about rendering queue #33

Closed grill2010 closed 1 year ago

grill2010 commented 1 year ago

Like already said several times, this library seems to be amazing. Exactly what I needed for my project. I'm a beginner to all this OpenGL stuff, so sorry if something doesn't make sense.

I'm currently playing around with the lib, in my project I receive a real time video stream which I decode with ffmpeg. I try to display the decoded AVFrames as fast as possible via OpenGL. It works so far but from what I have seen yet the whole mechanisme in openglfx works via the rendering callback which you can either trigger manually via the repaint method or if you specify a GLCanvasAnimator, so my question is, is there any possibility that I can run in the OpenGL context thread to redraw my canvas without the need to use the rendering callback?

For explanation and how it is currently, I receive the raw h264/ h265 stream from the server and I decode it via ffmpeg. Afterwards I put the decoded AVFrame in a BlockingQueue and then I trigger the repaint method. In the canvas rendering callback I run the OpenGL drawing code in a while loop which only exits when the BlockingQueue is drained (I read in a none blocking way in this loop from the queue). At first I tried to draw the frame via OpenGL right after the fffmpeg decoding but then I learned that this is not allowed because I'm in a different thread and not in the OpenGL context. This rendering callback is producing some unecessary overhead, so in the end it would be sufficient for me if I can enter the OpenGL rendering thread somehow, there I would start listening for raw stream data, start the decoding and then immediately do the OpenGL rendering (all in one place and same thread).

I don't know if this is possible at all? I'm aware that this is some kind of special request and project but just wanted to ask if you have some advice for me. Thanks in advance.

husker-dev commented 1 year ago

The rendering thread is created on JavaFX side. All we can do is to request the repainting. We can change OpenGL thread using makeCurrent, but it is very expensive and buggy solution in case with JavaFX.

I think that with this question it is better to go to the forums, or to Google. There will be the best solutions.

grill2010 commented 1 year ago

Okay I see, thanks for the clarification. This makeCurrent is it expensive in general or just when you call it? As far as I know this must only be called once, so the overhead should not be that huge? Don't get me wrong, overall everything runs fine I just play around with the lib and try to minimize the overhead.

husker-dev commented 1 year ago

The performance of makeCurrent depends on the implementation on both the software and hardware side.

In case with openglfx, makeCurrent will need to be called at least four times.

  1. In OpenGLFX thread to unbind the context makeCurrent(0)
  2. In your thread to bind the context makeCurrent(context)
  3. In your thread to unbind the context makeCurrent(0)
  4. In OpenGLFX thread to bind the context back makeCurrent(context)

But all the benefits are lost, because when onRenderEvent is called, the context must remain current after the end of the function.

Otherwise, in the best case, the openglfx simply will not work, and in the worst case, the entire javafx application will not be rendered (on MacOS and Linux)