[RunSofa] Grabbing the OpenGL viewport

[marques-bruno]

Hi guys,

I coded a component a few months ago that was designed to grab the OpenGL window of Sofa. It was pretty simple, a glReadPixels on the GL_VIEWPORT, and it was working fine. Today though, I ran a scene that was using this component, and noticed that it isn't only grabbing the sofa scene, but also the whole user interface.

I know the old GUI is destined to dissapear slowly, but the issue will be the same in sofaqtquick. It would be really interesting to be able to grab a specific opengl context from a widget for instance, or directly from the sofa scene.

I don't think the way I'm doing so is a great idea, and to be honest I don't know what would be a clean way of doing so, especially give how slow a glReadPixels() is...

It might be a very similar topic as the Headless recorder from @ErwanDouaille ? Except that I don't want to lose the GUI just to capture images

Any ideas?

---

Suggested labels:

[ErwanDouaille]

From what I understand, you want to grab images from a sofa scene wihtout the GUI and without the QT gui ? If yes, the headlessRecorder might be a good solution.

I know the old GUI is destined to dissapear slowly, but the issue will be the same in sofaqtquick. It would be really interesting to be able to grab a specific opengl context from a widget for instance, or directly from the sofa scene.

I don´t know how the current gui works so I can´t help :/ In my opinion. it depends of QT. QGLWidget + QGLFrameBuffer might be the trick.

From your idea, I like the "component" design. Be able to grab opengl context from any QWidget doesn´t make sense to me. In this case the only canvas you want is the canvas which display the sofa´s scene

[marques-bruno]

@ErwanDouaille , It's not exactly what I meant:

I would like to keep Sofa's GUI as is (not batch mode or anything like that. What I would like to be able to do though, is grabbing only the scene of sofa. An example might be more explicit: This picture is a screenshot of a Sofa scene where my component grabs the OpenGL viewport with glReadPixels() and then displays it back in Sofa's scene. If everything would work as I want it to, this scene shouldn't do anything (the grabbed image would be the displayed image, so I would constantly have the classic black background with the SOFA logo.

Instead though, I'm getting the whole UI on the left, the menu on top and runSofa's counter bar at the bottom.

(In addition the framebuffer is flipped on the X axis but that's just a bug in my code... =) The purpose of this is to perform image processing directly on the scene that's generated in Sofa.

In the case of the new GUI, you could imagine a scene that has multiple cameras, displaying in multiple widgets, and be interested in performing some post processing stages on these generated frames, so being able to retrive the framebuffer coming from those widgets could be useful.

[damienmarchal]

Hi bruno,

I think that the problem may be cause by the fact that the dimension of the gl viewport is now covering the whole screen (maybe because qt is using opengl for its internal rendering or something like that ?). To set the dimensions of the area to copy I assume you are using the viewports values from the VisualParams and not the one from glViewport otherwise this may explain where your problem come from.

On your picture I see that you have implemented a ScreenGrabber component to get the content of the 3D view of a Sofa scene. I think a cleaner design for what you want to do is the following:

• have in-scene cameras that specify the rendering viewpoint.
• connect this in-scene camera to a render texture (so the rendering is done not in the front buffer but in an hidden FrameBufferObject of specific dimension).
• the FrameBufferObject can then be used either to be retrieve in the main memory (with glRead) or without copy as an input texture of any other rendering operation you may need (texturing a plan or do GPU processing using GLSL).

This design have the advantage of scaling well and allows to have multiple in-scene cameras to render the values into multiple RenderTexture.

Having this kind of RenderToTexture feature in Sofa would be quite useful and is worth the effort to make it properly.

Damien