search

google-deepmind / mujoco

Multi-Joint dynamics with Contact. A general purpose physics simulator.
https://mujoco.org
Apache License 2.0
8.25k stars 823 forks source link

Support for Modern OpenGL Versions in MuJoCo Visualization (Beyond ARB Framebuffers) #2184

Open ShravanTata opened 3 weeks ago

ShravanTata commented 3 weeks ago

The feature, motivation and pitch

Issue:

MuJoCo’s visualization currently relies on a tight dependency with specific versions of OpenGL, particularly using ARB framebuffers. This creates a number of limitations for developers, especially when:

•   Developing cross-platform applications, where OpenGL versions and extensions differ across systems.
•   Integrating MuJoCo with other graphical libraries and tools such as Dear ImGui, which are designed to work with more modern versions of OpenGL.
•   Combining MuJoCo with custom graphics pipelines or leveraging newer OpenGL features.

As a result, the only workaround in many cases is to downgrade the entire application stack to match MuJoCo’s OpenGL requirements, which can become increasingly problematic, or many a time impossible, especially with newer hardware (or softwares for example on MacOS) and development environments that favor modern OpenGL versions.

Questions:

1.  Are there any plans or ongoing efforts to decouple MuJoCo’s rendering pipeline from the older ARB framebuffer-specific version of OpenGL?
2.  If this is not currently planned, could you provide some guidance or suggestions for developers who wish to extend MuJoCo’s rendering to support modern OpenGL versions?

Impact:

This enhancement would improve MuJoCo’s flexibility for developers working on modern software stacks, enhance performance, and open up new possibilities for integrating MuJoCo with additional graphical tools and libraries.

Thank you for considering this!

Alternatives

Related issues: #1778, #1452

Additional context

I am currently using Dear ImGui Bundle in conjunction with MuJoCo for visualizations and have run into this limitation while working with modern OpenGL features. The lack of support for newer OpenGL versions significantly hampers the integration. Platform-specific issues arise, especially in cross-platform applications where OpenGL versions vary, making it hard to ensure a consistent user experience without downgrading everything to the minimum version supported by MuJoCo.

yuvaltassa commented 3 weeks ago

Hi @ShravanTata, thanks for this.

  1. We are well aware of these limitations, they bite us just as often as they bite you 😐
  2. We are exploring alternatives, in particular looking at various external renderers that might serve as a replacement. However upgrading the native renderer would be a very valuable upgrade regardless of these possibilities.
  3. Is upgrading the openGL backend a project you think you might want to take upon yourself? If yes, please get in touch with me at tassa at google dot com. To be clear, we would compensate you for such a contribution.