jMonkeyEngine3-based visualizer proof-of-concept

[tangmi]

Hi @breiler et al.! Thank you for maintaining a lovely program that continues to surprise me with useful features (I just recently learned of the visualizer's "jog to here" action).

I was looking into filing an unrelated issue and stumbled on your comment here https://github.com/winder/Universal-G-Code-Sender/issues/737#issuecomment-486951520 suggesting changing the rendering backend for the visualizer. Despite not being a Java dev (or familiar with UGS, jMonkeyEngine, etc), curiosity got the better of me and after some hacking I've come up with a (highly incomplete) visualizer plugin that uses jMonkeyEngine3 with LWJGL2.

First, some screenshots:

Benefits

• Performance improvements: jME3 implements a more modern renderer than the current visualizer. Since the UGS visualizer scene complexity is low, we should be able to always hit the display refresh rate with time to spare.
• Anti-aliasing: We can get this for free by configuring the framebuffer to be multisampled. This especially improves the many thin lines of Gcode files. I was able to get anti-aliasing on the current renderer by enabling sample buffers and upping the sample count on GLCapabilities, but it had some issues with visuals ghosting that I didn't investigate further into.
• Monitor scaling support: jME3 seems to support DPI scaling, so we'd get this for free.
• No more raw OpenGL: the jME3 scene graph API feels straightforward and simple to use. It allows us to group together the implementation of e.g. rendering the orientation cube and detecting clicks on its faces (I have an implementation of this in this change). Note that I'm not familiar with Java OO programming and one could definitely make even better use of the engine. I think LWJGL can target down to OpenGL 2.0, so it should work everywhere the current visualizer does.

Why jMonkeyEngine3?

I evaluated some other options and here are my notes:

• jMonkeyEngine3
  • w/ JOGL: It seems like UGS uses JOGL, but the jme3-jogl backend was removed a couple years ago (though it might come back, maintained by JogAmp). There doesn't seem to be a backend today I can test with.
  • w/ LWJGL3: jme3-lwjgl3 does not support a JmeCanvasContext, which is required for embedding in AWT.
  • w/ LWJGL2: This is the backend that ended up working for me, but it's not without issues. There's a dependency packaging issue with 3.6.0-STABLE (though allegedly [https://hub.jmonkeyengine.org/t/solved-failure-to-find-org-jmonkeyenginejar-2-9-5-after-upgrading-to-3-6-0/46838/8}(fixed in 3.7.0-SNAPSHOT)).
  • None of the jME3 backends I looked at allow the application itself to drive new frame updates. This means the view will be constantly re-rendering itself even when there are no changes. I think this is what the current visualizer does (at a much lower framerate limit) and it is probably not an issue except in the case of very low power machines or when maximum power efficiency is a requirement.
• Ardor3D: This was suggested in your comment-- it seems that the project has forked since then.
  • JogAmp's Ardor3D Continuation 1.0: This seems to only have one contributor, not many recent changes, low activity on its forums, and doesn't publish to Maven Central. It requires updating to Java 21. Also, I wasn't able to get it displaying in the UGS visualizer test harness (this is probably a "me" issue, not the library).
  • Renanse/Ardor3D: This is the original project that was allegedly abandoned by its owner. Renanse seems to be reviewing PRs and submitting releases as recently as this year. The latest release on Maven central seems to be 1.0.0 in 2019. I was not able to poke the pom.xml file into accepting this dependency, though if you can depend on GitHub releases and are willing to drop JOGL, this seems like a good option.

jMonkeyEngine3 is much better documented and for sure has more community support and active development. However, it feels like a heavier and more opinionated engine when compared to Ardor3D. Ardor3D, from my cursory look, feels less like a game engine and more geared towards things like visualizations, etc. It allows the application to control the main loop and [drive frame updates](https://jogamp.org/deployment/ardor3d/javadoc/com/ardor3d/framework/FrameHandler.html#updateFrame()). Overall, the Awt integration seems more robust in Ardor3D.

What next?

If this change seems useful, I think some possible next steps could be:

• Have the UGS folks evaluate the jME3 API (especially the Wiki, which gives a great high-level overview of the API) or investigate what it would take to add Ardor3D as a dependency.
• Make a list of all the features the visualizer supports today.
• Port the existing visualizer to jME3 or Ardor3D, aiming for feature parity but not an exact 1:1 translation.
• Start adding new features, like orientation cube face picking, improving UX, etc.

I'm happy to discuss and help where I can, but I'll definitely need help with Maven and UGS integration.

I am currently on holiday and will soon not have much free time to work on this (sorry!).

Thank you again for UGS and thank you for time!

[breiler]

Thanks for looking into this!

I did some similar experiments trying to port from JOGL to LWJGL some time ago, but stumbled on the poor AWT integration. I could make it work decently in a standalone program but as soon as it was integrated with the NetBeans platform I got into problems when for instance resizing the window.

Unfortunately I get the same problem when trying your sample. My conclusion is that LWJGL is not suited to run in a desktop application. If these problems can't be fixed to make LWJGL run smoothly I think that we are stuck with JOGL. I looked at the old jme3-jogl library and it was using the older JOGL 2.3.2 which has problems running on both ARM32, ARM64 and MacOSX.

The Renanse/Ardor3D fork seems to also have dropped the support for JOGL.

So if we want a scene graph we will probably have to roll our own... 😞

[tangmi]

Thank you for taking a look! Bummer that it seems every option here is not without its own issues.

Unfortunately I get the same problem when trying your sample.

Out of curiosity, what are the resize issues you're seeing? I'll admit I was only able to figure out how to run the Visualizer3TestMain and not UGS itself (again, a "me" problem), but I did not see any issues at least on Windows (at least no more than normal for OpenGL 😄).

The Renanse/Ardor3D fork seems to also have dropped the support for JOGL.

It looks like the JogAmp fork has been keeping the JOGL backend support up to date. IIUC it was forked originally because the Renanse/Ardor3D repo was abandoned, but that doesn't seem to be the case anymore. gouessej seems aware of this and has ported some of the changes over, I wonder if renanse and gouessej can be convinced to re-merge the forks..?

So if we want a scene graph we will probably have to roll our own... 😞

I wonder if we can identify what's going wrong with LWJGL and get some help fixing it upstream. I'm not very familiar with LWJGL and JOGL, but my understanding is that they're pretty straightforward bindings to OpenGL with extra implementation to deal with input, windowing, etc. I imagine issues with resizing may be somewhere in that windowing code. It looks like LWJGL3 might use GLFW for windowing so perhaps better AWT support may be possible?

[breiler]

Out of curiosity, what are the resize issues you're seeing? I'll admit I was only able to figure out how to run the Visualizer3TestMain and not UGS itself (again, a "me" problem), but I did not see any issues at least on Windows (at least no more than normal for OpenGL 😄).

If you start UGS Platform and open your Visualizer3 panel and try to resize it. For me I can only make the panel larger - it will prevent me from making it smaller. When doing resizing or moving the panel to a different pane it is extremly sluggish.

It looks like the JogAmp fork has been keeping the JOGL backend support up to date. IIUC it was forked originally because the Renanse/Ardor3D repo was abandoned, but that doesn't seem to be the case anymore. gouessej seems aware of this and has ported some of the changes over, I wonder if renanse and gouessej can be convinced to re-merge the forks..?

Gouessejs version is distributed here as a snapshot: https://jogamp.org/deployment/maven-ardor3d/org/jogamp/ardor3d/ I tried adding the repository and allow snapshot builds, but I got stuck with the Java version as he is using Java 20. But that could may be a way forward...

I wonder if we can identify what's going wrong with LWJGL and get some help fixing it upstream.

If we managed to get it working properly on all (macosx x64/arm64, Linux x64/arm/arm64, Windows x64) platforms I'd prefer this over JOGL as it seems to have more developers. I am currently working on some other stuff, I might take this up later.

[tangmi]

Happy new years! Thanks for your replies!

If you start UGS Platform and open your Visualizer3 panel and try to resize it. For me I can only make the panel larger - it will prevent me from making it smaller. When doing resizing or moving the panel to a different pane it is extremly sluggish.

Strange, I'm not seeing that on Windows 10. I am, however, seeing a much lower frame rate and no multisampling, along with other general weirdness.

If we managed to get it working properly on all (macosx x64/arm64, Linux x64/arm/arm64, Windows x64) platforms I'd prefer this over JOGL as it seems to have more developers. I am currently working on some other stuff, I might take this up later.

I found lwjglx/lwjgl3-awt, which seems like a good implementation of AWT integration-- I assume you're already familiar.

I tried to hackily port UGS to lwjgl to investigate what AWT issues there were (https://github.com/winder/Universal-G-Code-Sender/compare/master...tangmi:Universal-G-Code-Sender:try-switch-to-lwjgl) but I think I'm making some mistakes in building-- I was not able to get the maven files or build commands to consistently build successfully... I don't think I know enough about Maven or NetBeans to be touching the build files productively 😄.

In my change linked above, I think I'm just running into an issue where the visualizer needs to depend on lwjgl and friends but doesn't have dependencies in the class path (?). If there's a smoking gun here, please let me know. Otherwise, I may need help with the dependencies or may need to wait until UGS switches over to jME3/Ardor3D before being able to contribute graphics changes.

Thanks again for your time!

[breiler]

I resurrected my ancient branch when I was looking at this: https://github.com/breiler/Universal-G-Code-Sender/tree/bugfix/lwjgl

It works okish on Linux.

On MacOSX the stand alone application works, but when integrating in NetBeans it didn't refresh the window properly.

[tangmi]

Thanks for the link, I've updated my branch and got it building and running on Windows. I'll see what I can do with the macOS issues.

~edit: FYI looks like lwjgl3-awt doesn't support arm64 mac. I'm working around this by using the x64 JDK (and running into other java issues). I'm not sure what it takes to package java libs but it may just be asking nicely to build for arm?~

[tangmi]

I did a little more digging and will write up some notes here. I'm going to close this for now to keep your PR queue clean.

• lwjgl3-awt does support arm64 mac, but does not support macOS >10.15 due to thread requirements introduced on NSOpenGLContext.
• ~The weird resizing jank you were seeing may have to do with the custom NSView that was implemented for LWJGL2.~ Nevermind, I realize you were using lwjgl3-awt also.
• As far as I can find, there's no LWJGL3-based AWT integration that works past macOS 10.15 (jMonkeyEngine continues to use LWJGL2 for its AWT examples)
• It might be possible to use ANGLE (which is an implementation of OpenGL ES on top of Metal), and use lwjgl3-awt's Metal/AWT integration, which is currently working for Vulkan. This would require a code change to lwjgl3-awt to support the Metal view with OpenGL ES and would require UGS to find a way to distribute ANGLE binaries (an example could be https://github.com/libgdx/gdx-angle-natives).