Disorted colors on SW pipeline

[streamingdv]

Upon further testing my application I encountered another strange issue and I'm not sure what could be the reason for that. So far I'm able to reproduce the problem on a virtual Ubuntu OS running in Virtual Box and on my Steam Deck (I have a Linux notebook with a Nvidea GPU and strangly there it works). From the rendering perspective everything works fine but the colors are completly disorted (not visible at all, too bluish). I have some custom shaders in place but to be completly sure it is not caused by my application I tried to run your test applications (Shared.java) on my Windows PC and on my Ubuntu OS on Virtual box. You can find below a link to the video I made to demonstrate the issue

https://drive.google.com/file/d/1vmZaQSV23MDmRGXu_0yXWDBv5cqT48o5/view?usp=sharing

Please note that on both machines the same code is executed.

Below you can find an image of the disorted rendering problem shown by my program. On the left side is the program executed on Windows and on the right side executed on Ubuntu in Virtual Box.

Do you have any ideas what could cause this issue? I can confirm OpenGL works fine with other rendering programs which are using OpenGL so I think it has something to do with the lib. I'm unfortuently not yet very experienced with OpenGL so I'm a bit lost.

Edit: I use LWJGL (as in your examples)

[husker-dev]

This may be due to different usage of OpenGL in JavaFX on different operating systems. Try this before creating canvas:

OpenGLCanvas.forceUniversal = true

if the problem disappears, then it's all about JavaFX. And I will have to adapt to its rendering

[streamingdv]

@husker-dev I tried that already and it doesn't make a difference. Problem is still the same

[streamingdv]

Any ideas what might be wrong? Are you able to reproduce the problem?

[husker-dev]

Have you tried to use jogl? Also try to play with core/compatibility settings and change lwjgl and JavaFX versions.

It looks like an issue with pixel format. It displays depth buffer or something similar instead of color.

I will try to reproduce this issue on my pc in a few days

[streamingdv]

No not yet, will try with jogl different java versions (i used java 17 lts). Core/ compatibility didn't make a difference.

Okay great, looking forward to your results. Maybe you have to use a VM with Ubuntu. It works on my main Linux dev machine but not on the steam deck or inside the VM so not sure what's the root cause. Just want to mention that again.

[streamingdv]

Run into another issue when trying the Shared example with jogl (same with forceUniversal to true)

kotlin.UninitializedPropertyAccessException: lateinit property gl has not been initialized
        at com.huskerdev.openglfx.jogl.JOGLExecutor.glViewport(JOGLFXExecutor.kt:62)
        at com.huskerdev.openglfx.core.implementations.UniversalImpl.onNGRender(UniversalImpl.kt:94)
        at com.huskerdev.openglfx.OpenGLCanvas$NGOpenGLCanvas.renderContent(OpenGLCanvas.kt:178)
        at javafx.graphics/com.sun.javafx.sg.prism.NGNode.doRender(NGNode.java:2072)
        at javafx.graphics/com.sun.javafx.sg.prism.NGNode.render(NGNode.java:1964)

With LWJGL the same problem occurs with JavaFX 13 (minimum required version by your lib) so no difference when switching the version. Also no difference when using compatibility or core mode. Waiting for your results now.

[streamingdv]

Some further investigation, on my native Linux dev machine with with Nvidea GPU it works (as mentioned before) but when I enable

OpenGLCanvas.forceUniversal = true

It crashes the entire application/ JVM. I'm not sure if this bug report is only a forcedUniversal mode problem to be honest but on SteamDeck it should never go to the universal mode by default I assume. But this distorted color problem still persists on the virtual Ubuntu and on SteamDeck (no difference when forcedUniversal is used and no crash).

[streamingdv]

@husker-dev did you have time to check it out already? Don't want to spam here but would like to share the resuts of my investigation. So far this problem seems to be a UniversalImpl problem. I tried to use forceUniversal on Windows and it works. I encountered two different problems on Linux machines when Universal fallback is used

1. The colors look distorted (everything else seems to work from the rendering perspective). This happens on Ubuntu in my Virtual machine and on SteamDeck.
2. On my dev Linux machine with Nvidea GPU the VM crashes when I use forceUniversal = true. I attach here a VM crash log hs_err_pid68070.log

I was curious why my SteamDeck is using the Universal implementation so I checked the rendering pipline and indeed for some unkown reasons it uses the SWPipline instead of the ES2Pipeline. Did some further testing and with -Dprism.forceGPU=true it works fine then as the correct ES2Pipeline is used, no distorted color problem anymore with the ES2Pipeline. So problem seems to be a UniversalImpl only issue.

[husker-dev]

Thanks for the testing!

Your investigation explains a lot. The only problem that can be in UniversalImpl is the wrong type of destination texture, which is what causes distortion. Given that everything works fine on ESPipeline and D3DPipeline, this is a problem exclusively in SWPipeline. I knew there was a problem with that pipeline. But to be honest, JavaFX's built-in 3D doesn't work in this case either. Moreover, on my PC there were some artifacts in the display of some UI with SW. So I figured it could be skipped.

Fixing the wrong texture type in this case will not be easy, because JavaFX gives a limited choice in pixel format.

I think this is also some kind of bug in JavaFX that turns on the SW pipeline on some PC.

I'm on a trip right now, so I can't really do much with this project. As soon as I have access to a better PC, I will immediately try to pay attention to this issue.

[husker-dev]

Fixed in commit https://github.com/husker-dev/openglfx/commit/247b449d8fab7b129a728ceb3f25899309744979. New version will be released in a few days.

[streamingdv]

Awesome, thanks a lot 👍