husker-dev / openglfx

OpenGL node for JavaFX
Apache License 2.0
88 stars 12 forks source link

Crash upon app start when calling 'NVDXInterop.Companion.isSupported()' on Windows 10/ 11 running on a VM #79

Open streamingdv opened 7 months ago

streamingdv commented 7 months ago

Describe the bug [text here] During startup I have following logic to set the prism order

    private static void setPrismOrderForWindows(final VideoRenderingType videoRenderingType) {
        if (NVDXInterop.Companion.isSupported()) {
            final PreferenceManager preferenceManager = PreferenceManager.getInstance();
            final boolean wasOpenedSuccessfully = preferenceManager.wasOpenedSuccessfully();
            if (Objects.equals(VideoRenderingType.OPEN_GL, videoRenderingType) && wasOpenedSuccessfully) {
                Logger.info("Running on Windows. DxInterop is supported on system but OpenGL was requested, set prism order to es2,d3d,sw. Rendering engine is {}", videoRenderingType);
                preferenceManager.saveOpenDetails();
                System.setProperty("prism.order", "es2,d3d,sw");
            } else {
                Logger.info("Running on Windows. DxInterop is supported on system, set prism order to d3d,es2,sw. Rendering engine is {}", videoRenderingType);
                preferenceManager.deleteOpenDetails();
                System.setProperty("prism.order", "d3d,es2,sw");
            }

            if (Objects.equals(VideoRenderingType.OPEN_GL, videoRenderingType) && !wasOpenedSuccessfully) {
                preferenceManager.updateVideoRenderingType(VideoRenderingType.AUTO);
            }

            return;
        }

        Logger.info("Running on Windows. DxInterop not supported on system, set prism order to es2,sw. Rendering engine is {}", videoRenderingType);
        System.setProperty("prism.order", "es2,sw");
    }

However the application crashes when calling NVDXInterop.Companion.isSupported(). I tried it in a Windows 11 and Windows 10 x64 environment running on Virtual box. Here is the crash report

hs_err_pid4124.log

To Reproduce Build a basic JavaFX application include the code NVDXInterop.Companion.isSupported() and try to run it on Windows 10 or Windows 11 in a Virtual Box environment.

Environment:

husker-dev commented 7 months ago

It looks like a fail in creation of OpenGL context. Can you try to create it manually at the beginning of the program using GLContext.create(...)?

If it works, then try also getting it after creation using GLContext.fromCurrent()

Virtual Box may not have graphics acceleration support

streamingdv commented 7 months ago

Yes that might be right, I can try that.

However it would be nice if the isSupported method would not crash so that applications can handle that appropriately. I can't control where users will run my application on and in such cases I would rather show a warning or something, not sure if such special cases where no graphics acceleration support is present can be handled by OpenGLFX?

husker-dev commented 7 months ago

I think it's possible. I'm currently doing some code rewriting, so I'll take into account