jMonkey Window on Mac Retina uses pixels instead of coordinates, so the viewport does not fill the window

[solo-seven]

I'm running a MacBook Pro with a Retina display. I have a very basic jMonkey application just trying to make sure everything is working. I set the application resolution as 1024x576 through the AppSettings. The window launches and fills the area according to the screen coordinates, but the viewport is in the lower left corner and appears to just be using pixels. On Retina screens, coordinates do not equal pixels. I have added -Dorg.lwjgl.opengl.Display.enableHighDPI=true -Dsun.java2d.dpiaware=true to the application start parameters which didn't seem to affect anything. Here is a screenshot of what the window looks like:

Here are my AppSettings:

AppSettings settings = new AppSettings(true);
settings.setRenderer(AppSettings.LWJGL_OPENGL41);
settings.putBoolean("RendererDebug", true);
settings.setWidth(Integer.valueOf(System.getProperty("ORS_SCREEN_WIDTH", "1024")));
settings.setHeight(Integer.valueOf(System.getProperty("ORS_SCREEN_HEIGHT", "576")));
settings.setVSync(Boolean.valueOf(System.getProperty("ORS_SCREEN_VSYNC", "false")));
settings.setFullscreen(Boolean.valueOf(System.getProperty("ORS_SCREEN_FULLSCREEN", "false")));

Here are the logs prior to my application:

Sep 08, 2018 9:17:12 AM com.jme3.system.JmeDesktopSystem initialize
INFO: Running on jMonkeyEngine 3.2-stable
 * Branch: HEAD
 * Git Hash: f85624a
 * Build Date: 2018-01-21
Sep 08, 2018 9:17:12 AM com.jme3.system.lwjgl.LwjglContext printContextInitInfo
INFO: LWJGL 3.1.2 build 29 context running on thread main
 * Graphics Adapter: GLFW 3.3.0 Cocoa NSGL dynamic
Sep 08, 2018 9:17:12 AM com.jme3.renderer.opengl.GLRenderer loadCapabilitiesCommon
INFO: OpenGL Renderer Information
 * Vendor: Intel Inc.
 * Renderer: Intel Iris Pro OpenGL Engine
 * OpenGL Version: 4.1 INTEL-10.36.19
 * GLSL Version: 4.10
 * Profile: Core
Sep 08, 2018 9:17:12 AM com.jme3.audio.openal.ALAudioRenderer initOpenAL
INFO: Audio Renderer Information
 * Device: OpenAL Soft
 * Vendor: OpenAL Community
 * Renderer: OpenAL Soft
 * Version: 1.1 ALSOFT 1.17.2
 * Supported channels: 64
 * ALC extensions: ALC_ENUMERATE_ALL_EXT ALC_ENUMERATION_EXT ALC_EXT_CAPTURE 
ALC_EXT_DEDICATED ALC_EXT_disconnect ALC_EXT_EFX ALC_EXT_thread_local_context 
ALC_SOFTX_device_clock ALC_SOFT_HRTF ALC_SOFT_loopback ALC_SOFTX_output_limiter 
ALC_SOFT_pause_device
 * AL extensions: AL_EXT_ALAW AL_EXT_BFORMAT AL_EXT_DOUBLE 
 AL_EXT_EXPONENT_DISTANCE 
 AL_EXT_FLOAT32 AL_EXT_IMA4 AL_EXT_LINEAR_DISTANCE AL_EXT_MCFORMATS AL_EXT_MULAW 
 AL_EXT_MULAW_BFORMAT AL_EXT_MULAW_MCFORMATS AL_EXT_OFFSET 
 AL_EXT_source_distance_model AL_EXT_SOURCE_RADIUS AL_EXT_STEREO_ANGLES 
 AL_LOKI_quadriphonic AL_SOFT_block_alignment AL_SOFT_deferred_updates 
 AL_SOFT_direct_channels AL_SOFT_gain_clamp_ex AL_SOFT_loop_points AL_SOFT_MSADPCM 
 AL_SOFT_source_latency AL_SOFT_source_length AL_SOFT_source_resampler 
 AL_SOFT_source_spatialize
 Sep 08, 2018 9:17:12 AM com.jme3.audio.openal.ALAudioRenderer initOpenAL
 INFO: Audio effect extension version: 1.0
 Sep 08, 2018 9:17:12 AM com.jme3.audio.openal.ALAudioRenderer initOpenAL
 INFO: Audio max auxiliary sends: 2

[tonihele]

Is this happening with LWJGL 2 too or just with LWJGL 3? And also what is the expected outcome? Is it the viewport is in the wrong position or mouse coordinates are invalid or what?

[tonihele]

http://wiki.lwjgl.org/wiki/Using_High_DPI_Mode.html seems to be documentation for the version 2 but the same stuff still exists in the current master repo code. This might be a thing in Windows 8+ too if those super displays are used. I'm not sure is that a thing :) For me this is the first time I hear about this kind of displays.

[solo-seven]

I've only tried it with LWJGL3, but I think it has something to do with the way the viewport is being created. The grey that you see in the image is from a SkyBox, so I'm expecting that to fill the entire screen. You can see the code that I'm using for this at https://github.com/bashburn/dagfinn-torbin-java. I've done some playing around with using direct LWJGL3 with GLFW to create a window and load a OBJ model, and it fills the entire window. I've been scanning through the jME code to try to find where the difference is. I think it might be something with the camera perspective, but I need to dig a little deeper to determine that.

[tonihele]

Well, https://github.com/jMonkeyEngine/jmonkeyengine/blob/master/jme3-lwjgl3/src/main/java/com/jme3/renderer/lwjgl/LwjglGL.java#L558 if you apply the Display.getPixelScaleFactor() factor. Does it help then? I'm not sure architecture wise if this is the correct location nor I don't know does it work. And after that the mouse coordinates are probably off.

[solo-seven]

The problem is that the Display class is no longer in LWJGL3. According to the GLFW documentation, we need to get the framebuffer size to pass to the viewport - http://www.glfw.org/docs/latest/window.html#window_fbsize. In order to do this, we need access to the GLFW Window handle. I'm going to work on finding the right way to get the size from that and get it passed in, since I can't do it directly.

[Thunderforge]

Is this happening with LWJGL 2 too or just with LWJGL 3?

I just ran into this problem myself while building the JME Tutorial #1 sample code (adding viewPort.setBackgroundColor(ColorRGBA.Gray); to better demonstrate this bug). When my project has jme3-lwjgl, I get this:

But when I switch to jme3-lwjgl3, I get this:

Run on a MacBook Pro 13-inch 2017 with macOS 10.14.6.

[grizeldi]

When switching from jme3-lwjgl to jme3-lwjgl? You probably meant to add versions?

[Thunderforge]

@grizeldi Sorry about that. The first screenshot is jme3-lwjgl. The second screenshot is jme3-lwjgl3. Both are version 3.2.4-stable.

[tonihele]

We need to use the glfwSetFramebufferSizeCallback in LWJGL 3. And always set the size from there, not the window.

Additional settings should be added to AppSettings, WindowWidth & WindowHeight. We should create the Window according to the wanted resolution and hook to the framebufferSizeCallBack to trigger the resize and update the AppSettings Width & Height. Also this framebufferSize should be checked on the init phase if it differs from the WindowWidth or WHeight to do an initial resize or at least set the values.

These additional settings I'm not sure do we need or not. Since I'm not sure how these are used inside JME & whether this would be a breaking change for JME apps themselves. The best scenario would be that these values don't need to be tampered with, just provide the correct width & height to the app itself. There is the resize listener.

[pspeed42]

re: update the AppSettings Width & Height

...I don't understand this. The app settings with and height are values the app sets. Not values that the app reads.

What would it mean for the app to set WindowWidth to 9000 and Width to 200? I can't see why it makes sense to have both of these settings.

[tonihele]

That's good. I mean I read width & height in my JME app. But I'm not sure do I need the view port (?) or the window size when I do that. If those can be left alone then this is really simple, just a matter of changing the listener on LWJGL 3 and initializing to correct size, right?

I don't have the hardware to test this. Unless changing DPI in Windows also triggers these scenarios.

[tonihele]

I think I found the offending lines... https://github.com/jMonkeyEngine/jmonkeyengine/blob/0fd70b81c93adde86a3590db49725a0c68fc3618/jme3-core/src/main/java/com/jme3/app/LegacyApplication.java#L303. The camera is setup with the AppSettings width & height. These are fake news on those HDPI things. This is what I was referring to when I talked about those additional AppSettings.

I have here: https://github.com/tonihele/jmonkeyengine/tree/issue-893 a branch that doesn't solve the issue just yet. But if you try the JME test TestResizableApp and try to resize, the problem should go away... Please test it @Thunderforge and/or @bashburn ?

I tried both Linux & Windows with the screen scaling but I'm unable to reproduce the issue. All I get is window size and pixel size are 1:1.

This all being said.. If it works, I still don't have a clear vision how this should be elegantly fixed. The commit in my branch I think is a valid start. But where to put the framebuffer size so that the application can use it....

[tonihele]

I'm leaning towards having the both attributes saved. As we want to in the future still create the window in specific size. So we don't want to override it with framebuffer size. And probably all code expects the width & height to contain the framebuffer size (apart from the LWJGL 3 window creation code)... Right?

[pspeed42]

To be honest, I always expect width/height to be the framebuffer size. Since I have no control over window placement, etc... I have no idea what size the window is going to be or where it's placed and so on.

If I set width to 1280 and height to 720, I want a 1280x720 framebuffer to draw into.

If we want the user to be able to see what the window size is, that's not at all the job of app settings. App settings is for configuring app settings. It's not a feedback class.

[pspeed42]

Why would the width and height be different than what's in app settings? That's what I wonder about. It seems the crux of the issue, no?

[solo-seven]

I haven’t looked at this issue in a couple of years and at the time I just made my own patched build. That being said, I should have some time to test this out this weekend on my MacBook Pro to verify the issue/fix.

-BA

On Thu, Jan 30, 2020 at 2:08 PM Paul Speed notifications@github.com wrote:

Why would the width and height be different than what's in app settings? That's what I wonder about. It seems the crux of the issue, no?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/jMonkeyEngine/jmonkeyengine/issues/893?email_source=notifications&email_token=AADO5O2OWMBKRDO2WAF75IDRAMQRVA5CNFSM4FT7P7AKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEKME3JY#issuecomment-580406695, or unsubscribe https://github.com/notifications/unsubscribe-auth/AADO5OYOZQD3O7H5VSMBGMLRAMQRVANCNFSM4FT7P7AA .

[tonihele]

What was your patch top the issue @bashburn ?

[tonihele]

Why would the width and height be different than what's in app settings? That's what I wonder about. It seems the crux of the issue, no?

Hmm, it is rather hard to fix blind. But should it then try to resize the window to find the buffer size to match the wanted resolution? It looks to me that you do need to create the window first, but then you of course can adjust it to get the size you want... But I'm not sure how that would look on a mac..

[pspeed42]

If I tell JME to create a buffer 1280x720, I should get a buffer 1280x720. Getting the window the right size for that is up to JME.

If the only way to create a buffer in lwjgl is to specify a window size then the API is even more broken than I thought.

[tonihele]

Well, that is kind of clear goal to strive towards. Quite impossible for me to continue though, not having the means to reproduce. Should be maybe now easy for someone to pick this up?

The framebuffer size callback is just needed to calculate new window size then I suppose and should also be checked when creating the window. With that the pixel scale factor can be obtained. This can be even called when the window is moved to another monitor. I'm only concerned how this will then just look on such setup? Kinda sounds that we then force the HDPI support off instead of trying to support it.

https://www.glfw.org/docs/latest/window_guide.html#window_size https://www.glfw.org/docs/latest/window_guide.html#window_fbsize

[pspeed42]

Yeah, I don't know.

I'm just saying from a user's perspective, there could be a dozen really good reasons I picked a very specific 'size'... whether because all of my GUI stuff is generated for that pixel resolution, or I'm trying to generate images/video of a specific size, or whatever. So if the app requests 640x480 pixels then that's what they should get in their framebuffer.

If something else is required for them to make better choices as to eventual buffer size or whatever then that's a separate issue.

[tonihele]

There is also this https://www.glfw.org/docs/latest/window_guide.html#window_scale for any GUIs. Kinda complicated. I guess these are all new things that came with ever increasing resolutions and monitor sizes. And multi monitor on top of this...

[Thunderforge]

I have here: https://github.com/tonihele/jmonkeyengine/tree/issue-893 a branch that doesn't solve the issue just yet. But if you try the JME test TestResizableApp and try to resize, the problem should go away... Please test it @Thunderforge and/or @bashburn ?

I'm happy to help, but I'm a little unsure of how to test it (being both new to JMonkeyEngine and development in general). Is it just a matter of cloning your repo, building jme3-lwjgl3, and then pointing my project to the output dependency?

[tonihele]

I'm happy to help, but I'm a little unsure of how to test it (being both new to JMonkeyEngine and development in general). Is it just a matter of cloning your repo, building jme3-lwjgl3, and then pointing my project to the output dependency?

Well, the easiest is to clone the https://github.com/tonihele/jmonkeyengine/tree/issue-893 branch and run the JME test TestResizableApp from that branch. Then you also don't need to change your project configuration and try to figure our whether it is now using the local version or not etc.

Also only the said test is kinda fixed, when you try to resize the window it should be fixed. The initial state will still be buggy.

[Ali-RS]

Is this issue still exist in the master branch?