High-DPI not working

[LinqLover]

I'm trying to use the high-dpi mode of Squeak 6.0 but whenever I open or move SqueakJS on a high-DPI screen, the VM seems to scale up the entire canvas instead of informing the image about the new scale factor via primitiveScreenScaleFactor (which always fails right now).

Shouldn't pixelation already work? I tried something like

https://9090-codefrau-squeakjs-39cuvd00s49.ws-eu53.gitpod.io/run/#pixelated=true&image=/FreshTrunk.image
https://9090-codefrau-squeakjs-39cuvd00s49.ws-eu53.gitpod.io/run/#pixelated=false&image=/FreshTrunk.image

But there was no visible difference.

[codefrau]

SqueakJS currently ignores window.devicePixelRatio which basically tells us how dense pixels actually are on the screen.

If we knew that the image supports highDPI settings, then we would scale the canvas by devicePixelRatio and I guess primitiveScreenScaleFactor should report that value.

So ... is there a way to tell if the image supports high DPI? Or is the call to that primitive the only indication? We can make that work, too.

[LinqLover]

Good question. Afaik there is no other information than whether the image is calling the primitive - but since there is no guarantee that it will use the result, this sounds also imperfect to me.

In the OSVM, newer versions of the VM just assume that the image will care about the DPI factor. But this has drawbacks regarding the display of older images, of course.

Would it be reasonable to use the image format for this decision?

[codefrau]

Are there images that call the primitive but don't respect the result? This would sound like an image bug to me that maybe we can ignore in the VM? All that would happen in that case is that everything looks tiny so you would have to manually adjust the preferences I think.

[marceltaeumel]

In the OSVM, newer versions of the VM just assume that the image will care about the DPI factor. But this has drawbacks regarding the display of older images, of course.

Not quite. We assume that the platform offers scaling if needed. Only macOS 12 dropped that support rather recently. Windows 11 is still fine. OSVM's Linux/X11 support never scaled and so didn't the Linux platform itself.

There are more than one attempts from me to explain that on vm-dev and/or squeak-dev during the last 2-3 months.

Would it be reasonable to use the image format for this decision?

Most recently, Eliot introduced an extra bit in the image header that we can set to indicate whether the VM supports high resolutions or not. However, it is still a lot of effort to support VM-based upscaling in case the platform (here: macOS) does not. Especially if we want to make it per-monitor aware. -.-"

When we find a good design to implement VM-based upscaling on demand for all desired platforms (macOS openGL/metal/codeGraphics, Windows DirectX/OpenGL, Linux X11/framebuffer/...), we might also re-release Squeak 6.0 with that flag set in the header. For now, we do not.

Are there images that call the primitive but don't respect the result? This would sound like an image bug to me that maybe we can ignore in the VM?

See above. I was really surprised by Apple's move here. It's puts even more pressure on our small community. :-( Older images do not try to call that primitive. Only Squeak 6.0 and 6.1alpha do that.

Squeak 5.3 is now also bundled with the latest OSVM release and scales up fine on Windows 11 because there it is a matter of what is in the .manifest file. Windows takes care of the scaling. macOS 12 just ignores its known flags from the Info.plist regarding upscaling. But they now have a "notch compatible full screen mode" ... wohoo ...

[marceltaeumel]

Yes, the image format changed in Squeak 6.0 because we enabled SistaV1. You might want to use this for now as a hack that indocates high-DPI awareness.

[codefrau]

I just tried out-of-the-box Squeak 6.0 on a Mac screen that has PlatformScaleFactor = 2.0 and when I use the menu to change the scale from the default 200% to 100% the host window's size shrinks by half, making every pixel smaller. Is that the intended behavior?

I find that confusing. Can you describe how a VM should behave that cannot change the visual size of the host window, but could support "small" or "big" pixels on a high-dpi screen?

Here's what I think SqueakJS could do:

1. add a "small pixel / high-dpi mode" flag meaning the display bitmap the VM creates would use devicePixelRatio times as many pixels for the given window size, disabled by default
2. add primitiveScreenScaleFactor to answer either 1.0 (if in big-pixel-mode) or devicePixelRatio (in small-pixel-mode)
3. if the image is a 6.0 image (as indicated by the Sista format bit), enable small-pixel-mode by default
4. optionally, we could add a VM option to "force" small-pixel-mode for older images, e.g. by adding highdpi=true to the URL

How does that sound?

[marceltaeumel]

Is that the intended behavior? I find that confusing.

Yes, that behavior mimicks the way platforms treat multi-monitor setups when moving an application window from, for example, a low-resolution monitor to a high-resolution monitor. The effective screen real estate should remain the same. That's why one also changes the host-window size.

If you refer to "100% should mean what the screen thinks is okay", there is a "Show relative scale factor" preference for that. Yet, comparing it to Windows and Linux, it is a rather macOS-specific way of communicating these things to the user.

Can you describe how a VM should behave that cannot change the visual size of the host window, but could support "small" or "big" pixels on a high-dpi screen?

Well, that's how Squeak's regular full-screen mode works. No change of host-window size, meaning still "full screen". Only the size of pixels change.

How does that sound?

Splendid. 😄

[codefrau]

Okay, so commit c6288043b59db529901c9aa8e0b0bf89ac1298d7 adds a highdpi mode via url option and 54a8539549f58afbdec326093f4f6f3c550db577 adds primitiveScreenScaleFactor. And I am somewhat pleased with how trivial these two are 😊

Appears to work, see e.g. https://squeak.js.org/run/#url=https://freudenbergs.de/vanessa/squeakjs&zip=[Squeak5.3-19452-64bit.zip,SqueakV50.sources.zip]&wizard=false&highdpi=true (note highdpi option in url)

I'm somewhat hesitant now to make this the default, even for 6.0 images, because with 4x the number of pixels to draw everything does become slower. What do we think?

[ccrraaiigg]

Agreed. I think it's enough to make it easy to do. SqueakJS still has a speed perception issue.