w3c / window-management

Window Management API
https://www.w3.org/TR/window-management/
Other
95 stars 25 forks source link

Provide Physical Dimensions / PPI of Displays #65

Open seflless opened 2 years ago

seflless commented 2 years ago

It would be very useful to be know the size of displays in physics units. This has applications in responsive designs to model users and in displaying graphics realistically.

Returning a ppi (pixels per inch) or display dimensions (size of entire display covered in pixels) in the payload of window.getScreens() would be awesome.

michaelwasserman commented 2 years ago

Thanks for proposing that enhancement! This may be possible in some circumstances, but not always (e.g. projectors). Additional requests and use cases will help inform future extensions of per-screen information.

seflless commented 2 years ago

Hi @michaelwasserman, sorry for the slow response. I came up with a lot of simple use cases, some shared in this tweet thread (Watch the 27 second video and the 2 or 3 responses below).

Design Tools Previewing

I think the first strong use case for web apps to have access to each displays sizes is in web based creative tools. Examples:

Shopping Item Sizing

It would reduce a lot of anxiety, reduce errors, and likely save a lot of time for people when they are shopping for physical items online. Examples:

electronicwhisper commented 2 years ago

In case it is useful to the drafting group or others reading this thread, here is how we do this at Cuttle.

Context: Cuttle is a web-based CAD tool for laser cutting. We provide a View > Zoom to Real Size feature which is helpful because you can hold up existing parts to your monitor to spot check that you got your dimensions correct before you fabricate.

image

We need to know the pixels-per-inch of your display in order to implement this feature. We assume people might use different monitors (e.g. laptop screen vs external display) at different times. We keep in localStorage a mapping from a key that we hope is unique per monitor (derived from window.screen and window.devicePixelRatio). For each key, we store the pixels-per-inch. If we don't already have the pixels-per-inch then we prompt the user to measure their pixels-per-inch with this dialog:

image

If web standards gave us the pixels-per-inch of the current display (similarly how window.devicePixelRatio is provided) then we could just use that number without having to prompt the user to measure their screen with a ruler or credit card. As @michaelwasserman mentions, it's not always possible to know the physical dimensions of the screen (like a projector) so I'd rather have the standard say when the physical screen size is unknown rather than making up a (wrong) value!

(Due to the keystone effect, projectors also create arbitrary perspective distortions, so a single pixels-per-inch value is likely insufficient to describe the mapping from pixels to physical units in this case. I assume this is outside the scope of the standard at this time, though I can go deeper into this based on my experience at Dynamicland if you want to go down this rabbit hole!)

It would be great if this feature were to make it into the standard. It would enable more digital-physical hybrid applications.

michaelwasserman commented 2 years ago

I really appreciate the detailed use cases you've both provided, thank you! This seems like a great enhancement request.

According to the definitions of window.devicePixelRatio and ScreenDetailed.devicePixelRatio it seems like they should be directly applicable for this purpose, given their basis in CSS Pixel, which is anchored to some physical measurement (1px = 1/96th of 1in) and the screen pixel's physical height. So, are user agents simply incorrectly implementing these attributes, or are these definitions actually inherently unusable for web applications to display actual physical sizes without calibration?

Do non-web applications provide similar functionality without calibration? It seems like EDID screen size values>) (plus screen resolution) could be used as the basis to expose the physical dimensions / PPI of displays.

seflless commented 2 years ago

I don't think user agents ever implement this stuff correctly. My impression is that device manufacturers have to decide when they want to report their coordinate as >1 devicePixelRatio which will likely effect the style of UIs because of responsive design. So usually wanting to ensure mobile, tablet, or desktop centric layouts are triggered by the typical app when using their device.

Browser API Experiments

I've tried various tricks using each css unit system and they all tend to report the same fixed values regardless of device. Try opening this page on any device or each screen of a single device to see for yourself (Make sure to refresh the browser when moving this window to a new display in a multi-monitor setup): https://codesandbox.io/s/ancient-wave-f8uci?file=/src/index.js. This is based on reading back the pixel dimensions based on an element set with the inches units: <div style="width: 1in;"></div>. Here are the results versus the reality from 3 recent tests:

All devices report 96ppi but have the real PPIs of:

Native APIs for Querying PPI

When I implemented an Electron based prototype for this in 2017, for macOS only, I was easily able to query for this information. I'm sure the latest APIs are based on the same approach. The Windows equivalent looks pretty similar. Here's the source code for my prototype. Here it is inlined:

long targetScreenId = atoi(argv[1]);
long count = [[NSScreen screens] count];

// Find the screen with the same screen ID
for(int i = 0; i<count; i++){
    NSScreen *screen = [[NSScreen screens] objectAtIndex:i];
    NSDictionary *description = [screen deviceDescription];
    NSNumber* screenId = [description objectForKey:@"NSScreenNumber"];

    if([screenId longValue] == targetScreenId){

        NSSize displayPixelSize = [[description objectForKey:NSDeviceSize] sizeValue];

        CGSize displayPhysicalSize = CGDisplayScreenSize(
                                                         [screenId unsignedIntValue]);

        // There are 25.4 millimeters per inch:
        // https://www.google.com/webhp?sourceid=chrome-instant&ion=1&espv=2&ie=UTF-8#q=mm+to+inches
        printf("%0.2f",(displayPixelSize.width / displayPhysicalSize.width) * 25.4f);

        // Exit successfully
        return 0;
    }
}

What's next if this is interesting to this standards group? Anything else you'd like to make a compelling case?

seflless commented 2 years ago

Something to note, my example code above doesn't factor in pixelDeviceRatio or what is called scale in Swift or backingScaleFactor in Objective-C. Here is a touched up full XCode project that will print out the ppi and devicePixelRatio of each monitor on a mac. PrintPPI.zip

michaelwasserman commented 2 years ago

Thanks again for the great info. We will consider our options to support these use cases more thoroughly as the spec continues to advance through the standardization process. I encourage other web application developers to express support for this request and share their use cases, as that can increase prioritization of this enhancement request.

patrickhlauke commented 1 year ago

possibly related (though the discussions there we always more around "can we change CSS to make mm etc be anchored on physical sizes) https://discourse.wicg.io/t/one-inch-is-not-an-inch/1228 https://discourse.wicg.io/t/one-inch-is-not-an-inch/1228 https://discourse.wicg.io/t/display-an-image-at-device-s-physical-resolution/1150

nicksherman commented 1 year ago

Yes please. Some more related content below for additional context …

Most relevant is a similar issue I submitted back in 2016. Warning: it is a loooong thread of discussion, and it was closed as "won't fix" (though the comments remain active with other request for the issue to be reopened): https://github.com/w3c/csswg-drafts/issues/614

Here's an article I wrote on the topic almost 10(!) years ago: https://alistapart.com/column/responsive-typography-is-a-physical-discipline/

A challenge on Twitter that still stands after almost 10 years: https://twitter.com/NickSherman/status/320655148240224257