[resize-observer] device-pixel-border-box size

[gregwhitworth]

device-pixel-border-box size

device-pixel-border-box size is Element's border-box size in device pixels. It is always an integer, as there are no fractional device pixels. It can currently be approximated by Math.round(borderBoxSize * window.devicePixelRatio), but it cannot be computed exactly, because native code uses a different rounding algorithm.

Use case

This unusual size request comes from Chrome's WebGL canvas team. It solves the long standing WebGL developers problem: "How to create HiDPI canvas without moire pattern?".

The existing "best practice" for creating a HiDPI canvas is to set size of canvas context to a multiple of canvas's css width/height. Example:

let cssWidth = 300, cssHeight = 200;
let c = document.querySelector('canvas');
c.style.width = cssWidth + "px";
c.style.height = cssHeight + "px";
let ctx = c.getContext("webgl2");
ctx.width = cssWidth * window.devicePixelRatio;
ctx.height = cssHeight * window.devicePixelRatio;

The webgl context will be HiDPI, one one canvas pixel should correspond to one device pixel. But, because ctx.width is pixel snapped, ctx.width can differ from "true" device pixel width. This difference can cause visible moire patterns when rendered.

Because of this, WebGL team believes that web platform needs to provide an API for true device pixel width.

Discussion

This size has several interesting differences from others reported by ResizeObserver:

• use case is narrow: only useful for canvas Elements.
• changing only Element's position, not size, can cause device-pixel-size to change.

Q: Does this size belong to ResizeObserver, or should we create a diferent DOM API for it?

I can't think of a clean API that would provide same functionality. Web developers must observe this size, and respond to its changes. ResizeObserver is the only size-observing API. Observing border-box size, and providing "devicePixelSize()" method will not work, because devicePixelSize could change without border-box changing.

Q: Should we observe device-pixel-size on all elements, or just canvas?

Observing device-pixel-size comes with performance cost, because size must be checked when Element's position changes. For all other sizes, we do not need to check size when position changes. Weak preference: Only allow device-pixel-size observation for canvas.

Q: Should we report device-pixel-size on all elements, or just canvas?

Weak preference: make it canvas-only, because other elements cannot observe this size.

Originally posted by @atotic in https://github.com/w3c/csswg-drafts/issues/3326#issuecomment-440041374

[gregwhitworth]

We discussed this today and we decided the following:

device-pixel-border-box size is just another word for border-box but for the canvas element when in a WebGL context for reporting device pixels since they can't be rounded.

In order to keep the API clean, we're recommending removing this as a value and just doing the right thing for canvas by reporting the device pixels alongside the CSS pixels under new properties.

[css-meeting-bot]

The CSS Working Group just discussed Device pixel border box removal from spec.

The full IRC log of that discussion

<emilio> topic: Device pixel border box removal from spec
<astearns> github: https://github.com/w3c/csswg-drafts/issues/3554
<emilio> github: https://github.com/w3c/csswg-drafts/issues/3554
<emilio> gregwhitworth: so, effectively what this comes down to is that this was a size you could watch, for canvas scenarios this'd be useful because CSS pixels get rounded in some way
<emilio> gregwhitworth: proposal is that for <canvas> only you'd provide the device pixel size of the same dimensions
<emilio> Rossen: that's super weird
<emilio> gregwhitworth: I agree, that's why aleks was going to propose this one
<emilio> iank_: I can do an argument
<emilio> gregwhitworth: the use case is there, the question is whether it makes sense to put the information there instead of in the canvas interface
<dbaron> +1 to thinking maybe it belongs on canvas, although interested in hearing the pro argument tomorrow
<emilio> gregwhitworth: we can wait for aleks
<emilio> everyone: Let's move this until tomorrow

[css-meeting-bot]

The CSS Working Group just discussed ResizeObserver device-pixel-border-box.

The full IRC log of that discussion

<gregwhitworth> Topic: ResizeObserver device-pixel-border-box
<gregwhitworth> Github: https://github.com/w3c/csswg-drafts/issues/3554
<gregwhitworth> atotic: I've been working on the spec
<gregwhitworth> atotic: the graphics team came to me - the problem they have is a way to determine the device pixel size of canvas
<gregwhitworth> atotic: why do they need to do this - the reason why they need to do this - is there is no way to do this
<gregwhitworth> atotic: the way the devs do hidpi in canvas it has CSS Pixels and what they do is they create the bitmap size they create the canvas in physical pixels it gets shrunk to CSS pixels so they can draw really fine hairlines
<gregwhitworth> atotic: device pixels are never rounded - there is no doubles or floating point, etc
<gregwhitworth> atotic: how we do this is by pixel snapping, there is no way for webdevs to determine this
<gregwhitworth> atotic: pixel snapping depends on not only pixels but also position
<gregwhitworth> atotic: I asked to add it to canvas, and they said sure but they also need to be notified when its resize
<gregwhitworth> atotic: currently the proposal will be that you will get device pixel inline and border size if its a canvas
<gregwhitworth> fantasai: are you proposing to add it somewhere else
<gregwhitworth> atotic: no
<gregwhitworth> fantasai: so the only way to get this is to add the resizeObserver
<gregwhitworth> atotic: yes
<gregwhitworth> dholbert: does devicePixelRatio?
<gregwhitworth> atotic: they get a bad pattern from hacking this because it can't do pixel snapping correctly
<gregwhitworth> dbaron: under what conditions does Chrome change the device pixel backing size when it's repositioned?
<gregwhitworth> dbaron: atotic is saying that, if you make a subpixel change in the top coordinate of a canvas, if it's 100.25 px tall
<gregwhitworth> dbaron: and it's top is positioned at a pixel aligned position it would round to 100, if it's further down it will be 101
<gregwhitworth> dbaron: do you actually change the backing store
<gregwhitworth> dbaron: the number of image pixels in the canvas
<gregwhitworth> dbaron: I guess thats the height and width
<gregwhitworth> dbaron: they want to change those the attrs based on the device pixel sizes
<gregwhitworth> atotic: the developer determines the size of the backing store, what Chrome does it will copy that bitmap to the CSS size of the canvas, if the ratio is a nice even number then we don't get moar patterns - if we don't
<bkardell_> how is that different from a url background image?
<gregwhitworth> atotic: there is a CSS size which can be fractional - but there is also the real bitmap to paint the canvas and that bitmap is in physical pixels
<bkardell_> " if the ratio is a nice even number then we don't get moar patterns - if we don't"
<bkardell_> that bit
<TabAtkins> s/moar/moire/g
<gregwhitworth> dbaron: bkardell_ is asking why don't you get moire pattern in background images
<gregwhitworth> atotic: it will scale, but background image is fundamentally different
<bkardell_> I can unmute and talk, I just dont want to interupt and don't knwo if this is enough to warrant a q
<gregwhitworth> atotic: we probably do get them but you usually aren't doing hidpi background image or you can't tell
<gregwhitworth> atotic: where they primarily have this issue is in the 3d context and they'll have patterns that look different
<AmeliaBR> q+
<gregwhitworth> bkardell_: the act of painting a thing on a canvas - the bitmap comes from an image and we rescale the image - it shoudln't matter who creates this image?
<gregwhitworth> atotic: you will see artifacts - but they'll create 3 different images
<dbaron> So I understand the use case -- but I'm a little suspicious of making it <canvas>-only, though.
<gregwhitworth> iank_: within painting, we know where the pixel snapping is so we can account for it
<gregwhitworth> atotic: also we're talking about every frame in video games, it happens a lot more
<gregwhitworth> dbaron: I understand the usecase I just don't know if I like that it's canvas only
<florian> q+
<gregwhitworth> atotic: I agree, but there is a performance penalty to calculate them and most people don't care about this except in a canvas context
<dbaron> I'd have expected it to be implemented a different way, but okay...
<gregwhitworth> atotic: if we find that it's useful in other areas it's easy to extend it
<gregwhitworth> AmeliaBR: if this is a canvas thing - why are we doing this here. Especially since this is a really hacky way to do it
<gregwhitworth> atotic: from what I understand from the history, there was a hidpi way in which to do this but the other standard didn't go anywhere
<gregwhitworth> hober: we also saw that people still create the larger one
<gregwhitworth> atotic: I agree RO is an odd place but the dimensions change if you move around you still need to be notified
<gregwhitworth> liam: checking the device pixel size, it seems easy to check every frame
<AmeliaBR> ack me
<gregwhitworth> liam: why would you not check it?
<gregwhitworth> atotic: if you're a game developer maybe
<heycam> q+
<gregwhitworth> frremy: If you want to draw a perfect line in your table you'll need to resnap it
<gregwhitworth> atotic: exactly, you need it or you add it to both specs
<heycam> ack florian
<emilio> q+
<gregwhitworth> florian: another thing - here we know what CSS & device pixel things and very easy to get confused
<gregwhitworth> florian: I don't know exactly what we want to add so let's not add it to too many places because it will get misused
<emilio> ack heycam
<gregwhitworth> atotic: it would only happen on canvas - you would see inlineDevicePixel and blockDevicePixel
<Rossen> gregwhitworth, recaps the resolution from yesterday
<gregwhitworth> heycam: before you were seeing a perf penalty - why not add a new box watching keyword?
<gregwhitworth> heycam: you can still have the device pixel border box of another element
<gregwhitworth> florian: but that was my point - if you make it broadly available they'll abuse it
<gregwhitworth> florian: they'll think they want device pixels but they'll probably be incorrect
<bkardell_> q+
<gregwhitworth> liam: to be clear, the only usecase this solves compared to RO plus having a method on canvas
<bkardell_> q-
<florian> q-
<dholbert> q+
<gregwhitworth> liam: this canvas snapping happens during painting or layout? If it happens at painting you wouldn't want to wait for painting to be done
<emilio> s/liam/emilio
<gregwhitworth> smfr: the only way to really know this you is to do it at painting you have to take transforms into account
<gregwhitworth> Rossen: if you have a scale it's broken
<florian> ScribeScribe: Florian
<gregwhitworth> gregwhitworth: I'm against adding this based on what smfr and liam said
<gregwhitworth> atotic: it will be blurry anyways so it doesn't matter
<emilio> s/liam/emilio again :D
<florian> gregwhitworth: in your use case, you may be right, but if we add this, there may be people who want to use it in the more general case that includes tranforms
<florian> gregwhitworth: so taking all web devs into account, this looks like a partial solution, not good enough
<gregwhitworth> iank_: I'll need to ask Chris, but we may compute the transforms
<gregwhitworth> atotic: resizeObserver doesn't watch transforms
<gregwhitworth> smfr: then this is the wrong API to bolt this onto
<gregwhitworth> atotic: ok, the hidpi with an ancestor that has a transform or something
<gregwhitworth> atotic: that should still work because it will then get transformed
<gregwhitworth> florian: then it's broken because the second you scale it's broken
<gregwhitworth> florian: people will assume they can and it will fail
<gregwhitworth> atotic: I thought one of the issues on the webplatform you don't want to expose zoom because it will break
<gregwhitworth> heycam: this is different
<heycam> s/this/pinch zooming and transforms on ancestors/
<gregwhitworth> atotic: we were talking about the pinch zoom of the page and they would end up with arbitrarily long page
<gregwhitworth> florian: pinch zoom, leaving it out - but transform?
<gregwhitworth> atotic: if we were going to do transforms, there will be a subset of these that should work
<gregwhitworth> atotic: we want this to be a size API not a quad API
<florian> gregwhitworth: I feel this is a chrome only perspective, safari and Firefox don't seem up to it. Should we go back to the github issue?
<gregwhitworth> atotic: RO itself does not work with transforms
<gregwhitworth> fremy: If you're going to draw a canvas you're not trying to do a transform on top of that
<gregwhitworth> fremy: transform is for animations
<gregwhitworth> fremy: why would you intend to do this if you're focused on pixel perfect drawing
<gregwhitworth> liam: I agree it doesn't make sense, it does make sense to center a canvas
<gregwhitworth> florian: offset path will also impact this as it can change the position
<gregwhitworth> atotic: we're going to notify you when the border box changes
<gregwhitworth> atotic: there's nothing that says we're not going to change on the transforms
<gregwhitworth> dholbert: how integral is the movement of the subpixel
<gregwhitworth> atotic: when you're moving it's blurry anyways, it's a video game
<gregwhitworth> dholbert: I'm feeling like a static API is more desirable than an RO API
<bkardell_> it is possible to make a subclass phase 2 of this that was specifically for observing that?
<bkardell_> .. or something
<gregwhitworth> dholbert: the canvas API makes sense to have this
<gregwhitworth> atotic: but people want to get an observation
<fremy> q+
<emilio> ack emilio
<dholbert> q-
<gregwhitworth> AmeliaBR: is it reasonable to trigger RO on something may have changed even if the sizes haven't changed - they then can do a canvas pixel
<gregwhitworth> atotic: I thought of that - but you end up getting a bunch of ROs without knowing why
<gregwhitworth> AmeliaBR: we have a way of adding device-pixel-border-box back?
<gregwhitworth> atotic: where most observers they look at the border box or content box, what is this box and creates a larger turbelance
<gregwhitworth> AmeliaBR: maybe I don't care about those and you're adding a perf hit
<gregwhitworth> heycam: you can rename the type of the box, canvas-bitmap-box if you're worried about
<gregwhitworth> atotic: it kind of locks us into watching the canvas only
<gregwhitworth> fantasai: I think agree with atotic named canvas - so we shouldn't lock that down
<fantasai> s/named/about not naming specific to/
<gregwhitworth> smfr: we simply snap device scale factor - if you're inside the scale transform we don't snap all the time, so we do need to go outside of layout a bit.
<fantasai> fantasai^: because there might be other elements in the future where we want to use it, and we should be able to just re-use the same thing for those cases
<gregwhitworth> atotic: it has a perf hit
<gregwhitworth> smfr: if you drag to another screen would it fire
<gregwhitworth> atotic: yes
<gregwhitworth> smfr: it's not real device pixels, because your not taking transforms into account
<gregwhitworth> chrishtr: its the dims of the texture of the backing store
<gregwhitworth> smfr: what you just said is very impl specific
<gregwhitworth> chrishtr: it's the CSS pixel size snapped and multiplied by the devicePixelRatio without taking into account transforms
<gregwhitworth> chrishtr: it has to take into account zoom
<gregwhitworth> smfr: ours doesn't
<gregwhitworth> chrishtr: the spec says to
<gregwhitworth> dholbert: doesn't it fire after layout?
<gregwhitworth> atotic: yes
<gregwhitworth> dholbert: so after layout
<gregwhitworth> dholbert: it would need to be extended to take into account after the fact
<gregwhitworth> liam: I don't think this a great fit
<gregwhitworth> iank_: it's where all other resizes are triggered, it's actually bad to try to keep it in sync with RO
<myles__> q+
<dbaron> So about 20 minutes ago there was a suggestion to take this back to the github issue... is there something we're actually going to resolve on here?
<gregwhitworth> fantasai: one thing to note if we have a RO if your 100 CSS pixels and I move it to the hidpi screen it's still 100 CSS pixels. And it's a canvas specific issue
<gregwhitworth> fantasai: it would be nice to not get the extra events unless I actually care about the device pixel border box
<gregwhitworth> atotic: yes
<gregwhitworth> fantasai: I think it should be the content box since that's what you're painting into
<dbaron> atotic: you could register to observer device-pixel-content-box
<gregwhitworth> atotic: that was my original proposal
<chrishtr> q+
<xfq> ack my
<gregwhitworth> myles__: so in conjuction with what fantasai was saying then you need to know the pos and size.
<gregwhitworth> fantasai: no only if those pixels change
<florian> gregwhitworth: that's already in the algo
<gregwhitworth> fantasai: you can resize in result of pos, but it may not
<Rossen> ack fremy
<gregwhitworth> fremy: I think it's going to be tricky in a sense, for the usecase your using - have you considered the paint() API?
<dbaron> (sounds like people were leaning towards registering for a separate box name rather than giving these notifications magically in some case)
<gregwhitworth> iank_: we don't have webgl and a bunch of other APIs
<gregwhitworth> fremy: oh ok
<fantasai> Proposal: Register for changes to devicePixelContentBox, get back sizes in device pixels, throws an error if registered on anything other than a canvas element, fires whenever device pixel size changes whether due to change in screen resolution, element position, or element resizing. Does not fire if device pixel size does not change.
<smfr> q+
<dbaron> still doesn't deal with the issues about rectilinear transforms, offset-path, etc.
<Rossen> ack chrishtr
<gregwhitworth> chrishtr: I just wanted to follow up on explicitly defining the API, in the CSS paint worklet callback you can see the device pixel backing size. It helps devs to dinstinguish between the two and can discover it. I think it would be useful to distinguish
<gregwhitworth> atotic: I think we have agreement to watch a seperate box?
<astearns> zakim, close queue
<Zakim> ok, astearns, the speaker queue is closed
<gregwhitworth> smfr: I just want to say that I don't think canvas is the only one, I think other situations may care about the same thing
<gregwhitworth> smfr: I think people will want it in other ways
<astearns> ack smfr
<smfr> s/in other ways/on other elements/
<gregwhitworth> smfr: I was proposing this is the snapped content box multiplied by device pixel ratio
<gregwhitworth> dbaron: but you all agree to ignore transforms?
<gregwhitworth> chrishtr: because it should be a paint based situation for perf
<gregwhitworth> dbaron: I don't think that's actually true
<gregwhitworth> chrishtr: unless overflow it's almost post paint
<gregwhitworth> krit: even SVG?
<AmeliaBR> I like the connection with Paint API. Even comes with a nice name, "paint size": https://drafts.css-houdini.org/css-paint-api/#paintsize
<gregwhitworth> Rossen: my proposed closing of the issue - let's bring the proposal back to github and then bring it back and we can resolve then
<gregwhitworth> atotic: thank you
<TabAtkins> Ooh, "paint size" sounds good.

[dbaron]

I think this is proposing to make observable some details of the Chromium transforms implementation and how it interacts with painting that aren't specified anywhere. I'm not sure we want to make those things observable -- but if we do, we definitely need a spec for them so that other browsers can match them.

[gregwhitworth]

To summarize the very large thread above, the request was placed on @atotic to create a new box with a better name and how the resulting API shape would change, if any.

In addition, there was discussion around when pixel snapping is done which seems to vary to some extent (pinch zoom for example) between engines and will need to be addressed (although not necessarily in the RO spec).

[kenrussell]

The problem of not being able to know exactly how many device pixels a given Canvas covers exists in all browsers, not just Chromium. All browsers' layout engines and compositors do some sort of rounding or snapping, and all of them do it subtly differently. For this reason it is not possible to create pixel-accurate canvas content. There are some longstanding bugs filed against WebGL on this topic:

https://github.com/KhronosGroup/WebGL/issues/2460 https://github.com/KhronosGroup/WebGL/issues/587

In the past the WebGL WG discussed adding properties to the canvas exposing this information, but doing so is sub-optimal. If the surrounding elements of the canvas changed, layout needs to occur in order to compute the values of these properties, so fetching them would induce a synchronous layout, and could induce infinite loops in applications. @grorg pointed this out years ago and there was no good solution.

ResizeObserver is a great innovation and a great place to put these additional observable properties. It allows the application to respond asynchronously, and 100% correctly, to resizing of the canvas. @atotic prototyped these new observable properties in Chromium and the results were excellent - they finally allowed perfectly pixel-accurate canvas content to be drawn.

Could we please reconsider adding device-pixel-border-box as an observable property? If refinements are needed we're happy to collaborate on them. Thanks.

[gregwhitworth]

@kenrussell I don't believe that's what @dbaron was meaning. Yes pixel snapping occurs and this will vary by browser as it isn't specified anywhere, nor is subpixel rounding. What David was getting at is when pixel snapping is done and if it includes transforms or not, Chromium does not include transforms - thus if the element is transformed the dimensions snapped may be incorrect if it's a position+size altering transform as it will result in different rounding which isn't observed by ResizeObserver which primarily captures Layout adjustments.

That said, while device-pixel-border-box is not currently in the spec, this issue is to work out how to go about it and what the tradeoffs are.

So to put it another way, are you ok with that tradeoff, being able to get device pixels in the general case but some authors may not if they apply certain transforms?

[dbaron]

In particular, it sounds like, in Chromium, the pixels being snapped to are a function of what does or doesn't have a layer in the compositor... which means that this is exposing that, and thus likely causing some sites to depend, for correct behavior (not just for performance), on particular layerization. I think Gecko might pixel snap across rectilinear transforms (translate and scale only), whereas it sounds like Chromium doesn't.

[kenrussell]

Sorry for the delay replying.

It's fine with me personally - and I suspect for most web authors - if this API works most reliably and portably only if no size-altering or rotational transforms are applied to the canvas. That should be the case for applications that are really trying to render 1:1 to device pixels, since I doubt they will apply any transformations to the canvas.

I don't know Chromium's compositor and it would be good to get confirmation from someone on that team that this direction is OK with them. Not sure who on that team watches this repo but I see that @chrishtr can be CC'd; Chris, do you have any feedback or suggestions?

Perhaps we can further specify and standardize the behavior as we get experience with it, and as multiple browsers implement it? It would be great to start somewhere.

[chrishtr]

Re compositing: in Chromium, it is not the case that the pixel snapped size depends on compositing. It depends only on the local layout size of the element.

Re pixel snapping across transforms: Chromium pixel-snaps across rectilinear translations, but not scales. It also does not pixel-snap across CSS containment isolation. (ref: FragmentPaintPropertyTreeBuilder::UpdateForPaintOffsetTranslation in the code)

The pixel-sizing of the canvas depends on pixel snapping, but is not scaled up to include scales of ancestor transforms.

This means that the device-pixel-border-box will change if certain ancestor transforms change. device-pixel-border-box also depends on paint offset generally.

I think we should just specify that if a ResizeObserver is configured to observe device-pixel-border-box, it needs to take into account all such sources of rounding and difference like I enumerated above. This is more expensive, and so should be an opt-in only for developers who need it.

[kenrussell]

The WebGPU community group discussed this at length in a recent face-to-face meeting, with WebGL working group members also present. This ResizeObserver feature is strongly desired for both graphics APIs. People principally involved in the discussion were @atotic (Google), @grorg (Apple), @litherum (Apple), @jdashg (Mozilla) and @kenrussell (Google). Perhaps @Kangz (Google) can provide a link to meeting minutes.

High-level takeaway is that there is general consensus between Safari, Firefox and Chrome that:

1. device-pixel-border-box should not take pinch-zoom into account, because that would allow easy mistakes by web developers. (Envision huge canvases being allocated.)
2. device-pixel-border-box should take page zoom into account.
3. device-pixel-border-box should ignore transforms applied to the element. It can't handle rotations, scales, etc. and shouldn't try to. Developers should know what they're doing when using this box to render 1:1 to device pixels.

(Please correct me if I've misrepresented the conclusions of this discussion.)

Is this feedback sufficient to help move this proposal forward? @atotic pointed out during the meeting that ResizeObserver v0 is implemented in all of Safari, Firefox and Chrome now, and we would collectively love to try prototypes of device-pixel-border-box in all of these browsers.

[smfr]

WebKit's primary objection to this API is that in WebKit, device-pixel snapping is a paint-time operation and the snapped rectangle width is a function of the rectangle origin. So to provide the correct information to ResizeObserver (which fires before paint), we'd have to do a fake paint just to get the right device-pixel size, which we don't want to have to do.

[kdashg]

It should be sufficient for ResizeObserver to be eventually consistent, so a frame of latency would be acceptable, though it sounds like the spec doesn't presently allow this behavior.

FWIW, I'm playing with a pixel-pre-snapping resize: canvas.style.width = canvas.width / window.devicePixelRatio, which appears to work well on at least Windows. It does, of course, depend on an assumed pixel snapping behavior, which may be insufficient to work on all platforms. (But theoretically, the behavior seems sound for intuitive pixel snapping)

device-pixel-border-box does satisfy this use-case, but it seems incredibly specific. Perhaps it needs to be because of layout rules I'm not familiar with, but @dholbert indicated that this should work in at least some cases. I'm following up with our Painting/Compositing people.

[kdashg]

A related discussion on the WebGL repo: https://github.com/KhronosGroup/WebGL/issues/587

[chrishtr]

It should be sufficient for ResizeObserver to be eventually consistent, so a frame of latency would be acceptable, though it sounds like the spec doesn't presently allow this behavior.

Re "a frame of latency would be acceptable": are you saying you think that it's ok for the canvas to draw wrong for one frame and then fix itself? If so, I don't agree. Fixing this is the point of ResizeObserver after all..

FWIW, I'm playing with a pixel-pre-snapping resize: canvas.style.width = canvas.width / window.devicePixelRatio, which appears to work well on at least Windows. It does, of course, depend on an assumed pixel snapping behavior, which may be insufficient to work on all platforms. (But theoretically, the behavior seems sound for intuitive pixel snapping)

This does not work, because pixel snapping on browsers takes into account position as well as device pixel size. This is necessary in order to have consistent and high-quality rendering when a canvas is contained within layout boxes, and between layout phases of a box. Subpixel layout and pixel snapping are necessary and inherent parts of high-quality DOM rendering.

device-pixel-border-box does satisfy this use-case, but it seems incredibly specific. Perhaps it needs to be because of layout rules I'm not familiar with, but @dholbert indicated that this should work in at least some cases. I'm following up with our Painting/Compositing people.

I don't think this is incredibly specific at all. The reason canvas is special is that it's an immediate-model API that draws directly to a backing, as opposed to CSS+HTML rendering, which is mediated by the browser. Therefore the canvas needs to know exactly how big the backing is.

However, the canvas is not rendered in isolation, but rather is an immediate-mode rendering contained within a browser-mediated DOM that has subpixel rendering, pixel snapping, and render timing that is UA-controlled. For this reason, the developer needs a way for the canvas to participate in the rendering lifecycle, via an appropriate callback once the size of the canvas backing is known. The correct timing of this is once layout is complete and before/during paint.

This is very similar to the basic ResizeObserver use-case, except that the canvas needs to also observe the pixel snapping inputs to paint. Therefore, as Simon mentioned, "part of" paint needs to run, in order to compute paint offset, which is an input to the pixel snapping algorithm. I agree that this is more work, but it's necessary for high-quality rendering, and is not that hard to implement.

[Kangz]

For reference here are @kenrussell's notes of the discussions in the WebGPU CG.

[kdashg]

If part of paint needs to run, it's a tough pill to swallow to duplicate that work, which is why a frame of latency allowance helps.

Canvas cannot possibly stay 1-1 throughout a CSS Animation, so we know we're already working on an imperfect solution.

Ideally, a pre-snapped rect will be exactly N pixels tall regardless of offset, unless we round the edge y0 and y1, instead of y0 and height. Anything else will require spurious resizes, which not only do we not want in webgl, but we don't want in general web content painting. Is my intuition leading me astray here?

[chrishtr]

If part of paint needs to run, it's a tough pill to swallow to duplicate that work, which is why a frame of latency allowance helps.

Chromium's implementation does not duplicate any work. There is a pre-paint rendering lifecycle phase that is after layout and before paint, and among other things computes paint offset.

Canvas cannot possibly stay 1-1 throughout a CSS Animation, so we know we're already working on an imperfect solution.

A CSS animation for anything other than transform or opacity induces layout as necessary, which runs the ResizeObserver on every frame. Transforms, as mentioned by Ken above, are not taken into account for the device-pixel-border-box, and so is not a problem. Transform animations without layout/ResizeObserver is one motivating reason for this rule, in addition to the ones Ken mentioned.

Ideally, a pre-snapped rect will be exactly N pixels tall regardless of offset, unless we round the edge y0 and y1, instead of y0 and height. Anything else will require spurious resizes, which not only do we not want in webgl, but we don't want in general web content painting. Is my intuition leading me astray here?

Suppose you have a canvas inside of a div of exactly the same CSS pixel size. The border of the div should match up exactly to the edge of the canvas. For this reason, the div and canvas have to have the same rounding algorithm.

As for why the rounding needs to depend on position: consider two sibling divs that have fractional widths and paint offsets and need to touch each other with no gap. In this case the rounding needs to be consistent, and the way to achieve that is to include subpixel accumulation (which in turn depends on fractional paint offset) in the rounding, so that they round the same direction. This is why the rounding depends on paint offset.

[kdashg]

Can we pre-snap the position and the size? Alternatively, what about getDeviceRects?

Relying on observers for this seems likely to cause unexpected observer callback ordering issues.

[chrishtr]

Can we pre-snap the position and the size?

How would this work?

Alternatively, what about getDeviceRects?

Do you mean a new method exposed to script that returns these rects? The problem is that the canvas doesn't know when to ask, since it doesn't known when it might change due to layout invalidations.

[kenrussell]

@jdashg can you reply to @chrishtr 's questions?

Being able to observe this device-pixel-border-box size would categorically solve a longstanding rendering quality problem on the web platform. Since ResizeObserver is an asynchronous API, it solves problems where providing these attributes on the canvas element forces a synchronous layout.

In Safari, would it eventually be possible to implement this?

Is there any way we can reach consensus that this is a needed feature, and that ResizeObserver is a good place to expose it to applications?

[kdashg]

Sure! As for presnap of position and size, I updated my testcase to pre-snap the top/bottom/left/right coords, and with the exception of what seems like a defect in odd-device-pixel handling on Chrome+Mac (as mentioned in KhronosGroup/WebGL#587), this approach works (as far as I can tell!) for ensuring on-demand 1-1 webgl backbuffer resizing on all desktop browsers today.

Behavior on Android still seems anomalous, but I have less interest in debugging that at the moment.

However, apps with unpredictable relayouts, who additionally don't redraw every frame, could rely on ResizeObserver for events instead of polling every rAF. It just doesn't seem to be a hard requirement for 1-1 webgl rendering for most apps.

I have updated WebGL's wiki page on HighDPI best-practices to include this new approach based on existing primitives.

[juj]

"a frame of latency would be acceptable": are you saying you think that it's ok for the canvas to draw wrong for one frame and then fix itself?

If part of paint needs to run, it's a tough pill to swallow to duplicate that work, which is why a frame of latency allowance helps.

Alternatively, what about getDeviceRects?

Only being able to obtain the pixel perfect guaranteed size of a canvas asynchronously via an event does not strike as a good enough solution. That will lead to all sorts of crazy patterns where one cannot properly specify the size of a canvas in one block of code, but must do a two stage algorithm to change the size.

Further, the upcoming OffscreenCanvas specification may make such types of two-passers even more difficult, since a Worker cannot know the CSS size of a canvas, so a GL app running in a Worker will already need to postMessage to get the CSS size; ultimately this may become a jungle of async CSS size and device pixel size events ping ponging to get something as innocuous as setting up a canvas size right. Async events tend to lead to needing to carefully audit race conditions with other program flow, especially if user interactions are in the loop, which they often are (enter/exit fullscreen, resizing browser window).

I'd very much be in favor of looking for a canvas.getDeviceRects() style of synchronous API that one could use and be done with it. If I understood the conversation above correctly, I find it odd that "browser may need to relayout" was stated as a reason to not be able to implement such synchronous API - after all the .getBoundingClientRect() API already exists and that is not asynchronous either? (Iiuc calling .getBoundingClientRect() will require resolving a relayout if one is pending?)

[chrishtr]

Only being able to obtain the pixel perfect guaranteed size of a canvas asynchronously via an event does not strike as a good enough solution. That will lead to all sorts of crazy patterns where one cannot properly specify the size of a canvas in one block of code, but must do a two stage algorithm to change the size.

Further, the upcoming OffscreenCanvas specification

It's not upcoming, this is already a supported feature in Chrome. :)

That being said, since an OffscreenCanvas in a worker is in a different thread, it is not possible to resize it perfectly in concert with the rest of the page, when resized. What should be done is that a resize observer for it should postMessage the resulting device pixel border box to the worker, causing a subsequent resize + redraw of the canvas texture.

If I understood the conversation above correctly, I find it odd that "browser may need to relayout" was stated as a reason to not be able to implement such synchronous API - after all the .getBoundingClientRect() API already exists and that is not asynchronous either? (Iiuc calling .getBoundingClientRect() will require resolving a relayout if one is pending?)

The problem with such an API is that: (a) there is no time the developer can call it and be sure that it's right. getDeviceRects() would clean layout, but it would clean a layout that is not actually what is drawn on screen, and (b) there is no method other than polling to know when to check that it might have changed.

An example of (a) being hard is that the inputs to layout may change after the call to getDeviceRects(), but before javascript yields to rendering:

canvas.getDeviceRects(); ... divContainingCanvas.style.width = '200px'; .. yield to rendering

In a large, multi-widget application such patterns are common.

[kdashg]

OffscreenCanvas is "upcoming" until multiple implementations support it. Until then, it's premature to call it a finished spec, regardless of spec document status. (A web that only works in Chrome is not the web)

Likewise, the effort here is to cooperate on a direction amongst the implementations.

I disagree with your characterization of (a), as the same issue can happen with the resize observer. (b) is accurate, absent a window.devicePixelRatio observer.

However, the ergonomics of an explicit poll are much better for most apps, in particular much easier to integrate into existing codebases. If an app relayouts after getting a rect, that's on them, just as it's on them when they relayout after canvas resize today. "Move your resize into a ResizeObserver event" sounds good but in practice tends to be a pain. As echoed by @juj, non-trivial apps have a preference for a simpler more explicit API here.

As such, explicit poll is my preference, even if we also add an event observer. Requiring use of the event observer seems worse. (Additionally, the explicit poll is easier to implement, and thus easier to ship quickly)

[kenrussell]

@juj @jdashg a polling API introduces all sorts of problems like forcing a layout, as @chrishtr has shown, which can easily cause major performance and correctness issues in applications. @grorg also raised this issue early on while the WebGL working group was considering adding new properties to the canvas element providing the size in device pixels.

ResizeObserver has solved this problem elegantly - the ability to observe resizes of individual HTML elements, which has been a longstanding missing feature on the web.

Given the problems with adding synchronous polling APIs for this functionality, can you support adding this functionality to ResizeObserver? I don't think there's any application which resizes its elements on a continuous basis - it would be too jarring for the user. The scenario that needs to be addressed is getting accurate, steady-state measurement of the canvas's size in device pixels, and adding this device pixel box size to ResizeObserver will do it. I think that in both existing JS code bases as well as WebAssembly compiled applications it should be feasible to integrate the ResizeObserver-based solution. For the entire time OpenGL has been available as an API, GLUT's callback-based glutReshapeFunc has been the way to respond to window resizing in OpenGL applications.

If we can simply reach agreement that this small addition is an acceptable step forward, then we can collectively make a lot of progress with client applications. Thanks for considering this.

[chrishtr]

OffscreenCanvas is "upcoming" until multiple implementations support it. Until then, it's premature to call it a finished spec, regardless of spec document status. (A web that only works in Chrome is not the web)

Ok, sure.

Likewise, the effort here is to cooperate on a direction amongst the implementations.

Yes, of course. Agreed.

I disagree with your characterization of (a), as the same issue can happen with the resize observer.

Can it? Is there an example in this thread that I missed? (If so, apologies.)

I agree that transform animations can't do it, but that is why I think the rect needs to not include transform.

[juj]

An example of (a) being hard is that the inputs to layout may change after the call to getDeviceRects(), but before javascript yields to rendering:

canvas.getDeviceRects(); ... divContainingCanvas.style.width = '200px';

In this example, the problem is not that the .getDeviceRects() API exists, but the second line: developer changes the size after querying the device pixel size. (what would developer expect to happen anyways?) With same rationale(?), one could claim that having e.g. a synchronous String.length API is bad since developer can change the string afterwards, leading to String.length to return the wrong value. The only way to fix this would be to move to purely functional programming.

Having a resize event observer is fine and great, being able to get events from when a size changes is much nicer than polling, I don't argue against that, but I don't see why that would prevent having a .getDeviceRects() API?

@juj @jdashg a polling API introduces all sorts of problems like forcing a layout, as @chrishtr has shown, which can easily cause major performance and correctness issues in applications.

I see forcing a layout being the correct thing to do, and not a problem. In 99% of applications with this canvas use case by the time the WebGL context is being initialized the DOM has already been set up and the DOM content is static. Relayouting would not do anything in such case, and even if it has to relayout right there on the spot, I don't see it correctness issues. If the API is forced to be async, it will be the async nature that will cause performance and correctness issues. If I have code e.g. like

var canvas = new Canvas();
canvas.style.width = '100%';
canvas.style.height = '100%';
document.body.appendChild(canvas);
var deviceSize = canvas.getDeviceRects();
canvas.width = deviceSize.width;
canvas.height = deviceSize.height;
var ctx = canvas.getContext('webgl');
requestAnimationFrame(...);

there does not really exist a more performant, correct, unambiguous nor self-documenting way to initialize the rendering backbuffer size and the GL context than the above code. It has a good guarantee that no badly sized temp GPU backbuffer resources will be allocated. Having a separate resize observer event to allow reacting to when user resizes the page or similar is then great as well (is the existing 'resize' event not already usable for that purpose?)

If we need to have an async event to get the proper size, we will end up with tons of code out there that first initialize the canvas with a framebuffer with size 1920x1079 or 1921x1080 or something random off-by-one like that, then render one or more(?) frames with wrong size, then get the event, then fix up the framebuffer to proper 1920x1080 size. That will have worse performance than the code example above. Or we place an implicit requirement to codebases that initializing a GL context on a canvas is practically an async operation if you don't want temp memory pressure or silly temp init work, which will lead to more complicated init sequences and correctness bugs from developers.

Also e.g. transitioning to fullscreen would become an asynchronous operation, where one would .requestFullscreen() (as response to user input) either have to pause rendering until the resize event is received, or render with incorrect backbuffer size for unspecified number of frames, leading to possibly glitched visuals.

I don't think allocating these kind of temp framebuffers on the GPU would be exactly free memory-wise either, and GPU memory intensive applications might run into GPU OOMs because they allocated temporary off-by-one framebuffers on the GPU.

[chrishtr]

In this example, the problem is not that the .getDeviceRects() API exists, but the second line: developer changes the size after querying the device pixel size. (what would developer expect to happen anyways?) With same rationale(?), one could claim that having e.g. a synchronous String.length API is bad since developer can change the string afterwards, leading to String.length to return the wrong value. The only way to fix this would be to move to purely functional programming.

I think the String analogy you gave is not all that useful in this case. The difference is that the DOM is laid out according to a global constraint algorithm that has many non-local effects that developers cannot feasibly predict in all cases. Even for the case of changes to the constraints that are introduced by the developer, in a large and dynamic web application it's very difficult to predict every single case that a size or offset of an element has changed. This is extra true because pixel snapping is (intentionally) not spelled out in the web specs.

In addition, there are cases where layout occurs even without the developer doing anything. Examples include the user resizing the page or changing zoom factors (you mentioned the former in your earlier comment, I know). Another example is a canvas embedded in a cross-origin iframe that has no idea what its containing frame might do, because that frame is controlled by a third party.

Having a resize event observer is fine and great, being able to get events from when a size changes is much nicer than polling, I don't argue against that, but I don't see why that would prevent having a .getDeviceRects() API?

It's because the return value of getDeviceRects is incorrect and misleading. There is also the problem of deciding whether to return a stale value from the prior frame, or an "up to date" (but not really) value that forces layout. The former doesn't work in cases where the canvas has just been resized since the last frame, and the latter doesn't work if something will happen after the call to getDeviceRects and before the next render to the screen. In addition, the forced layout caused by the latter approach is bad in and of itself, because forced layouts interfere with efficient pipelining of the rendering system (multi-threading, for example). And finally, polling is required on top of getDeviceRects, which is fundamentally worse than an observer which is called only when things actually change.

I see forcing a layout being the correct thing to do, and not a problem. In 99% of applications with this canvas use case by the time the WebGL context is being initialized the DOM has already been set up and the DOM content is static.

As I mentioned above, I don't think this is accurate. Most web pages are quite dynamic these days, and even for cases where they are not, the user can cause layout changes.

there does not really exist a more performant, correct, unambiguous nor self-documenting way to initialize the rendering backbuffer size and the GL context than the above code.

Code like the above is what sites already do, except for the getDeviceRects part. Instead they take the CSS sizing and multiply by devicePixelRatio. For sites that want to always be exactly aligned to device pixels no matter what, they can use a ResizeObserver to adjust when needed.

It has a good guarantee that no badly sized temp GPU backbuffer resources will be allocated. Having a separate resize observer event to allow reacting to when user resizes the page or similar is then great as well (is the existing 'resize' event not already usable for that purpose?)

I agree that avoiding re-allocating GPU buffers is a good thing. Developers can reduce this by allocating the canvas buffer in a ResizeObserver at start, and only re-allocating it in a ResizeObserver subsequently.

ResizeObsever is for sure not a fully "direct" API, but it's the consequence of embedding an immediate-mode API like Canvas into the retained mode + inversion-of-control system that is HTML+CSS. Developers must do a certain amount (and really not all that much in this case) of work to adapt to this paradigm vs a fully immediate-mode they may be used to from prior experience.

[juj]

Talked with Ken a bit more about ResizeObserver, and it sounds like it can accommodate the use case that we have at Unity.

One question I have is regarding how proper WebGL context creation now looks like? I.e.

Developers must do a certain amount (and really not all that much in this case) of work to adapt to this paradigm vs a fully immediate-mode they may be used to from prior experience.

so is the new proper paradigm to create a WebGL context

<html>
<body>
<canvas id='canvas' style='width: 100%; height: 100%'> </canvas>
<script>
var canvas = document.getElementById('canvas');

new ResizeObserver(entries => {
  var size = entries[0].devicePixelBorderBoxSize;
  canvas.width = size.width;
  canvas.height = size.height;
  var ctx = canvas.getContext('webgl');
  // init game engine/game/app logic here:
  initRendering(ctx);
}).observe(canvas, { box: 'device-pixel-border-box' });
</script>
</body>
</html>

does the above work?

Another question was how about if the JS code needs to be loaded dynamically? E.g. applying the above to the current structure of Emscripten compiled WebAssembly applications would look like:

<html>
<body>
<canvas id='canvas' style='width: 100%; height: 100%'> </canvas>
<script>

function downloadScript(url) {
    return new Promise((ok, err) => {
      var s = document.createElement('script');
      s.src = url;
      s.onload = () => { ok(); };
      document.body.appendChild(s);
  });
}
downloadScript('page.js');

page.js:

var canvas = document.getElementById('canvas');

new ResizeObserver(entries => {
  var size = entries[0].devicePixelBorderBoxSize;
  canvas.width = size.width;
  canvas.height = size.height;
  var ctx = canvas.getContext('webgl');
  // init game engine/game/app logic here:
  initRendering(ctx);
}).observe(canvas, { box: 'device-pixel-border-box' });
</script>
</body>
</html>

Does that work? I.e. will observing the resize cause an observe event to be fired on the next frame?

Or does the user need to dynamically create the canvas element with

var canvas = new Canvas();
canvas.style = 'width: 100%; height: 100%;';
new ResizeObserver(......).observe(canvas);
document.body.appendChild(canvas);

in order for the resize event to be observable for the first time?

[kenrussell]

<html>
<body>
<canvas id='canvas' style='width: 100%; height: 100%'> </canvas>
<script>
var canvas = document.getElementById('canvas');

new ResizeObserver(entries => {
  var size = entries[0].devicePixelBorderBoxSize;
  canvas.width = size.width;
  canvas.height = size.height;
  var ctx = canvas.getContext('webgl');
  // init game engine/game/app logic here:
  initRendering(ctx);
}).observe(canvas, { box: 'device-pixel-border-box' });
</script>
</body>
</html>

does the above work?

The game initialization logic should be factored out from the ResizeObserver, or at least, check to see if it's already been run before running it again. Ideally only resizing of the back buffer would be handled in the ResizeObserver's callback.

If this approach is documented, then in the same place it also needs to be documented that apps might want to run at lower resolution and could observe a different box via ResizeObserver for that purpose.

Another question was how about if the JS code needs to be loaded dynamically?

...

Does that work? I.e. will observing the resize cause an observe event to be fired on the next frame?

That's a good question. Per these docs: https://drafts.csswg.org/resize-observer-1/ https://developers.google.com/web/updates/2016/10/resizeobserver

it looks like yes, if observation starts, the element is being rendered, and its size is greater than (0,0), the observer will fire. So the canvas can be placed in the page rather than needing to be dynamically created.

[kdashg]

It seems to me that ResizeObserver is more or less polyfillable given some reliable getBoundingDeviceRect(). (which I've previously largely implemented on top of getBoundingClientRect and devicePixelRatio)

In particular, there doesn't seem to be a huge difference logically between using ResizeObserver and calling getBoundingClientRect at the top of RAF, given that to be useful the same frame, ResizeObserver must fire /before/ RAF.

Is there a logical difference I'm not seeing?

[chrishtr]

In particular, there doesn't seem to be a huge difference logically between using ResizeObserver and calling getBoundingClientRect at the top of RAF, given that to be useful the same frame, ResizeObserver must fire /before/ RAF.

ResizeObserver callbacks are not called before rAF callbacks. They are called after all rAFs are done, after layout, and before drawing to the screen. If the ResizeObserver callback mutates DOM state, then layout is re-computed, after which ResizeObserver callbacks are called again. This may happen multiple times (though the number of times is bounded by the DOM depth).

Because ResizeObserver is called after layout and before drawing, it has the opportunity to do things like resizing canvases, without any lost frames, no matter what the layout change was that resulted in the need for canvas resize.

[kdashg]

If ResizeObserver is called after RAF, then there's a frame of latency in the WebGL case we've discussed. Is this intentional?

[chrishtr]

I’ve added this issue to the agenda of the next CSSWG call and also to TPAC.

[chrishtr]

If ResizeObserver is called after RAF, then there's a frame of latency in the WebGL case we've discussed. Is this intentional?

There is not an extra frame of latency. It’s true that if a WebGL app updates its canvas rendering state in a rAF callback, and then a ResizeObserver for in the same frame causes the canvas to resize, the WebGL app may need to update its state right then and there, but it can do it. And it will be displayed to the screen in the same frame as the rAF callback.

[kdashg]

There's simply not time to re-render a webgl frame in response to a late resize observer event.

[kdashg]

For the purposes of a preponderance of WebGL content, if the resize event comes after RAF if it comes at all, that's a frame of latency.

As such I don't see a robust approach here for WebGL's needs that isn't getBoundingDeviceRects.

[kdashg]

I can join the next CSSWG call.

[chrishtr]

There's simply not time to re-render a webgl frame in response to a late resize observer event.

For sure there is less time before the next vsync.

For the purposes of a preponderance of WebGL content, if the resize event comes after RAF if it comes at all, that's a frame of latency.

Yes, the frame is delayed, and if it misses vsync there is an extra frame’s delay. I thought you were talking about wrong frames, not delays to a frame.

With a ResizeObserver approach, on the frames where the canvas happens to resize (which are rare), then two WebGL rendering updates will occur and the frame will take longer to draw to the screen. However, all other frames are unaffected. And on top of that, there will never be a wrong frame (defined as a frame where the WebGL drawing does not properly match the sizing of the canvas element.

With a getBoundingClientRect * devicePixelRatio approach, or even if getBoundingDeviceRects were added, in simple situations you can indeed call getBoundingDeviceRects during a rAF callback and resize & re-draw the canvas. However, wrong frames are impossible to avoid in general, because of situations like:

• Another rAF callback unrelated to the WebGL one may run after it in the same frame, and may change DOM in a way that changes the layout size of the canvas.
• A ResizeObserver on another element could also cause layout to change, with the same result.

I do not think these situations are uncommon in complex apps. When encountered, the only way to avoid them is try to coordinate all activity on the page, leading to lots of complexity for the developer. Or in many cases, they will just accept a worse web app and the web is worse for it. ResizeObserver exists to solve this exact problem, in a decentralized way, that is pretty easy to use.

Further, if the canvas is quiescent and not updating its drawing, there is no solution to detecting when it resizes, except polling or adding a new resizing event callback, which suffer from the problems above as well. ResizeObserver also solves this problem with no additional work for the developer.

I can join the next CSSWG call.

Great, thank you.

[css-meeting-bot]

The CSS Working Group just discussed device-pixel-border-box size.

The full IRC log of that discussion

<dael> Topic: device-pixel-border-box size
<dael> github: https://github.com/w3c/csswg-drafts/issues/3554#issuecomment-513045449
<dael> chrishtr: I'm the one that added it to the agenda
<dael> chrishtr: Main thing is resolving that this is a good approach to resolving responcive design for canvases.
<dael> chrishtr: Exact details of API can be followed up if agree on first step
<dael> jeff: The main thing on figuring out how this works is when we're thinking about scheduling we get a request at beginning of frameso can render webgl
<dael> jeff: Then main thing is take stock of width and height and that's usually when resize happens. I understand resize is only latter after raff when browser thinks we can take current layout
<dael> jeff: Then resize observer kicks off events and says to canvas you're resized. Issues for WebGL is that time is non-trivial. Late in the frame is hard to respond. You can realize in time for end of frame you rendered wrong and can do minor fix but can't render at new resize if observer after raff
<dael> chrishtr: Design of resize observer is to support this responsive design wehre decentralized action causes layout to change. if resize observer callback thinks sized changed non-canvas can change internal layout. If there's change it causes another layout. non-canvas the frame is twice as long and I think that's unavoidable.
<dael> chrishtr: Key is resizes are uncommon and it's only when user changes browser or orientation. COuld be doing layout animation but one that changes layout in every frame is perf problem
<dael> chrishtr: For webGL canvas it is an unavoidable slowdown
<dael> gregwhitworth: Is your issue that you may potentially miss a frame due to ascyn nature?
<dael> jeff: yeah
<dael> gregwhitworth: In current state you're sync. Feels you prefer have a performance thing where effecting frame vs the inverse
<dael> jeff: Not clear. If you rely on resize to tell you size and can't correct ahead of time it's hard to from the get go render frame with proper size
<dael> dbaron: Almost think for this use case resize-observer feels like wrong thing due to structure. Request animation frame was designed as a callback where can write layout information.
<dael> dbaron: One of the problems here is that raff, you get a raff [echo issues]
<astearns> s/raff/RAF/
<dael> dbaron: RAF is early in cycle and you can do it before layout. Layout as result of resize is after raf. Some things people do will flush layoutanyway and may layout twice. What you want here is to know canvas size and then do WebGL stuff
<dael> dbaron: If you want to know size of canvas can do resize-observer but it's designed to not flush layout and you might not need changes. In your use case you want to know this is the new size of box after alyout this cycle. Way you do that is use a sync method that flushes layout. Assuming 1 canvas
<dael> dbaron: What works for this use case is late in the RAF you get width and height of canvas, do WebGL work, don't use resize-observer. You did minium layout and get new canvas size
<dael> jeff: THat's what I've been centering on the solution for the common case. Where it gets muddy is part of the goal is device-pixel-border-box size. This is predominantely useful for canvas and WebGL.
<dael> jeff: A bing chunk of WebGL cases will have this workflow and not want resize-observer to get late resizes. Want the early layout to get box sizes.
<dael> jeff: What we'd be lacking is a way to sync query the device-pixel-border-box
<dael> jeff: Right now can get bounding client rects but it's not device pixel. That's what we want for WebGL.
<dael> jeff: Sounds like what we want is a a getBoundingDeviceRects or something sync to get pixel size. THen resize-observer is helpful for the other use cases. WebGL where they flush layout uses this othercommand
<bradk> I have to leave early. Bye!
<dael> chrishtr: Two things wrong about desc dbaron . Resize-observer is not async. Delivered immediately and cause another layout on same frame. Second is you can't actually know size of WebGL canvas even if you calls a sync method because there can be a different callback regisered by someone else that dirties layout.
<dael> chrishtr: Cna't do it the way you desc
<dael> dbaron: Those are fixable.
<dael> dbaron: Problem with resizeobserver is jeff's use case is a notification every time, not jsut resize.
<dael> dbaron: There's tricks to put yourself after next RAF. There's always problem of multi-observers.
<emilio> q+
<dael> dbaron: I think fundamentlly people want to do different things. THis prop makes a set of data only available in resize-observer but not in other ways and I don't think we want to put to use only resize-observer and other solutions
<dael> chrishtr: Good to only provide right information in right spot
<dael> dbaron: THink it'swrong spot
<dael> ken: resize-observer solves a lot of problems. Fantastic all browsers are impl v0 so that we can encourage everyone to use that. We'd like to recommend just use device-pixel-border-box if they want it exactly correct. sync apis have a host of problems like going to while loops to make layout settle down. resize-observer callback solves problems
<dael> jeff: It does, but it has remaining problems. If WebGL content wants to render...there's no way to deterministically always try and render correct frame size and not occationally double-paint
<dael> ken: I see that. THe live resizeing case is vanishingly small. You're moving a handle on the webcase. In the steady state you'll render nromally. If you end up double painting during live resize not end of world
<dael> Rossen_: A little under statement. Let's not diminish value of resizing and only put it ina corner. I can see multi frame being in same overall app layout each with own observer and then the drawback will be magnified. Let's not diminish effect of this
<bkardell_> if it was really only based on the size of the window, this feature wouldn't be necessary
<astearns> ack emilio
<dael> emilio: DId we figure out...what this device wants to add is a snapped rect. We figured for some combo of transforms that can create non-rect we also need to do transform calcs to get right device snap rect
<Rossen_> q?
<dael> emilio: It stashes a bunch of paint information onto an API. Want a reminder of what the web reprecussions of that discussion were
<smfr> q+
<dael> gregwhitworth: Talked about that asSanFran F2F. Most people thought okay beca'sescope was to canvas. Use case for canvas we won't have transforms and those types of scales so we don't need to go that far down stack. I think chrishtr said they are doing that. You are stashing, but primary case is you want pixel snapping because we're WebGL and want draw to pixel snapping and want ot know about resize to get perfect frame
<dael> gregwhitworth: Not opposed to dbaron where we can't do this anywhere else. But in this specific case where the resize gets a less clean seam and it's only on resize
<dael> gregwhitworth: To your point you have to shove painting knowledge into it. And you don't have all information.
<dael> chrishtr: I think transforms shouldn't apply at all
<dael> chrishtr: Exact def. or impl of pixel thing is different then other discussion. even if want to do something to instruct dev to always resize at beginning of frame you can't get bounding rect and multiple by pixel. Needs to be fixed or canvas can't render
<dael> jeff: Can do work to estimate eventual pixel boundries. I'm told people who know pixel snapping rules find it sketchy because rules aren't defined and we'd have to guess. Need a library that sniffs for UAs and gets right rules.
<gregwhitworth> they're not defined and vary by UA
<astearns> ack smfr
<dael> chrishtr: That's something where it's not going to be a great solution. Dev will hack around and that's not good
<dael> smfr: Remind people not all AU have pixel snapping before paint. We don't know before we go to paint. Even if added the device pixel we'd have to do a fake paint. THat's non-trivial for WK. And makes forced layout be even more expensive and they're already a source of perf issues. Hesitant to add an API that makes fake paints
<dael> jeff: Maybe alternative for WebGL usecase is if there were a callback following RAF like resize-observer but happens as early in the frame as possible and deterministicly every frame might be more workable if people object to a sync call for pixel layout
<dael> jeff: Solves double paints, gives webGL a way to know this is final size as long as nothing else changes it. As long a snothing else changes it is a problem we can't ever get around. might be the way forward there.
<smfr> https://html.spec.whatwg.org/multipage/webappapis.html#update-the-rendering
<dael> smfr: HTML 5 has rendering steps. Where in those would callback fire?
<dael> jeff: I don't know
<dael> dbaron: Not sure I believe rendering steps in HTML5 make sense. I have an open issue to fix some of it.
<dael> dbaron: I support basic idea from Jeff. I think there's a similar Google proposal. Requires people have separate callbacks to separate writing layout and reading layout. In a refresh you want all writing before all reading
<dael> smfr: GOod idea except APIs we have make it easy to do wrong things. That only works is everyone does reading and writing at same time. Need a whole new set ofDOM APIs that read without forcing layout
<dael> dbaron: resize-observer feels designed for this, but it's a very particular solution
<dael> chrishtr: It's providing a responsive decentralized layout at one time.
<dael> chrishtr: What you desc about post layout callback if you have arbitrary code running it dirties layout
<dael> jeff: Not solvable
<dael> chrishtr: Resize-observer solves it. If you dirty layout should you layout again? If yes it's same as resize-observer.
<dael> jeff: Keep in mind my objections to have resize-observer isn't determatively fired. If we're trying to solve WebGL problem here, the solution shouldn't be half baked. There are problems with as exists resize-observer.
<dael> jeff: Main problem is we can't hit our frametimes if we get resize-observer late. Solutions are sync call or get device-pixel size and other is a resize call that always happens early in theframe. THat solves WebGL problem.
<dael> jeff: Getting confident WebGL problem isn't solved by current PR
<drousso> gtg :(
<dael> ken: USing resize-observer is simplier to explain. It gets 80-90% to solution. I realize poss of duplicate renders but that resize-observer takes into account relayouts that may happen is I think good design. I would advocate adding device-pixel-border-box to resize observer API as a pretty good solution. We can iterate on it in the future to maybe get alyout earlier and less change of dup. frames
<dael> jeff: I have an 80% solution in hand already
<dael> ken: But it doesn'timpl the real pixel snapping the browsers do. Need access at app level
<dael> jeff: But my JS hack sounds like a better solution then the API
<dael> chrishtr: I don't htink that's true. resize-observer 100% solves get size of canavs
<dael> jeff: Do you think my rec. to have deterministic frame that says we finished layout is unworkable.
<dael> gregwhitworth: That's what it does.
<dael> jefBut we can't hit frame times because takes 12ms
<bkardell_> you always get 1 yes
<dael> dbaron: jeff wants a notification if it does or doesn't change size. For resize-observer when you register a new one do you always get one notifiation?
<dael> ??: Yeah
<dael> dbaron: What happens if you re-register resize observer every frame? Does that solve?
<dael> jeff: Depends how early it fires. Is it designed to fire as soon as possible?
<dael> gregwhitworth: Following layout for as accurate as poss. TO get pixel snapping need to get into paint. I don't understand as early as possible, has to be late to understand layout
<dael> dbaron: Wants it before he does WebGL painting to canvas
<tantek> sounds like maybe you want to both register a RO and a RAF callback, RO for the first notification, and RAF for no-resize notifications, and RO when there is an actual resize
<dael> gregwhitworth: Whichi s fine if not respond to resize. THat's what RAF does now
<dael> dbaron: What Jeff wants si to do layout, know what size results, and then paint to canvas
<dael> gregwhitworth: It's a callback to say hey we changed
<dael> dbaron: He want sto paint when no resize too
<tantek> painting when no resize is what RAF is for right?
<dael> chrishtr: yOu're desc a post-layout animation callback which doesn't exist. One objection is it gets in theway of running layout off main thread. resize prob has same problem
<gregwhitworth> tantek: that's my confusion
<gregwhitworth> if you don't want resize calls you add it to RAF
<gregwhitworth> and what chrishtr just said if you want both
<dael> chrishtr: Can get same thing except potentially slower during resize with resize-observer but need request animation callback as well.
<tantek> hah literally what I just said in IRC
<gregwhitworth> chrishtr: added it to TPAC
<dael> astearns: Nearly at time. I don't think we will resolve today, but good to have back and forth
<tantek> jeff, does that solve your use-case? using *both* RO and RAF?
<gregwhitworth> sorry, meant that to be a statement - not to you
<dael> astearns: Interesting to have fuller proposal for the post layout callback. I htink should continue in GH and come back.
<dael> many people: thanks

[dbaron]

One note is that I think (although being sure requires executing the entire ResizeObserver spec in my head) calling ResizeObserver.observe() on the canvas element in question on every requestAnimationFrame should lead to a reliable ResizeObserver notification every cycle, whether the canvas has resized or not. That might be a path forward for @jdashg's use case.

[chrishtr]

One note is that I think (although being sure requires executing the entire ResizeObserver spec in my head) calling ResizeObserver.observe() on the canvas element in question on every requestAnimationFrame should lead to a reliable ResizeObserver notification every cycle, whether the canvas has resized or not. That might be a path forward for @jdashg's use case.

Just to repeat what I said in the call also: that would indeed work and be fine, as long as you could observe the device-pixel border box of the canvas. Implementation for the developer would be:

function observerCallback(entry, observer) {
  let devicePixelBorderBoxRect = entry[0].devicePixelBorderBox;
  render(devicePixelBorderBoxRect); // WebGL rendering
  observer.unobserve(myCanvas);
  observer.observe(myCanvas, {box: 'device-pixel-border-box'});
}

var observer = new ResizeObserver(observerCallback);
observer.observe(myCanvas, {box: 'device-pixel-border-box'});

@jdashg would this meet your needs?

[atotic]

@christr: I think you meant

function observerCallback(entry, observer) {
  let devicePixelBorderBoxRect = entry[0].devicePixelBorderBox;
  render(devicePixelBorderBoxRect); // WebGL rendering
}
function rafCallback() {
  observer.unobserve(myCanvas);
  observer.observe(myCanvas, {box: 'device-pixel-border-box'});
}
var observer = new ResizeObserver(observerCallback);
document.requestAnimationFrame(rafCallback);

@dbaron that is very clever!

[chrishtr]

Yes that would work (@chrishtr btw :))

One typo about requestAnimationFrame though, new code below:

function observerCallback(entry, observer) {
  let devicePixelBorderBoxRect = entry[0].devicePixelBorderBox;
  render(devicePixelBorderBoxRect); // WebGL rendering
}
function rafCallback() {
  observer.unobserve(myCanvas);
  observer.observe(myCanvas, {box: 'device-pixel-border-box'});
  requestAnimationFrame(rafCallback);
}
var observer = new ResizeObserver(observerCallback);
requestAnimationFrame(rafCallback);

[juj]

function rafCallback() { observer.unobserve(myCanvas); observer.observe(myCanvas, {box: 'device-pixel-border-box'}); requestAnimationFrame(rafCallback); }

This feels like an abuse of the observer model, and I think also causes running garbage collection pressure from {box: 'device-pixel-border-box'} each frame. (Wasm applications strive to run garbage free to avoid JS performance issues). :/

How do browsers like if rendering would always take place outside the rAF function?

Does the following achieve the same?

function observeCanvasSizeChange(canvas) {
  function observerCallback(entry, observer) {
    let devicePixelBorderBoxRect = entry[0].devicePixelBorderBox;
    canvas.deviceWidth = devicePixelBorderBoxRect.width;
    canvas.deviceHeight = devicePixelBorderBoxRect.height;
  }
  var observer = new ResizeObserver(observerCallback);
  observer.observe(canvas, {box: 'device-pixel-border-box'});
}

function rafCallback() {
  render(myCanvas.deviceWidth, myCanvas.deviceHeight);
  requestAnimationFrame(rafCallback);
}
observeCanvasSizeChange(myCanvas);
requestAnimationFrame(rafCallback);

or does that also have the problem that rendering may occur to wrong(unsynchronized) size e.g. under continuous DOM size animation?

[chrishtr]

This feels like an abuse of the observer model,

Maybe, maybe not. The example script I gave, which fleshed out the idea @dbaron suggested, shows that there is a way to avoid the double-WebGL-render problem @jdashg raised in situations where a resize occurs. And if it turns out to be a very useful mode, we could formalize that in a new API.

Regarding GC: if it turns out to be a real problem for a WASM app of the future, it can definitely be solved pretty easily with an API tweak.

How do browsers like if rendering would always take place outside the rAF function?

There is nothing at all that forces developers to render within a rAF. In fact, many popular frameworks, such as React, don't actually render during rAF, in part to have more time to render than rAF allows, as rAF often tries to fire as late as it can to minimize latency to the screen.

As the Chromium rendering lead, I would say my view is that "do all rendering within rAF" is not a feasible solution for many cases, especially in complex apps, due to the issue of wasting CPU time being idle. (*)

or does that also have the problem that rendering may occur to wrong(unsynchronized) size e.g. under continuous DOM size animation?

It has several problems; one of them is the wrong-size issue you mentioned. There is also the issue of needing to poll (or constantly rAF, which amounts to the same thing) just in case sizes change, and also that extra forced layouts can occur because deviceWidth would force layout, and other rAF callbacks may dirty layout also. These are mentioned in more detail in comments above in this issue.

(*) Aside that may be useful:

To address the framework use-case mentioned above, postAnimationCallback is a new callback being proposed that is intended to be run after rendering is complete, so that reading back layout is more likely to be free, and to support the use cases of starting rendering of the next frame as soon as possible.

However, this callback happens "post-commit", which means that the rendering display list has already been sent to the browser for display to the screen. Doing it post-commit is important to avoid postAnimationCallback ending up being the same as ResizeObserver, and thereby accidentally delaying frames.

[greggman]

Sorry to butt in here but I just wanted to point out the HTML snipped above is probably not the best practice for a full window canvas.

If you want a canvas to fill its container, in this case the <body> then you should set the canvas to display: block. Canvas defaults to display: inline which ends up adding extra space at the end as is the reason people often get a scrollbar and then they end up hacking around their mistake by adding overflow hidden etc...

As well if you're using 100% and you don't set the size of the body and html you won't get 100% height. Example

These are 2 recommended ways to get a full page canvas

1. use vw an vh

<html>
<style>
body { margin: 0; }
#canvas { width: 100vw; height: 100vh; display: block; }
</style>
<body>
  <canvas id="canvas"></canvas>
</body>
</html>

Example

This will give you a fullscreen canvas with no scrollbar on all desktop browsers.

The problem comes in on mobile where you have to decide what you want full page to mean. Both Chrome and Safarai (and Firefox?) decided that to deal with browsers on mobile hiding and showing the address bar and/or other UI then 100vh = the size when the UI is at its smallest and 100% = the size actually displayed (smaller when UI is visible, larger when not)

To get that to work cross browsers you have to do this

<html>
<style>
html, body { height: 100%; }
body { margin: 0; }
#canvas { width: 100%; height: 100%; display: block; }
</style>
<body>
  <canvas id="canvas"></canvas>
</body>
</html>

Example

Which is more correct depends on the use case. if the canvas is some background element over which content scrolls then you probably want 100vh. If the canvas is just supposed to fill the screen then you probably want 100%

[css-meeting-bot]

The CSS Working Group just discussed ResizeObserver Device Pixel Border Box.

The full IRC log of that discussion

<fantasai> Topic: ResizeObserver Device Pixel Border Box
<fantasai> ScribeNick: fantasai
<fantasai> chrishtr: device-pixel-border-box is the actual device pixel bounds of a canvas element
<fantasai> chrishtr: including pixel-snapping feature which is ued by browsers to avoid blurriness by snapping to pixel grid
<fantasai> chrishtr: fundamental problem with no complete solution
<fantasai> chrishtr: authors that use canvas
<fantasai> chrishtr: have no reliable method to determine on-screen pixel size of the canvase
<fantasai> chrishtr: if off by one pixel due to pixel-snapping, rounding, or other issue
<fantasai> chrishtr: will end up with wrong or blurry results
<fantasai> chrishtr: pixel-snapping is intentionally not specced to allow UAs to do their best
<fantasai> chrishtr: acocunt for varying rendering architecture
<fantasai> chrishtr: and evolution
<fantasai> chrishtr: so no way to reliably find this size
<fantasai> chrishtr: proposal is to add a box that observes device pixel border box
<fantasai> chrishtr: to ResizeObserver
<fantasai> chrishtr: which will notify if that box changes
<fantasai> chrishtr: will happen at similar timing as other ResizeObserver
<astearns> github: https://github.com/w3c/csswg-drafts/issues/3554
<fantasai> chrishtr: there were two main objections to adding this
<fantasai> chrishtr: one was raised by Jeff Gilbert
<fantasai> chrishtr: had a long discussion with him and Kem Russel, our WebGL expert (?)
<fantasai> chrishtr: have a compromise proposal that both of us agree to
<heycam> s/Kem Russel/Ken Russell/
<fantasai> chrishtr: objection he raised was that if you have a WebGL-centric application
<fantasai> chrishtr: e.g. full-screen game that uses WebAssembly and only DOM in minimal way
<fantasai> chrishtr: want to have a continuous requestAnimationFrame loop
<fantasai> chrishtr: drawing the canvas
<fantasai> chrishtr: in that model where you are canvas-centric
<fantasai> chrishtr: it still lives in a DOM shell or browser window that has a size
<fantasai> chrishtr: need to observe that size
<fantasai> chrishtr: where pixel snapping will work
<fantasai> chrishtr: but ... not in JS
<heycam> s/.../in Web Assembly/
<fantasai> chrishtr: most convenient to query device border box directly from canvas during rAF loop
<fantasai> chrishtr: rather than by observer
<fantasai> chrishtr: if layout was dirty at the time during making the query, it will force layout and other things in order to compute pixel snapping
<fantasai> chrishtr: in taht use case maybe it's OK
<fantasai> chrishtr: other use case, which I was most focused on
<fantasai> chrishtr: you have canvas element embedded in a DOM-centric website
<fantasai> chrishtr: maybe photo-app or ad or widget,
<fantasai> chrishtr: potential multi-actor scenario
<fantasai> chrishtr: want to avoid layout thrashing
<fantasai> chrishtr: cases for which ResizeObserver was designed
<fantasai> chrishtr: this makes most sense
<fantasai> chrishtr: compromise proposal is that we just have both APIs
<fantasai> chrishtr: The End! :D
<fantasai> jeff: ....
<fantasai> jgilbert: ... moving forward with both propsoals works out
<fantasai> jgilbert: having it be in ResizeObserver also makes sense
<fantasai> jgilbert: like you went over, there was some concerns about it covering espeically more easily some more trivial cases
<fantasai> jgilbert: having both lets people pick and choose appropriate API
<fantasai> jgilbert: default should be to use ResizeObserver, most reliable
<fantasai> jgilbert: kindof like getClientBOundingRect() , need to be careful if calling that often
<fantasai> jgilbert: maybe warn in UA
<fantasai> jgilbert: if called too often
<fantasai> jgilbert: but getDeviceClientRect() API alone ...
<fantasai> chrishtr: 2nd objection smfr raised
<fantasai> chrishtr: pixel-snapping requires more information than just layout in today's engines
<fantasai> chrishtr: in Chrome requires pre-paint step
<fantasai> chrishtr: in Safari requires paint
<fantasai> chrishtr: don't have solution to that problem
<fantasai> chrishtr: just note readback method for rAF
<fantasai> chrishtr: canvas.getThing
<fantasai> chrishtr: also has to run the same steps
<fantasai> Rossen_: but that's not blocking to accept the proposla
<fantasai> smfr: in some ways make sit worse
<fantasai> smfr: because 2 places we have to do this painting calculation
<fantasai> Rossen_: was asking if this second objection is blocking accepting the proposal
<fantasai> smfr: so to clarify the proposal is having API to return the box on canvas and also in ResizeObserver
<fantasai> smfr: I'm not a fan of doing partial paitn to get this info
<fantasai> smfr: I would expect for games , especially in full-screen, to position on ? boundaries
<fantasai> smfr: surprised if using fractional pixel positioning
<fantasai> jgilbert: for full-screen, unlikely, but for partial screen almost always get this
<fantasai> jgilbert: both on Windows and on Android, I believe, Windows will happily give you 1.6574 device pixel ratio
<fantasai> jgilbert: you just have to deal with it
<fantasai> jgilbert: you end up trying to reverse-engineer what pixel-snapping is to figure that out
<fantasai> jgilbert: pre-snap a rect to a non-integre CSS size , if that works out to integer pixels... then no artifacts? but it's a huge mess
<fantasai> smfr: other case that's confusing is on our iPhone's that have retina displays
<fantasai> smfr: already 2.7? scaling factors
<fantasai> smfr: so can cause hairlines to disappear aetc.
<fantasai> smfr: seems like also on windows, too
<fantasai> smfr: getting pixel perfection, seems like OS is doing scaling behind your back, how can you expect it to work?
<fantasai> jgilbert: out of the box cases where if you play around with virtual scaling
<fantasai> jgilbert: on a Mac and play with scaling on a retina screen
<fantasai> jgilbert: no way to get 1-1 device pixel
<fantasai> myles: even worse, the default ratio is not 1-1 or 2-1
<fantasai> jgilbert: it really depends on how the OS is doing this sort of scaling
<fantasai> jgilbert: I'll add that what you end up with, for instance in Mac, when you have this OS-level virtual resolution scaling
<fantasai> jgilbert: can't get one to one
<fantasai> jgilbert: if can't get 1-1 in application, can still get moire effects
<fantasai> jgilbert: can't entirely eliminate scaling artifacts, but can do better
<fantasai> jgilbert: than naively try to grid on the screen and hope it looks good
<fantasai> mattwoodrow: difference is on Windows the scaling isn't hidden
<fantasai> mattwoodrow: can attempt to match
<fantasai> jgilbert: windows 10 implements scaling that llows 1-1 rendering, not virtual scaling as default
<fantasai> jgilbert: Mac doesn't expose effective scaling, just says 2x all the time
<fantasai> jgilbert: Android says 3x all the time
<Rossen_> q?
<fantasai> ??: regarldess of what operating systems do, if application renders pixel -perfect
<fantasai> ??: Result will be better than if it wasn't
<fantasai> ??: I think we agree on that
<fantasai> myles: if goal is to get hairlines, even if you get a little closer to hairline, can still disappear
<fantasai> ??: Display scaling might give you smoother color if checkerboard is pixel-perfect before the scaling
<fantasai> ??: and if you have checkerboard before scaling, then less smooth design
<fantasai> s/??/mstage/
<jgilbert> s/mstage/mstange/
<fantasai> mstange: still discussing value of getting an accurate box? what's status?
<fantasai> jgilbert: if we decide that we don't want to give ppl this box, then ?
<fantasai> jgilbert: do we want to get into that? or do we want to take it as a given that we're trying to give ppl most correct idea of device pixel than we can?
<fantasai> mstange: what we would need to do in Firefox to get this result
<fantasai> mstange: Firefox takes into account the full zoom, and takes into account CSS Transforms
<fantasai> mstange: space in which we snap is established by the closest non-animated transform or the root
<fantasai> mstange: we only know this info during painting
<fantasai> mstange: so would need to run more steps before giving info
<fantasai> smfr: this device border box would not be affected by transforms
<fantasai> smfr: so would have to do *special* calculation
<fantasai> jgilbert: 3D transforms no idea what to fee dback
<fantasai> jgilbert: pixel-snapped bounding box?
<fantasai> atotic: if you're doing transforms won't be pixel-perfect anyway, so don't worry about it
<fantasai> atotic: your'e also talking about implementation that you need to get this nformation during pre-painting
<fantasai> atotic: remember you had this information ??
<fantasai> atotic: Might be ok to deliver incorrect information and not have to paint
<fantasai> myles: there's a chance of you never get correct info
<mstange> s/closest non-animated transform/closest animated ancestor transform/
<fantasai> atotic: I think you want, once things have settled down want accurate box size
<fantasai> atotic: if animating, don't care so much
<fantasai> jgilbert: deliver informmation lately
<fantasai> jgilbert: would be nice if we could try so that you could do 1-1 perfect every time within certain set of constraints
<fantasai> jgilbert: be nice if we didn't immediately settle on a 1 frame late policy
<fantasai> jgilbert: matching what native APIs are able to do
<fantasai> jgilbert: don't want to suffer if dont' have to suffer on native
<fantasai> myles: if it is one frame late then the exact timeline that we drop frames is exposed to the Web so don't think we want
<fantasai> myles: similarly, worried that we'd have to reverse-engineer chrome's pixel-snapping strategy
<fantasai> jgilbert: shouldn't have to do more normalziation than you do today, I think
<fantasai> jgilbert: if trying to use this to get anti-moire, in order to do this today have to ??
<fantasai> smfr: one frame late version also problem of which rectangle to trust
<fantasai> chrishtr: not late if layout occured
<fantasai> chrishtr: only late if you have a threaded animation
<fantasai> smfr: thought it would be late because we woudl collect info at paint time
<fantasai> chrishtr: I don't think we shoudl do that, disagree with atotic
<fantasai> chrishtr: think we can do it therefore we should
<fantasai> chrishtr: in cases where layout has occured
<fantasai> chrishtr: agree with point about threaded animations and not syncing with JS thread
<fantasai> smfr: want to tie together two of previous points, first that if we implement this paint day device pxel border box, will be more expensive for us
<fantasai> smfr: secondly because of physical mismatch, wouldn't have some user benefit
<fantasai> smfr: display is scaling anyway
<fantasai> smfr: can we get examples?
<fantasai> smfr: want to try on Apple devices, see what would happen if looks right
<fantasai> atotic: moire patter
<fantasai> atotic: if you're moving canvas around the screen, the moire is animated. looks really bad
<fantasai> smfr: please give us some tests
<fantasai> chrishtr: I think time is up?
<fantasai> chrishtr: propose to add the feature?
<fantasai> Rossen_: does the group feel there's enough merit to add this?
<fantasai> Rossen_: would be adding both APIs
<fantasai> Rossen_: is there any objection?
<fantasai> smfr: I dont' think we can accept new API without evidence that it's so much better that it's worth the extra cost
<fantasai> fremy: you can agree with the API and just return some approximate result if you feel it's good enough
<fantasai> myles: your'e saying implement the API without implementing teh API
<fantasai> fremy: might not be useful on your device
<fantasai> fremy: but might be useful on Android
<fantasai> fremy: so just return the result
<fantasai> AmeliaBR: Not having an API is better than having API with bad results
<fantasai> AmeliaBR: if a particular browser has particular concerns about implementing the particular API
<jgilbert> fantasai: in this case it would be, if you as an author were trying to do this thing, and you had this browser gives me the actual DPR, and this one gives me some approximation of the size
<jgilbert> ... I still need to size my box either way
<jgilbert> ... if my choice is I can get teh actual device pixel size from Chrome, but calcaulate it myself from widht/height props in WebKit, and get an approx result, then I don't know it might be better to have an API that does that calculation for me
<jgilbert> ... then I only need to write one code flow
<fantasai> AmeliaBR: might make sense, but have existing cases where you can't trust the API
<fantasai> florian: In general I agree with your statement, this doesn't seem to be such a case
<fantasai> Rossen_: I woudld like to end this topic
<fantasai> Rossen_: going to call for objections to adding the API, if we have objections we'll deal with them and have additional conversatiosn with the TAG and whatnot
<fantasai> Rossen_: any objections to having these APIs?
<fantasai> smfr: Can we wait until we ahve the examples?
<fantasai> astearns: I might object because seems like we need more data
<fantasai> astearns: so I will object on Safari's behalf
<fantasai> Rossen_: so back to you to get test cases, we'll discuss again on telecon

[kdashg]

My manual testing demo page: https://jdashg.github.io/misc/webgl/device-pixel-tester.html