Closed palemieux closed 1 year ago
@nigelmegitt See revised PR:
We probably also need to note that the expectation is that empty ISDs are presented but the complexity/presentation time cost of clearing the display is counted only once.
Where my mind is at: painting and presenting empty ISDs involves no complexity since not displaying content (as opposed to clearing a buffer) does not involve work.
Where my mind is at: painting and presenting empty ISDs involves no complexity since not displaying content (as opposed to clearing a buffer) does not involve work.
I don't think this is consistent with the spec, which does assign some complexity to CLEAR. Setting all the pixels of the output buffer to be empty/transparent clearly does take finite time.
I'd be happy with an adjustment that separates clears from paints and assigns a fixed complexity to clears. Doing that as a general case would obviously be a worst case scenario, since a real implementation could cache a set of rectangles in which content had been painted in the most recently presented ISD and only clear those rectangles. I think worst case is what we want here.
I don't think this is consistent with the spec, which does assign some complexity to CLEAR. Setting all the pixels of the output buffer to be empty/transparent clearly does take finite time.
The specification considers that the contents of the document are drawn into buffers that are composited onto the related video element. The idea is that clearing a buffer (prior to painting a new ISD) adds complexity, but not displaying a buffer does not add significant complexity.
The Timed Text Working Group just discussed IMSC-HRM - Coalesce empty ISDs into non-empty ISDs w3c/imsc-hrm#50
, and agreed to the following:
SUMMARY: Discussions to continue offline and possibly in additional call
The Timed Text Working Group just discussed Coalesce empty ISDs into non-empty ISDs w3c/imsc-hrm#50
, and agreed to the following:
SUMMARY: @palemieux to update the pull request
This change has them the other way around!
I agree. DECE had adopted the opposite convention. Let's fix it now.
This feels like a nit, and I'm not sure how important it is, but I was surprised by the ordering of front buffer and back buffer. I expected them to be the other way around. By way of quick research I found Wikipedia page about double buffering which also seems to suggest that the front buffer is the one that is closest to the display, and the back buffer is the one you draw into before copying into the front buffer. This change has them the other way around!
I like the diagrams: the isd-rendering-time.svg might usefully have some indication of when rendering begins and ends vs when presentation begins and ends. It looks a little strange that the rendering duration for every ISD exactly fills the available time: a document that passes will always have the rendering duration be shorter, and one that fails will have at least one ISD that takes too long to render. I wonder if it's worth adjusting to show that?
See https://github.com/w3c/imsc-hrm/pull/50/commits/696f8fe51a16ecd0982768c14d8fc36a2134d912
Quoting myself:
we assume that the document is available and parsed into ISDs even before the user begins playback of the related video object
Now I think about it, there's another weird thing here. Is the implementation supposed to pre-render the first ISD into the back buffer just in case the user begins playback? Or is it supposed to predict when playback is going to begin and, IPD earlier than that, begin rendering the first ISD? For a sequence, e.g. of fragmented MP4, this is actually reasonable, since the playback time is predictable. But for a single document instance associated with a video resource that may begin playback at any time, it is strange.
In that specific case, arguably the first ISD should begin being rendered at time zero, to give the implementation a chance. Could this be too philosophical and not something to worry about? I have seen real world implementations where the first ISD did not display if its begin time was too early, including examples where begin == 0s.
Is the implementation supposed to pre-render the first ISD into the back buffer just in case the user begins playback?
There is latency inherent in starting A/V playback, e.g., the playback start time does not necessarily align with video I-frames or audio frame boundaries. The HRM acknowledges this by introducing a latency of IPD and gives the player some time, before playback starts, to render the first ISD that will be displayed.
There is latency inherent in starting A/V playback
It seems like a fragile design choice here to rely on implementation-dependent and/or codec-dependent features of the related video playback.
Closes #49 and #52
Preview | Diff