Review of IMSC-HRM

[nigelmegitt]

Wotcher TAG!

I'm requesting a

TAG review of IMSC-HRM

What it is

The IMSC Hypothetical Render Model (HRM) constrains the presentation processing complexity of subtitle and caption documents that conform to the IMSC Recommendation.

The HRM is not a new concept: it has been included in all versions and editions of the IMSC Recommendation and has remained substantially unchanged.

In order to simplify future maintenance, the TTWG wishes to refactor the HRM into its own Recommendation.

Key info

• Explainer¹ (minimally containing user needs and example code): https://github.com/w3c/imsc-hrm/blob/main/misc/explainer.md
• Specification URL: https://www.w3.org/TR/imsc-hrm/
• Tests: will be located at https://github.com/w3c/imsc-hrm-tests when they have been created
• User research: none specific to the IMSC-HRM
• Security and Privacy self-review²: https://github.com/w3c/imsc-hrm/blob/main/misc/security-and-privacy.md
• GitHub repo (if you prefer feedback filed there): https://github.com/w3c/imsc-hrm/
• Primary contacts (and their relationship to the specification):
  • Nigel Megitt (@nigelmegitt), BBC (Chair, TTWG)
  • Pierre-Anthony Lemieux (@palemieux), Movielabs (Editor, IMSC-HRM)
  • Gary Katsevman (@gkatsev), Mux (Chair, TTWG)
• Organization(s)/project(s) driving the specification: W3C TTWG
• Key pieces of existing multi-stakeholder review or discussion of this specification: This specification has been published in IMSC Recommendations and has therefore gone through Wide Review and TAG review previously as part of those IMSC Recommendations. Usage of the IMSC-HRM to analyse real world subtitle and captions documents from a variety of sources has led to issues such as w3c/imsc-hrm#49.
• External status/issue trackers for this specification (publicly visible, e.g. Chrome Status): None that we're aware of

Further details:

• [X] I have reviewed the TAG's Web Platform Design Principles
• Relevant time constraints or deadlines: We would like to transition from WD to CR in the next 2-3 months if possible
• The group where the work on this specification is currently being done: W3C TTWG
• The group where standardization of this work is intended to be done (if current group is a community group or other incubation venue): same, W3C TTWG
• Major unresolved issues with or opposition to this specification: None.
• This work is being funded by: TTWG members

Also

You should also know that...

As part of Horizontal Review, we requested review from PING and Security based on the text at https://github.com/w3c/imsc-hrm/issues/19#issue-1073404996 . The self-questionnaire linked above was produced after that, and written to be in agreement with that text.

• PING tracking issue: w3cping/privacy-request#65
• Security tracking issue: w3c/security-request#18

One question we are considering is whether we should update existing IMSC Recommendations to remove the existing HRM text and then either to 1) normatively reference IMSC-HRM as a conformance requirement or 2) informatively reference the IMSC-HRM as an optional conformance requirement for users to decide on.

The goal of the IMSC-HRM is to set a maximum presentation complexity for subtitle and caption documents to maximise the likelihood that presentation processors will successfully produce an accessible experience for end users, in particular timely presentation. It is out of scope for IMSC-HRM to model or constrain readability complexity.

Feedback

We'd prefer the TAG provide feedback as:

🐛 open issues in our GitHub repo for each point of feedback

[rhiaro]

Hi @nigelmegitt. We discussed this in our TAG call today, and have a couple of questions. As the substance of this spec has been previously published before, what level of review would you like from us? Would you like a complete re-review of the whole HRM (which would make sense if it has changed significantly), or a review of the delta (in which case, could you provide a summary of changes)? Are there any particular architectural issues you would like input to address?

either to 1) normatively reference IMSC-HRM as a conformance requirement or 2) informatively reference the IMSC-HRM as an optional conformance requirement for users to decide on.

Could you clarify this a little more please, and I'm sorry if I'm missing something obvious - which component(s) in the ecosystem would be required to conform to the IMSC-HRM in this context?

My personal feeling (not at this point representing TAG consensus) is that the utility of the IMSC-HRM algorithm as a tool in the ecosystem is clear. What I'm less clear on are the benefits of publishing it as a REC. If an implementation of the algorithm takes the form of a validator, and (if I understand correctly) 1) the ecosystem doesn't have much need for more than one validator against which to test content; and 2) the content itself is not constrained by the HRM, but simply subject to analysis whereby a value for complexity of the content is output; then an informative NOTE seems to me to be the appropriate way to express this.

On the other hand, if 1) there are to be multiple validators which all need to yield the same results for any given content; and/or 2) if a system which produces the content is itself constrained by a particular complexity value which it then needs to produce content in accordance with; I can see the case for a REC. In the latter case, the content producing system would need to implement the HRM in order to ensure it produces content with a particular complexity value (established by some external means) - is that correct?

[hadleybeeman]

Hi @nigelmegitt ! We're just looking at this at our W3CTAG face-to-face (Moon Base Alpha).

Several of us agree with Amy's suggestions above, thinking they could be the most expedient way for the working group to move forward with your important work. Is there anything else we can do or say or look at that would be helpful to you?

[nigelmegitt]

Hi @rhiaro and @hadleybeeman apologies for not noticing the Jan 9 comment until now.

Delta or complete review

In terms of the review request, there have been some significant changes, to make the algorithm clearer but also to deal with some cases discovered in real world content that caused unwanted conformance failures. It's been a while since the original HRM was reviewed in the original IMSC specification, so taking into account both of those factors, I think a complete re-review is reasonable at this stage.

However if you would prefer a delta review, we can certainly prepare a list of changes. (there's already a commit history link at the top of the working draft).

I do not think there are any particular architectural issues that we would like input on, other than the question about normative or informative referencing. Historically the most challenging parts of the discussion have been the treatment of character code points vs glyphs vs grapheme clusters, but I think we have netted out at an acceptable position, following i18n review.

The other architectural change that we made to the hypothetical render model is that we have introduced an assumption that disconnecting the front buffer from the display (buffer) when there is no content to display is a zero cost operation. Likewise re-connecting when there is content to display.

Editorially, we have changed some of the terms that we use, for clarity. For example we renamed the glyph and image "buffers" to be "caches" which we felt reflected their role better.

Normative or informative referencing from IMSC

The components in the ecosystem that need to conform are the subtitle/caption documents themselves.

Currently, with the existing published IMSC Recs, document conformance requires conformance to the HRM in its previous state. The question we are considering is if we should relax that constraint and offer it as an option for adopters via this distinct IMSC-HRM specification, or if we should refactor the existing Recs to retain the constraint, by reference to the IMSC-HRM specification rather than inclusion of the model within each of the specifications.

Rec or Note

The idea is that content out in the wild is produced to meet the HRM constraints. And that the specification is defining content conformance. Of course, checking such conformance could be done using a validator, and ideally there would be multiple validators; certainly, if there are multiple validators then it is important that they agree with each other. Production tools should effectively be validators too, since they need to produce conformant content.

In other words, the answer to your last question is Yes, that's correct. In my view, this tips the balance in favour of Rec track.

[nigelmegitt]

Discussed today in TTWG: we would really like to proceed to CR, and are targeting 2023-05-16 for first CR publication. Would it be possible to conclude this review ticket in time for us to do that?

[hadleybeeman]

Thanks for this request. We've looked at the spec and had several conversations with you and the working group through the past few months. We don't see that this proposal has any impact on the architecture of the web, so we won't hold you up any further. We are really sorry this has taken so long!