inconsistent use of "shall" in observations

[jpiesing]

The observations are inconsistent in how they are written, in particular how they use "shall". Some of them are written with "should" - that's OK as long as the intent is that this is a recommendation. Some of them are just written in the present tense. It should be clear which requirements really are requirements.

[jpiesing]

For example, looking at 8.10.

8.10.5.1 General If the above algorithm is carried out, the following observations are expected: • Every sample S[k,s] shall be rendered and the samples shall be rendered in increasing presentation time order. • The playback duration matches the duration of the CMAF Track, i.e. TR[k, S] = TR[k, 1] + td[k]. • The start-up delay should be sufficiently low, i.e. TR[k, 1] – Ti < TSMax. • The presented sample matches the one reported by the currentTime value within the tolerance of the sample duration.

8.10.5.2 Video In addition, for video the following is expected to be observed: • The rendering for each track is scaled to the height and width of the predetermined window. • No visible shifts of objects in the video. • No visible spatial offset of pixels in the video.

Based on this, if the track is a video track, then: • Every video frame S[k,s] shall be rendered such that it fills the entire video output window.

8.10.5.3 Audio In addition, for audio the following is expected to be observed: • The audio plays with no glitches, clicks or dropouts.

[haudiobe]

Option 1: Write present tense and someone else (could even be elsewhere in the spec) writes a shall against this clause Option 2: use consistently shall/should in observations, but is it unambiguously implementable. If not, do make it implementable or do we accept this gap as something to be solved in the test design. Option 3: Leave as is and document what should/shall/is expected means

Let's go with option 3, but do some diligence wherever necessary.

[gitwjr]

@haudiobe @jpiesing I did a search for "is expected to" in technical standards. It appears its use is typically limited to state an anticipated outcome, or action when a product, person, etc., "is expected to" adhere to a level of quality or performance defined elsewhere in the standard or guidance, but does not appear to be used in clauses. I did find a definition of "SHOULD" in a NIST guidance document for Quality Assurance, albeit for Blood Stain Pattern Analysis. (See Definitions and Terms below). Following the NIST definition we could either define SHOULD as "Unless required or prohibited elsewhere in the document, IS EXPECTED TO and IS NOT EXPECTED TO indicate a result, action, function or feature is anticipated and preferred but not required or, in the negative, not prohibited." Or keep the current definition of SHOULD and add the preceding as the definition for "IS EXPECTED TO/IS NOT EXPECTED TO".
In reviewing the document it seems that SHOULD can be used in many cases such as "An application that receives a manifest referring to CTA WAVE content is expected to have access to two primary high-level APIs:". On the other hand some uses seem to actually be a requirements (SHALL) such as "A source buffer is expected to be established for each media type individually by:" or "The device platform is expected to provide rendering capabilities." since I don't see how a device can meet the standard without a source buffer or rendering capabilities. It also appears that the its use in observations needs attention. In the following, the first 2 observations are not defined as required (no SHALL). If they are required either the "is expected to" needs to state SHALL or they need to explicitly state SHALL. As written, they will be interpreted as SHOULD (expected to). "In addition, for audio the following is expected to be observed 1) The mediaTime of the presented sample matches the one reported by the currentTime value within the tolerance of +/- (2/framerate + 20ms). 2) The audio plays with no glitches, clicks or dropouts. 3) Every audio sample S[k,s] shall be rendered and the audio samples shall be rendered in increasing presentation time order.

Definitions and Terms Terms - The following terms are meant to convey the meanings specified: Shall – Done without exception Should – Expected to be done (unless otherwise documented for non-compliance) Recommended – Appropriate, but not mandatory

[gitwjr]

Implemented in v1.38 of spec. Needs final editorial review for consistent use of capitalization e.g. SHALL vs shall.