search

cta-wave / technical-working-group

7 stars 0 forks source link

Topics for discussion at the CTA Fall Forum Workshop 2022 #21

Open ablasgen opened 2 years ago

ablasgen commented 2 years ago

WAVE is holding a workshop on Tuesday, September 27 from 9 am-12 pm PT to cover some of the decision points which require a deeper discussion than the regular TF/group meetings.

Use this issue tracker to contribute discussion items. These will be collated by the TWG Chair before the meeting and converted in to a presentation deck to drive the agenda for the day.

For each item, please suggest a title, and provide some background text as well as the decision to made or call-to-action. Also state which TF or group you feel the work falls under.

ablasgen commented 1 year ago

It has been suggested that WAVE consider leveraging the upcoming Device Playback test suite to allow for companies to self-certify their products' compliance with WAVE requirements. Interested companies would need to pay fees and thus this approach could generate some revenue. However, it has been pointed out that there needs to be a clear driver for such a program, otherwise there is no incentive to go through the process.

The Workshop can be used to discuss this proposal with the goal of exploring potential interest and requirements.

javierbarellano4v commented 1 year ago

DPCTF execute tests with "archived" test content - is there any strong interest in supporting testing with a live stream?

Previously, supporting and hosting a 24x7 live stream was proposed as a backlog item at one point.

Once support for testing with a live stream is implemented, a potential next step is to support sourcing any arbitrary, live video source in DPCTF, i.e., sourcing live video not provided/hosted by CTA. Supporting this feature is likely much more complex and its feasibility not clear.

Deploying and hosting a live stream for testing would be beneficial to device manufacturers.

jpiesing commented 1 year ago

DPCTF execute tests with "archived" test content - is there any strong interest in supporting testing with a live stream?

Previously, supporting and hosting a 24x7 live stream was proposed as a backlog item at one point.

Once support for testing with a live stream is implemented, a potential next step is to support sourcing any arbitrary, live video source in DPCTF, i.e., sourcing live video not provided/hosted by CTA. Supporting this feature is likely much more complex and its feasibility not clear.

Deploying and hosting a live stream for testing would be beneficial to device manufacturers.

Examples of sort-of live test streams can be found at https://dvb.ebu.io/ and http://refapp.hbbtv.org/livesim/02_llamav2/manifest.mpd.

cta-source commented 1 year ago

AV Sync -- Requirements and Testability

This discussion has its roots in DPCTF Issue #100, "Choice of value for A/V sync requirement in DPCTF spec and testing #100".. That thread, plus final discussion in DPCTF, led to a decision to use +40/-120 mS as the DPCTF A/V spec. However, that decision did not take into account the video timing resolution in our test design, which has previously been asserted as "20 mS + 2*(1/frame rate)".

It's important that a sync requirement is something that can be tested. An ITU recommendation Rec. ITU-R BT.1359-1 1 Relative Timing of Sound and Vision for Broadcasting plus the audio white noise measurement resolution was the genesis of the DPCTF decision for +40/-120 mS.

We will reconsider our A/V sync requirements and test capabilities in this Fall Forum Workshop discussion item. A sidebar led to the following table of tolerances on the video capture - mediaTime - currentTime and a possible resulting upper bound on what we can measure.

<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:x="urn:schemas-microsoft-com:office:excel" xmlns="http://www.w3.org/TR/REC-html40">

Ref. | Location | Type | Expected Value | Tolerance Stack | Comments -- | -- | -- | -- | -- | --   | Error in the content playout on the DUT |   |   |   |   A |   | Source Content Dither (sync uncertainty) encoded into the CMAF adaptation set and elementary streams | Negligible | +/- 0 mS | Assume negligible for now. B |   | Source Video Resolution, of the video frame rate of the test vector | 2/(vector frame rate) | +/- 80 mS | For a content frame rate of 25 fps, this is 80 mS (worst-case scenario) C |   | Source Audio Resolution, of the audio sample rate of the test vector | Negligible | +/- 0 mS |   D |   | DUT Player Dither, injected by the JS player and playout mechanism | Subject of test | TERROR |     | Uncertainty in the capture and processing |   |   |   E |   | Capture Video Resolution, frame-rate limitation of the test harness camera | 1/(capture frame rate ) | +/- 8 mS | For a camera of 120 fps, this is about 8 mS F |   | Capture Video Digitization Dither, skew from camera sensor activation on a frame, to assigning a system time value in a record saved on HDD | Budget 10 mS? | +/- 10 mS | Once the first frame is logged at a system time, the subsequent frames should track that time very well. G |   | Capture Audio Sample Period, white noise observation period | 20 mS | +/- 20 mS |   H |   | Capture Audio Sampling Dither, skew from audio reaching mic, to assigning a system time value in a record saved on HDD | Budget 10 mS? | +/- 10 mS |   I |   | Capture Audio Algorithm Dither, white noise algorithm error between samples | Negligible | +/- 0 mS | From testing, the value is too small to measure.   |   |   |   |   |   K |   | Total |   | +/- 128 mS | Measurement should be this value plus TERROR

gitwjr commented 1 year ago

The following lists items that were in the original DPCTF spec but not in the DPCAT spec and were therefore deleted when the text in DPCAT21 sections was added. The DELETED text is in bold text e.g. "with the earliest presentation time tf[k,1]=0."

There were other cases where the original spec only had a couple of requirements which were revised or rewritten in the corresponding DPCAT21 section. Those deletions are not listed below.

  1. 8.17.2; 8.20.2; 8.20.2; 8.21.2; 8.22.2; 8.23.2 Pre-Condition A CMAF Track is available for playback following the properties in clause 5.3.2.4 with the earliest presentation time tf[k,1]=0. <The deleted section in BOLD is not in the DPCAT spec.>

NOTE: The above full text without the deletion “A CMAF Track is available for playback following the properties in clause 5.3.2.4 with the earliest presentation time tf[k,1]=0” does appear in DPCAT21 section 8.19.2 Pre-Condition and is therefore included in that section in its entirety.

  1. 8.17.5 (and 8.17.5.1, .2, .3) Required Observations

8.17.5 Required Observations Original Text ● The waiting state is signaled via a QR code by the application for usage by the observation framework. ● Every sample S[k,f] shall be rendered and the samples shall be rendered in increasing presentation time order. ● The playback duration matches the duration of the CMAF fragments as defined in playout taking into account: ○ missing starting and ending frames ○ potential start frames being rendered before playback ○ frozen last frame after the playback ended ○ waiting_duration as the time to wait before appending new data to the buffer after a stall event ● The presented sample matches the one reported by the currentTime value within the tolerance of the sample duration

Revised Text (The revised text below is not in DPCAT spec. It was added to DPCTF spec after DPCAT updates were done). 8.17.5 Required Observations The waiting state is signaled via a QR code by the application for usage by the observation framework. [Deleted] 8.17.5.1 General 1) Every sample S[k,f] shall be rendered and the samples shall be rendered in increasing presentation time order. 2) The playback duration matches the duration of the CMAF fragments as defined in playout taking into account: o missing starting and ending frames o potential start frames being rendered before playback o frozen last frame after the playback ended o waiting_duration as the time to wait before appending new data to the buffer after a stall event

8.17.5.2 Video If the track is a video track, then The presented sample matches the one reported by the currentTime value within the tolerance of +/- (2/framerate + 20ms).

8.17.5.3 Audio The mediaTime of the presented sample matches the one reported by the currentTime value within the tolerance of +/- 20ms.

  1. 8.18.2 Pre-Condition [Following list entry 2] The CMAF fragments of the required CMAF Switching Sets and CMAF Switching Tracks are aligned and continuous (compare to fragment 4 and fragment 5 in the example above) <The deleted section in BOLD is not in the DPCAT spec. >

  2. 18.20.2 Establishing a proper output environment. In order to simulate low latency playback with CMAF chunks it is necessary to setup an HTTP server that supports HTTP 1.1 chunked transfer encoding and emulates delivery of CMAF chunks at encoder speed (usually realtime speed, e.g one second of content results in approximately one second delivery time). <The deleted section in BOLD is not in DPCAT doc.>

  3. 8.20.5 Required Observations • The playback duration fulfills: TR[k, S] <= TR [k, 1] + td[k] + gap_duration +/- tolerance <The deleted section in BOLD is not in DPCAT doc.>

wilaw commented 1 year ago

I'd like to present an overview of the Common Media Server Data specification, which will soon be ready for TWG review.

haudiobe commented 1 year ago

DPCTF 2022/09/21

wilaw commented 1 year ago

Let's remember to take of picture of the group during a break, with the WAVE logo up on the screen and everyone in front.