Player is parsing in-band 708 captions when ffmpeg says they do not exist

[evanfarina]

Description

I'm seeing an issue where ffmpeg and the player disagree on which in-band captions exist within the video stream. ffmpeg shows a single subtitle track 'eia_608' whereas the player is parsing both 608 and 708 captions. More details about the stream, how I'm testing, what I've found happening in the player, and expected behavior are outlined below.

Sources

• Public stream

  Steps to reproduce

  Explain in detail the exact steps necessary to reproduce the issue.

1. Visit the http-streaming test page
2. Enter https://streamweu-livectorprodmedia17-euwe.licdn.com/26a5d57a-de50-47c0-a3ca-676e34b35ef1/L4E5f1d4af9aa014000-livemanifest.ism/manifest(format=m3u8-aapl-v3) for the stream and application/x-mpegURL for the type
3. Seek to the ~30 min mark (this is when the 708 track comes in)
4. Notice that the captions menu contains both 708 (CC_708_1) and 608 (CC1) captions

Results

Expected

The player should not find caption tracks that are not present in ffmpeg's output.

Additional Information

Here is the ffmpeg command used to verify the in-band captions: ffmpeg -f lavfi -i "movie=file2.ts[out0+subcc]" -map s output.srt Where file.ts was this segment`.

ffmpeg output:

Input #0, lavfi, from 'movie=file.ts[out0+subcc]':
  Duration: N/A, start: 2916.028000, bitrate: N/A
  Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 512x288 [SAR 1:1 DAR 16:9], 29.97 tbr, 90k tbn, 90k tbc
  Stream #0:1: Subtitle: eia_608

Debug notes

The captions are first parsed by the parseCaptionPackets function. In this function, the below code determines the caption's type:

for (i = 0; i < count; i++) {
          offset = i * 3;
          data = {
            type: userData[offset + 2] & 0x03,
            pts: pts
          }; // capture cc data when cc_valid is 1

          if (userData[offset + 2] & 0x04) {
            data.ccData = userData[offset + 3] << 8 | userData[offset + 4];
            results.push(data);
          }
        }

Once the type is returned from the above call, CaptionStream$1.prototype.flushStream runs the following logic where type is used to decide whether a caption is 608 or 708:

this.captionPackets_.forEach(function (packet) {
          if (packet.type < 2) {
            // Dispatch packet to the right Cea608Stream
            this.dispatchCea608Packet(packet);
          } else {
            // Dispatch packet to the Cea708Stream
            this.dispatchCea708Packet(packet);
          }
        }, this);

I don't fully understand the bitwise operation happening in the above parseCaptionPackets logic (If someone would like to explain it to me I'd really appreciate it) but assuming ffmpeg's parser is correct, it seems like there is a bug in the assignment of type

videojs-http-streaming version

what version of videojs-http-streaming does this occur with? I believe the demo page points to 'main'

Browsers

what browsers are affected? please include browser and version for each

• Chrome, Firefox (any browser other than Safari since Safari provides its own parsing logic)

Platforms

what platforms are affected? please include operating system and version or device and version for each

• Mac OS

Other Plugins

are any other videojs plugins being used on the page? If so, please list them with version below.

• None

Other JavaScript

are you using any other javascript libraries or frameworks on the page? if so please list them below.

• None

[gesinger]

Hey @evanfarina , I grabbed a random segment in the middle of the stream and ran mediainfo on it and it shows two text tracks, one 608 and one 708:

Text #1
ID                                       : 300 (0x12C)-CC1
Menu ID                                  : 1 (0x1)
Format                                   : EIA-608
Muxing mode                              : SCTE 128 / DTVCC Transport
Muxing mode, more info                   : Muxed in Video #1
Duration                                 : 2 s 1 ms
Bit rate mode                            : Constant
Stream size                              : 0.00 Byte (0%)
CaptionServiceName                       : CC1

Text #2
ID                                       : 300 (0x12C)-1
Menu ID                                  : 1 (0x1)
Format                                   : EIA-708
Muxing mode                              : SCTE 128 / DTVCC Transport
Muxing mode, more info                   : Muxed in Video #1
Duration                                 : 2 s 1 ms
Bit rate mode                            : Constant
Stream size                              : 0.00 Byte (0%)

[evanfarina]

Thanks, @gesinger! I can confirm that I'm seeing the same output for multiple segments throughout the stream. It seems like different parsers provide different outputs. But, I think there is still an issue here which is that the player is showing two caption options when each contains the exact same text. Notice that for the same stream Safari only shows one. Should we fix this at the public player level or is it preferred that I handle this in my company's internal player?

[gkatsev]

@evanfarina afaik, Safari doesn't support 708 captions.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.