SCTE-35 being read as x-emsg rather than x-scte35

Version

Media3 1.5.0

More version details

No response

Devices that reproduce the issue

Pixel 8 Pro API 35 Emulator

Devices that do not reproduce the issue

No response

Reproducible in the demo app?

Not tested

Reproduction steps

Add a new renderer and set it to your player

val renderersFactory: RenderersFactory = object : DefaultRenderersFactory(context) {
        override fun buildMetadataRenderers(context: Context, output: MetadataOutput, outputLooper: Looper, extensionRendererMode: Int, out: ArrayList<Renderer>) {
            val metadataRenderer = MetadataRenderer(
                    output,
                    outputLooper,
                    CustomMetadataDecoderFactory()
            )

            out.add(metadataRenderer)
        }
    }

Create the CustomMetadataDecoderFactory()

@UnstableApi
class CustomMetadataDecoderFactory : MetadataDecoderFactory {
override fun supportsFormat(format: Format): Boolean {
    Log.i("CustomMetadataDecoder", "Format: ${format.sampleMimeType}")

    // Ensure support for the MIME type used by SCTE-35 metadata
    return format.sampleMimeType == "application/x-scte35" || format.sampleMimeType == "application/x-emsg"
}

override fun createDecoder(format: Format): MetadataDecoder {
    Log.i("CustomMetadataDecoder", "Decoder Metadata: ${format.metadata}")

    if(format.sampleMimeType == "application/x-emsg") {
        Log.i("CustomMetadataDecoder", "Decoding using EventMessageDecoder")
        return EventMessageDecoder()
    }

    Log.i("CustomMetadataDecoder", "Decoding using SpliceInfoDecoder")
    return SpliceInfoDecoder()
}
}

Play the DASH file

DASH File Period Sample:

<Period id='1732293437' start="PT0S" >
      <EventStream schemeIdUri="urn:scte:scte35:2014:xml+bin" timescale="1000">
        <Event id="0" presentationTime="1732294630134">
          <Signal xmlns="http://www.scte.org/schemas/35/2016">
            <Binary>/DBOAAAAAAAAAP/wBQb+gnX6iAA4AjZDVUVJAYAIhH/DAAD3NtEOIE0xU2F1SjZkeUtiQnl2WUY1b0hZSkhINWtKNG1XT1JvNAAAAABYZSZy</Binary>
          </Signal>
        </Event>
        <Event id="0" presentationTime="1732294636140">
          <Signal xmlns="http://www.scte.org/schemas/35/2016">
            <Binary>/DBOAAAAAAAAAP/wBQb+gnX6iAA4AjZDVUVJAYAIhH/DAAD3NtEOIE0xU2F1SjZkeUtiQnl2WUY1b0hZSkhINWtKNG1XT1JvNAAAAABYZSZy</Binary>
          </Signal>
        </Event>
        <Event id="0" presentationTime="1732294642146">
          <Signal xmlns="http://www.scte.org/schemas/35/2016">
            <Binary>/DBOAAAAAAAAAP/wBQb+gnX6iAA4AjZDVUVJAYAIhH/DAAD3NtEOIE0xU2F1SjZkeUtiQnl2WUY1b0hZSkhINWtKNG1XT1JvNAAAAABYZSZy</Binary>
          </Signal>
        </Event>
        ...

Expected result

SCTE-35 metadata will be read as application/x-scte35, and data would be available under onMetadata callback with the ID, Duration (optional), and Value.

Actual result

SCTE-35 metadata was read as application/x-emsg and not able to access data such as ID, and value.

tracks [eventTime=702.96, mediaPos=0.00, window=0, period=0
  group [
    [X] Track:0, id=tracks-v1, mimeType=video/avc, container=video/mp4, bitrate=2344000, codecs=avc1.4d401f, res=1280x720, fps=29.97003, supported=YES
  ]
  group [
    [X] Track:0, id=tracks-a1, mimeType=audio/mp4a-latm, container=audio/mp4, bitrate=128000, codecs=mp4a.40.2, channels=2, sample_rate=48000, language=en, supported=YES
  ]
  group [
    [X] Track:0, id=urn:scte:scte35:2014:xml+bin/, mimeType=application/x-emsg, supported=YES
  ]
]

Media

https://fs01nyc03.mdusolutions.net/CBSNewsLive24/7/index.mpd

Bug Report

[X] You will email the zip file produced by adb bugreport to android-media-github@google.com after filing this issue.

I think what you are describing is the expected behavior for metadata coming in as an e-msg. Unlike in-band metadata that is in the media, e-msg metadata is coming from the manifest and is delivered that way with the mime type application/x-emsg as it's track mime type.

The expectation that the track group has a different mime type than application/x-emsg for metadata from the event stream in the manifest isn't right I think. It can for instance well be that the event stream in the DASH manifest has different message types, so keeping the common denominator which is application/x-emsg seems correct and plausible to me.

With the existing API, I think you can recognize the type of the metadata in the MetadataOutput though. It delivers you the schemaId and the bytes that it reads from the event stream. So I think you should be able to decode the bytes when receiving them.

callback with the ID, Duration (optional), and Value.

Yeah, sorry. I don't think the event stream is provided in such a convenient way. I think mostly because of it's nature of being an event stream coming from the manifest that may contain messages of various schemeIds. I agree that this potentially can be improved in some ways as it could be made easier, but the flexibility of the current solution has it justification as well.

If I understand correctly you can do something like this:

    @Override
    public void onMetadata(Metadata metadata) {
      for (int i = 0; i < metadata.length(); i++) {
        Metadata.Entry entry = metadata.get(i);
        if (entry instanceof EventMessage) {
          EventMessage eventMessage = (EventMessage) entry;
          if ("urn:scte:scte35:2014:xml+bin".equals(eventMessage.schemeIdUri)) {
            Log.d(
                "metadata",
                "schemeIdUri="
                    + eventMessage.schemeIdUri
                    + ", id="
                    + eventMessage.id
                    + ", durationMs="
                    + eventMessage.durationMs
                    + ", size in bytes="
                    + eventMessage.messageData.length);
          }
        }
      }
    }

I tried your stream but I've never seen metadata be emitted I'm afraid. I also saw HTTP 500 playback failures usually after an ad but I don't think this is related to not seeing metadata emitted.

Can you let me know if the approach above works for you? I specifically mean wehther you see metadata emitted and then reported by EventLogger? Or don't you see the callback being triggered at all?

androidx / media