stream titles (metadata) in js player?

I recognize the js player acts in a way "best possible stream version match". However our use case is for the user to choose among different stream content, as in "CAM1, CAM2" etc. Therefore I am looking to label these streams in the js player.

I can tell ffmpeg to do -metadata:s:v:1 title="my title" and judging from an srt output via Mistserver (mpegts packaging), this information successfully passes through Mistserver. VLC reads this.

However with WebRTC I wouldn't know where to add this information to the js player and how. Is this a matter of skins? If so: Where are these skins stored? Or can it be done post-load via MistObject?

In any case I'd be thankful for a guiding hint or an example even.

Thanks Regards Manu

Hey Manu,

Interesting, I wouldn't expect the title metadata to stay if it's parsed through MistServer. Do you perhaps use raw mode with SRT?.

If I understand correctly, you have a stream with several video tracks and want to label them so that the video track is named something else than just the track number when you select between them? Do you want this visible just in the webplayer, or in VLC as well? Both are doable, but require some changing...

This is something we could add to MistServer, but isn't ready yet. And considering the (paid) projects we have currently planned it will probably take us quite a while before we get to this by ourselves.

I can potentially see how you might be able to abuse the language track for this though. We default to showing a track language if all of them have a language set, then overriding the language to another word should be possible as well... It would be a very roundabout method, but you could ingest the stream, set up an encoder for the video track through MKV-exec https://docs.mistserver.org/mistserver/concepts/stream_process#matroska-inout then add that metadata or set a language, and only use the processed tracks. A copy of a video stream is quite cheap resource-wise.

With kind regards,

Balder Viëtor Head of Testing

MistServer

On Fri, Aug 2, 2024 at 12:58 PM labor4 @.***> wrote:

Hi

I recognize the js player acts in a way "best possible stream version match". However our use case is for the user to choose among different stream content, as in "CAM1, CAM2" etc. Therefore I am looking to label these streams in the js player.

I can tell ffmpeg to do -metadata:s:v:1 title="my title" and judging from an srt output via Mistserver, this information successfully passes through Mistserver. VLC reads this.

However with WebRTC I wouldn't know where to add this information to the js player and how. Is this a matter of skins? If so: Where are these skins stored? Or can it be done post-load via MistObject?

In any case I'd be thankful for a guiding hint or an example even.

Thanks Regards Manu

— Reply to this email directly, view it on GitHub https://github.com/DDVTECH/mistserver/issues/200, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAN2RJE7DXDUGWX6ILAXYIDZPNQ65AVCNFSM6AAAAABL4KHHTSVHI2DSMVQWIX3LMV43ASLTON2WKOZSGQ2DINZRGA2DKOA . You are receiving this because you are subscribed to this thread.Message ID: @.***>

Hi Balder Thanks kindly for your reply

Interesting, I wouldn't expect the title metadata to stay if it's parsed through MistServer. Do you perhaps use raw mode with SRT?.

I definitely took something raw. I guess it was a ts stream, all tracks were visible. I wonder... I thought I'd connected via SRT... not sure now, my testing rig is not ready.

If I understand correctly, you have a stream with several video tracks and want to label them so that the video track is named something else than just the track number when you select between them?

Currently there is (probably something like)

lang
bitrate
codec
channels

so yes: "append track title or comment if present"

Do you want this visible just in the webplayer, or in VLC as well? Both are doable, but require some changing...

Some background on my motivation:

"Achieve real time discussions of deliveries while on the phone, for remote workflows in post production, directly off an SDI link."

Of course this is special interested and not priority.

Currently I argue for the use case of browser, but only because WebRTC is fast, and you can hook an AUTH before it, and one cannot hook up wss://STUFF?myauth=abcto a standalone player.

However if there is a means to do something very realtime in an external player then that becomes interesting because of easy codec support.

I can potentially see how you might be able to abuse the language track for this though. We default to showing a track language if all of them have a language set, then overriding the language to another word should be possible as well...

Agree.

But do you mean that setting...

    -metadata:s:v:0 language="insert non-conform string"

...is an easy change, OR is already implemented, IF all tracks have a non-empty entry?

I already tried above (but not on all v tracks populated at once), and it seemed (I concluded) the language convention is mandatory.

BTW I also tried (v)

    -metadata:s:v:0 language="eng"

but that wouldn't appear even on the stream status page (not tested on the very latest release). But then again, I probably have not populated all video streams at once, if that is needed.

It would be a very roundabout method, but you could ingest the stream, set up an encoder for the video track through MKV-exec

I did it via TS over SRT.

But, this is a very big penny waiting to drop. I know this is off-topic, but would you expand a little on that? I didn't understand the docs. I'm scratching my head how to get something like AV1 into mistserver from ffmpeg. TS doesn't work yet.

I understood that mkv-exec:/dir/myscript.sh is a valid input. So inside /dir/myscript.sh there is ie. an ffmpeg listener, which outputs MKV to STDOUT? If so, what is the basics between the two ffmpeg's (one being the GPU encoder)? Do we do this via SRT? I'd be so happy about a simple example how you do this!

# sender
ffmpeg -i INPUT -params -f mkv SRTTARGET 
=>
# mistserver (/dir/myscript.sh)
ffmpeg -i SRTLISTENER -params -f mkv -

Thank you very much, best wishes Manu

Hey Manu,

Language codes In the current builds you can set any language, but it needs to be the 3 letter code variant. Now we would store/accept any 3 letter code, but only some are official languages according to the ISO639. We will still store any other combination, which you could use/abuse.

Titles This is something we can add and honestly it is something we're interested in adding. I just can't say when we would get to it. Now if it is critical to you I can check what it would cost to fasttrack this, but generally that requires some compensation.

AV1 AV1 can be done through MP4 and MKV right now. using FFmpeg MKV would be the easier fit for sure. You can either use mkv-exec or from ffmpeg send it into the standard if in of an MistInEBML. For example:

ffmpeg: ffmpeg INPUT CODECS -f matroska - | MistInEBML -s streamname -

or source: mkv-exec:ffmpegscript.sh

(We recommend using a script to output matroska as you cannot use argument string quoting due to a longstanding bug). Though that said AV1 over TS is something we've experimented with and made, the specification is not 100% crystalized yet so we're a bit doubtful on whether we should release this.

SRT would be using TS internally, so that cannot carry AV1 at the moment. I would expect ffmpeg to also not support this yet.

Between server transport the easiest right now would be DTSC, which is MsitServers internal format. Any codec support in any protocol that MistServer can provide is supported in DTSC as well. So AV1 is supported there as well. You can share between your servers using the following push target:

dtsc://address:dtscport/streamname

The other side needs to be set up to accept a push (so set to a push:// format as source).

I would recommend making sure you only allow that dtsc port to be used by trusted addresses though. The default is 4200.

With kind regards,

Balder Viëtor Head of Testing

MistServer

On Mon, Aug 5, 2024 at 8:00 PM labor4 @.***> wrote:

Hi Balder Thanks kindly for your reply

Interesting, I wouldn't expect the title metadata to stay if it's parsed through MistServer. Do you perhaps use raw mode with SRT?.

I definitely took something raw. I guess it was a ts stream, all tracks were visible. I wonder... I thought I'd connected via SRT... not sure now, my testing rig is not ready.

If I understand correctly, you have a stream with several video tracks and want to label them so that the video track is named something else than just the track number when you select between them?

Currently there is (probably something like)

lang

bitrate

codec

channels

so yes: "append track title or comment if present"

Do you want this visible just in the webplayer, or in VLC as well? Both are doable, but require some changing...

Some background on my motivation:

"Achieve real time discussions of deliveries while on the phone, for remote workflows in post production, directly off an SDI link."

Of course this is special interested and not priority.

Currently I argue for the use case of browser, but only because WebRTC is fast, and you can hook an AUTH before it, and one cannot hook up wss://STUFF?myauth=abcto a standalone player.

However if there is a means to do something very realtime in an external player then that becomes interesting because of easy codec support.

I can potentially see how you might be able to abuse the language track for this though. We default to showing a track language if all of them have a language set, then overriding the language to another word should be possible as well...

Agree.

But do you mean that setting...
-metadata:s:v:0 language="insert non-conform string"
...is an easy change, OR is already implemented, IF all tracks have a non-empty entry?

I already tried above (but not on all v tracks populated at once), and it seemed (I concluded) the language convention is mandatory.

BTW I also tried (v)
-metadata:s:v:0 language="eng"
but that wouldn't appear even on the stream status page (not tested on the very latest release). But then again, I probably have not populated all video streams at once, if that is needed.

It would be a very roundabout method, but you could ingest the stream, set up an encoder for the video track through MKV-exec

I did it via TS over SRT.

But, this is a very big penny waiting to drop. I know this is off-topic, but would you expand a little on that? I didn't understand the docs. I'm scratching my head how to get something like AV1 into mistserver from ffmpeg. TS doesn't work yet.

I understood that mkv-exec:/dir/myscript.sh is a valid input. So inside /dir/myscript.sh there is ie. an ffmpeg listener, which outputs MKV to STDOUT? If so, what is the basics between the two ffmpeg's (one being the GPU encoder)? Do we do this via SRT? I'd be so happy about a simple example how you do this!

sender

ffmpeg -i INPUT -params -f mkv SRTTARGET =>

mistserver (/dir/myscript.sh)

ffmpeg -i SRTLISTENER -params -f mkv -

?

— Reply to this email directly, view it on GitHub https://github.com/DDVTECH/mistserver/issues/200#issuecomment-2269611247, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAN2RJC4AK44GT6XMMUZCK3ZP64ULAVCNFSM6AAAAABL4KHHTSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENRZGYYTCMRUG4 . You are receiving this because you commented.Message ID: @.***>

Language codes In the current builds you can set any language, but it needs to be the 3 letter code variant. Now we would store/accept any 3 letter code, but only some are official languages according to the ISO639. We will still store any other combination, which you could use/abuse. Titles This is something we can add and honestly it is something we're interested in adding. I just can't say when we would get to it. Now if it is critical to you I can check what it would cost to fasttrack this, but generally that requires some compensation.

It's not critical right now. But I keep it on my radar if a change or "hack via lang" appears.

Thanks for the insights and clarifications around ingest (MistInEBML handler) and state of affairs around TS/AV1.

I pinpoint that:

or source:
mkv-exec:ffmpegscript.sh

... conceptually carries the first part of the previous example, and will probably be piped to MistInEBML

$ cat ffmpegscript.sh
#!/bin/bash

ffmpeg INPUT CODECS -f matroska -

so I understand this better now.

Thanks for your time. Best, Manu

since the request is under consideration anyways, I close the issue.

DDVTECH / mistserver

stream titles (metadata) in js player? #200

sender

mistserver (/dir/myscript.sh)