[New feature] LLHLS Dump for VoD Service

[getroot]

LLHLS Dump for VoD Service

Set <Application><Publishers><LLHLS> as follows.

<LLHLS>
    <Dumps>
        <Dump>
            <Enable>true</Enable>
            <TargetStreamName>stream*</TargetStreamName>

            <Playlists>
                <Playlist>llhls.m3u8</Playlist>
                <Playlist>abr.m3u8</Playlist>
            </Playlists>

            <OutputPath>/service/www/ome-dev.airensoft.com/html/${VHostName}_${AppName}_${StreamName}/</OutputPath>
        </Dump>
    </Dumps>
        ...
</LLHLS>

LLHLS Live will be dumped for VoD to the directory you configured. This allows you to provide VoD streaming with Apache or Nginx.

You can test it with the master branch, it will be released from 0.14.11.

[Trungdo253]

wow a great feature. Thank team AirenSoft!

[NkInKaIgIe]

@getroot Can this feature combine multiple VODs of a stream key multiple times. Currently using only the most recent VOD live stream

[getroot]

@NkInKaIgIe Do you mean merging videos? Or do you mean separating directories? Combining videos is difficult. Re-transcoding may be required. Separating directories can be solved by "time macro {time:ddhhss}" or creating a new folder when there are duplicate folders. (Or let me know if you have any good ideas.)

[p4block]

Hello, this feature is incredibly useful for us too!

Working great at a technical level, but we cannot use it because re-streaming to the same endpoint keeps recreating the llhls.m3u8 as @NkInKaIgIe said. I tried the time macro from the FILE publisher but it didn't work.

[getroot]

@p4block I haven't implemented the time macro in LLHLS dump yet. I'll add it soon.

[p4block]

I found a strange quirk, it's only recording the first 24ish seconds of our test streams. Files sizes on disk match this and ovenplayer just stops. Happened with sample videos and an ffmpeg testsrc. Are we doing something wrong on our end?

[getroot]

@p4block I don't know what the sample videos and ffmpeg testsrs are.

Did you try to play VoD after the live stream completely ended? Or did you try to play VoD while the live stream was in progress? Have you tried refreshing the page that is playing VoD? The chunklist is constantly being updated. Of course, that can happen if you try to play in the middle of a live stream. Players load the chunklist once when playing VoD and do not automatically update it again. Because it is VoD. If there is no problem, try uploading your dumped m3u8 files for me to see. Uploading the ovenmediaenigne.log file also helps.

[p4block]

@getroot I'm tring to play after the streaming ends, and it's not about the player itself, as the files on disk are actually that small. I tested with video files dumped from IP cameras and with this ffmpeg testsrc command.

ffmpeg -re -f lavfi -i testsrc2=d=3600:s=1280x720:r=24,format=yuv420p -f lavfi -i sine=f=440:b=4 -shortest -c:v libx264 -vf format=yuv420p -preset ultrafast -crf 24 -bf 0 -an -f flv "rtmp://ourtestserver:1935/app/stream-desktop"

cat default_app_stream-desktop/llhls.m3u8 
#EXTM3U
#EXT-X-VERSION:7
#EXT-X-INDEPENDENT-SEGMENTS

#EXT-X-STREAM-INF:BANDWIDTH=2000000,RESOLUTION=1280x720,FRAME-RATE=24.0,CODECS="avc1.42c01f"
chunklist_0_video_llhls.m3u8

These videos have quirky timestamps, but that wasn't an issue before with streaming or file output. Can test tomorrow with the Big Bunny movie or something. Logs in OME show nothing relevant, although I did not enable any kind of debug mode.

[getroot]

@p4block I need the chunklist_0_video_llhls.m3u8 file. Please upload that and does your llhls live play normal? and please check if there are any problems when using OBS and let me know.

[naanlizard]

What is the advantage of this method for VODs vs recordings

[getroot]

@naanlizard LLHLS Dump can provide VoD-ABR because it dumps m3u8 and all track segments. Even while Live is in progress, you can immediately provide VoD with the files dumped up to that point.

[naanlizard]

Ah thank you, that is useful.

Does Ovenplayer offer a seek bar? eg a page loads, the stream is live and by default the player loads the most recent video, but you can click the seek bar to return to the beginning of the stream or anywhere in the middle?

So these files are just dumped to the disk as they are generated? No need to dump them at a point in time?

[p4block]

@getroot It works great with OBS, this feature is honestly amazing.

The test streams we've been using however only generate up to 20-something-ish seconds though and I can't figure out why, specially when it also bugs out with a camera stream we have saved.

Those streams work fine for webrtc/llhls live playback though.

I dumped the correct file now, from a 1.5 minute stream

 cat chunklist_0_video_llhls.m3u8 
#EXTM3U
#EXT-X-VERSION:6
#EXT-X-TARGETDURATION:6
#EXT-X-MEDIA-SEQUENCE:0
#EXT-X-MAP:URI="init_0_video_FwLTYVtv_llhls.m4s"
#EXT-X-PROGRAM-DATE-TIME:2022-10-19T13:56:22.607+00:00
#EXTINF:8.333000,
seg_0_0_video_FwLTYVtv_llhls.m4s
#EXT-X-PROGRAM-DATE-TIME:2022-10-19T13:56:30.940+00:00
#EXTINF:8.333000,
seg_0_1_video_FwLTYVtv_llhls.m4s
#EXT-X-PROGRAM-DATE-TIME:2022-10-19T13:56:39.273+00:00
#EXTINF:8.327000,
seg_0_2_video_FwLTYVtv_llhls.m4s
#EXT-X-ENDLIST

The m3u8 just in case

cat llhls.m3u8 
#EXTM3U
#EXT-X-VERSION:7
#EXT-X-INDEPENDENT-SEGMENTS
#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="1",NAME="none",AUTOSELECT=YES,DEFAULT=YES,CHANNELS="1",URI="chunklist_1_audio_llhls.m3u8"

#EXT-X-STREAM-INF:BANDWIDTH=4617503,RESOLUTION=1280x720,FRAME-RATE=30.0,CODECS="avc1.42c01f,mp4a.40.2",AUDIO="1"
chunklist_0_video_llhls.m3u8

and a view of the folder

24 seconds in OvenPlayer:

It's definitely possible the test stream is not encoded properly, although it has always served us well, the command to generate it is in the previous comment. Very much open to suggestions on better ways to test streams aside from big_buck_bunny.mp4 :grimacing:

[getroot]

@p4block I suspect that the keyframe interval of your ffmpeg input is very long. Since the segment must start with a keyframe, it is not created unless a keyframe is entered. Try adding an option like -r 30 -g 30 .

[p4block]

@getroot I added those params to the output of the encoder and it seems like it's working normally now, thanks a lot.

For reference, our command is now this:

paths:

  benchmark:
    runOnInit: ffmpeg -re -f lavfi -i testsrc2=d=3600:s=1280x720:r=30,format=yuv420p -f lavfi -i sine=f=440:b=4 -shortest -c:v libx264 -vf format=yuv420p -preset ultrafast -tune zerolatency -profile main -crf 32 -bf 0 -r 30 -g 30 -f rtsp rtsp://localhost:$RTSP_PORT/$RTSP_PATH
    runOnInitRestart: yes
    runOnReady: ffmpeg -i rtsp://localhost:$RTSP_PORT/benchmark -c copy -f flv "rtmp://testserver:1935/app/stream-testo"
    runOnReadyRestart: yes

Now I have to edit a lot of ffmpeg commands around the company, but it's good to know you've been doing things wrong.

[getroot]

Ah thank you, that is useful.

Does Ovenplayer offer a seek bar? eg a page loads, the stream is live and by default the player loads the most recent video, but you can click the seek bar to return to the beginning of the stream or anywhere in the middle?

So these files are just dumped to the disk as they are generated? No need to dump them at a point in time?

@naanlizard This is not yet a Live DVR feature. (This is an intermediate result in the process of providing Live DVR.) If you designate the dump folder location as an Apache or Nginx folder, you can immediately provide VoD service.

In fact, if you can afford to use a lot of memory now, you can enable Live DVR. (This should be improved by saving the old segments to a file and reading from the file for long live streams.) It's just not exposing the seek bar in OvenPlayer. Set the SegmentCount to 600 and play the stream from the hls.js demo page (https://hls-js.netlify.app/demo). Like Live DVR, you can watch past videos. Expose seek bar in OvenPlayer will be updated.(@SangwonOh )

[naanlizard]

That is very exciting. We will try and test this ASAP, I will watch git but if you ping me when it is released, it will be even faster :)

[p4block]

Feature working great, really excited for what it brings to OME but without the date macro or another way to separate streams to the same endpoint I'm a bit stuck :(

Hopefully gets released soon!

[getroot]

@p4block

ah! This is a very simple task, but I've been so busy lately that I've forgotten about it. I just added a simple macro and committed. You can test it with the master branch. The following macros have been added.

${YYYY} : year
${MM} : month
${DD} : day
${hh} : hour
${mm} : minute
${ss} : second

Set it like this:

<Dumps>
    <Dump>
        <Enable>true</Enable>
        <TargetStreamName>stream*</TargetStreamName>

        <Playlists>
            <Playlist>llhls.m3u8</Playlist>
        </Playlists>

        <OutputPath>/service/www/ome-dev.airensoft.com/html/${VHostName}_${AppName}_${StreamName}/${YYYY}_${MM}_${DD}_${hh}_${mm}_${ss}</OutputPath>
    </Dump>
</Dumps>

[p4block]

Wow, thanks a lot! That was really quick.

While testing I hit the "keyint too big bug" again with our real hardware cameras, found they have a 10s (250 frame) interval and that was enough for the LLHLS dump to only record 20s or so of stream.

After changing it, seems to work great except for a minor quirk. the "ss" macro seems to create "3" instead of "03", making it difficult to make predictable file names. Is this intended?

[p4block]

Now the recording seems to happen automatically thanks to the wildcard <TargetStreamName>stream*</TargetStreamName>, is/will it be wired up to the recording API that currently controls the FILE output?

[getroot]

@p4block

Now the recording seems to happen automatically thanks to the wildcard <TargetStreamName>stream*</TargetStreamName>, is/will it be wired up to the recording API that currently controls the FILE output?

I guess I didn't quite understand this question. The wildcard in TargetStreamName is a feature that has been around since the beginning.

After changing it, seems to work great except for a minor quirk. the "ss" macro seems to create "3" instead of "03", making it difficult to make predictable file names. Is this intended?

That part I developed without much concern. Which method do you think is best?

[p4block]

Sorry, let me rephrase. I wonder if the OME API to start/stop recording a stream (FILE output) currently controls the LLHLS dump.

As for the time format, I tried to just copy the ISO 8601 date standard

So around now for example would be: 20221121T134403

              <Dumps>
                <Dump>
                  <Enable>true</Enable>
                  <TargetStreamName>*</TargetStreamName>

                  <Playlists>
                    <Playlist>llhls.m3u8</Playlist>
                    <Playlist>abr.m3u8</Playlist>
                  </Playlists>
                  <OutputPath>/vods/${VHostName}_${AppName}_${StreamName}/${YYYY}${MM}${DD}T${hh}${mm}${ss}</OutputPath>
                </Dump>
              </Dumps>

            </LLHLS>

But due to that behavior it is impossible(?) to generate proper ISO dates as it will generate as 20221121T13443

[getroot]

@p4block FILE Output and LLHLS Dump are different functions. Your opinion is reasonable. I'll update it to output in %02d format.

[getroot]

@p4block I modified the existing macro to output in %20d format. And added the following macros.

${ISO8601} : output current time in ISO 8601 format ${S} : tm zone ${z} : UTC offset (ex: +0900)

[getroot]

@p4block LLHLS Dump now forcibly dumps the file even if the keyframe interval is very long, like you had before. However, if the keyframe interval is very long, the dumped VoD may not play smoothly, so it is better to keep the interval of 1-2 seconds like now.

[p4block]

Everything working great :)

The warning on the OME logs about the too big keyframe has helped solve a few wrongly configured sources we had.

The lack of API control of whether the VOD is generated or not is a bummer but we're working around it. Feature would be awesome though :grinning:

[getroot]

LLHLS Dump can now be controlled by API. https://github.com/AirenSoft/OvenMediaEngine/issues/973

[p4block]

<3

[ChR-iSz]

@NkInKaIgIe Do you mean merging videos? Or do you mean separating directories?

Combining videos is difficult. Re-transcoding may be required. Separating directories can be solved by "time macro {time:ddhhss}" or creating a new folder when there are duplicate folders. (Or let me know if you have any good ideas.)

Re-Transcoding is not required. Files can cat (as example ffmpeg concat) without any re-transcoding.