Support generation of multivariant HLS playlists

[fastfading]

Describe the feature

Description

many stream server has supported multi-resolution conversion convert media data in servers.
and generate multi-variant m3u8 and support llhls.

[mpisat]

it is possible with ffmpeg but I wish there was a way to enable ffmpeg transcoding to all streams with the config file.

if we can get something like paths: all: ..... ffmpeg with $STREAMNAME ---> $STREAMNAME_720p

when we publish stream to mediamtx, it'll transcode with ffmpeg and re-publish as stream_720p or whatever ffmpeg configuration is set to.

Another idea is to transcode AAC audio to OPUS whenever webrtc client connects on demand, so audio will work.

[mpisat]

I got these errors when I try to add transcoding to streams :) my config is

    runOnReady: ffmpeg -loglevel error -i rtsp://localhost:$RTSP_PORT/$RTSP_PATH -c:v libx264 -tune zerolatency -preset faster -b:v 2500k -bsf:v h264_mp4toannexb -g 30 -keyint_min 30 -profile:v baseline -level 3.0 -max_muxing_queue_size 1024 -f rtsp rtsp://localhost:$RTSP_PORT/ffmpeg/$RTSP_PATH

    runOnReadyRestart: yes

2023/06/18 11:35:29 INF [RTSP] [session 78d59090] created by 127.0.0.1:39004
2023/06/18 11:35:29 INF [path ffmpeg/ffmpeg/ffmpeg/ffmpeg/ffmpeg/ffmpeg/ffmpeg/ffmpeg/ffmpeg/ffmpeg/ffmpeg/ffmpeg/ffmpeg/ffmpeg/ffmpeg/ffmpeg/murat] runOnReady command started
2023/06/18 11:35:29 INF [RTSP] [session 78d59090] is publishing to path 'ffmpeg/ffmpeg/ffmpeg/ffmpeg/ffmpeg/ffmpeg/ffmpeg/ffmpeg/ffmpeg/ffmpeg/ffmpeg/ffmpeg/ffmpeg/ffmpeg/ffmpeg/ffmpeg/murat', with UDP, 2 tracks (H264, MPEG4-audio-gen)
2023/06/18 11:35:29 INF [path ffmpeg/ffmpeg/ffmpeg/ffmpeg/ffmpeg/ffmpeg/ffmpeg/ffmpeg/ffmpeg/ffmpeg/ffmpeg/ffmpeg/ffmpeg/ffmpeg/ffmpeg/ffmpeg/murat] runOnReady command stopped
2023/06/18 11:35:29 INF [RTSP] [session 78d59090] destroyed (torn down by 127.0.0.1:39004)
2023/06/18 11:35:29 INF [RTSP] [conn 127.0.0.1:39004] closed (EOF)
2023/06/18 11:35:29 INF [RTSP] [conn 127.0.0.1:34124] closed (terminated)
2023/06/18 11:35:29 INF [path ffmpeg/ffmpeg/ffmpeg/ffmpeg/ffmpeg/ffmpeg/murat] runOnReady command stopped
2023/06/18 11:35:29 INF [RTSP] [session ab0587cd] destroyed (session timed out)
2023/06/18 11:35:29 INF [RTSP] [conn 127.0.0.1:34128] closed (terminated)
2023/06/18 11:35:29 INF [RTSP] [session 105bcffe] destroyed (terminated)
av_interleaved_write_frame(): Broken pipe
Error writing trailer of rtsp://localhost:8554/ffmpeg/ffmpeg/ffmpeg/ffmpeg/ffmpeg/ffmpeg/ffmpeg/murat: Broken pipe
2023/06/18 11:35:29 INF [RTSP] [conn 127.0.0.1:34112] closed (terminated)
2023/06/18 11:35:29 INF [path ffmpeg/ffmpeg/ffmpeg/ffmpeg/ffmpeg/murat] runOnReady command stopped
2023/06/18 11:35:29 INF [RTSP] [session c79a5ec5] destroyed (session timed out)
2023/06/18 11:35:29 INF [RTSP] [conn 127.0.0.1:34114] closed (terminated)
2023/06/18 11:35:29 INF [RTSP] [session 7645cd5f] destroyed (terminated)
av_interleaved_write_frame(): Broken pipe
Error writing trailer of rtsp://localhost:8554/ffmpeg/ffmpeg/ffmpeg/ffmpeg/ffmpeg/ffmpeg/murat: Broken pipe
av_interleaved_write_frame(): Broken pipe

[fastfading]

I have read the readme of transcoding.

but multi-resolution llhls needs more than that like multi-variant m3u8 synced multi-resolution fmp4 or ts .

[fastfading]

@mpisat how to do transcoding with 1 rtmp input, and transcode it to 360p hls output

[mpisat]

I solved that problem by using another application like runOnDemand: node pull.js $RTSP_PATH $RTSP_PORT where my pull.js code launches ffmpeg, and receives $RTSP_PATH $RTSP_PORT arguments

Here is the full code if you want to play with it. You should modify it to pull video from mediamtx, and transcode, and then push to mediamtx via rtsp with stream_720p or whatever you want. Having said that, now I realized my mistake, as runonready runs in a loop and it generates these errors. I should check incoming stream or separate paths to not transcode every incoming stream again (infinite loop)

const axios = require('axios');
const { exec } = require('child_process');

const pullStream = async (RTSP_PATH, RTSP_PORT) => {
    const lowerRTSP_PATH = RTSP_PATH.toLowerCase();

    // Check if the stream already exists
    const pathsResponse = await axios.get('http://127.0.0.1:9997/v2/paths/list');
    const streams = pathsResponse.data.items;
    const streamExists = streams.some(item => item.name === lowerRTSP_PATH && item.source && item.sourceReady === true && item.bytesReceived > 0);

    if (streamExists) {
        console.log('Stream already exists');
        return;
    }

    console.log(`RTSP_PATH: ${RTSP_PATH}`);
    console.log(`RTSP_PORT: ${RTSP_PORT}`);

    // Make a connection to the API
    const response = await axios.get(`https://my-remote-url${RTSP_PATH}`);
    const data = response.data;

    console.log(data);

    // Check if the stream is online
    if (data.online !== '1' && data.online !== '6') {
        console.log('Stream is offline');
        return;
    }

    const url = `rtmp://--this-is-for-my-custom-business-logic`;

    console.log(`RTMP URL: ${url}`);

    const ffmpegCommand = `/usr/bin/nohup /usr/local/bin/ffmpeg -hide_banner -loglevel info -fflags nobuffer -flags low_delay -i "${url}" -c:v copy -c:a libopus -b:a 96000 -ar 48000 -f rtsp rtsp://localhost:${RTSP_PORT}/${RTSP_PATH}`;

    exec(ffmpegCommand, (err, stdout, stderr) => {
        if (err) {
            console.log(`error: ${err.message}`);
            return;
        }
        if (stderr) {
            console.log(`stderr: ${stderr}`);
            return;
        }
        console.log(`stdout: ${stdout}`);
    });
}

const RTSP_PATH = process.argv[2];
const RTSP_PORT = process.argv[3];
pullStream(RTSP_PATH, RTSP_PORT);

[fastfading]

@mpisat thank you very much ,
but I did not get this example work https://github.com/bluenviron/mediamtx#remuxing-re-encoding-compression this session did not explain how to use ffmpeg scale 1 rtmp stream into another hls

here is my paths configuration

paths:
  all:
  original:
    runOnReady: ffmpeg -loglevel error -i rtsp://localhost:$RTSP_PORT/$RTSP_PATH -tune zerolatency -g 150 -vf scale=640:360 -f rtsp rtsp://localhost:$RTSP_PORT/360/$RTSP_PATH
    runOnReadyRestart: yes

what is the rtmp url should be ? rtmp://x.x.x.x:1935/original/abc ? and what should the playback url be ? http://x.x.x.x:8888/compressed/abc ?

I have trouble to make it work

[mpisat]

if you use runonready it may cause another loop I think? I have to test it again and I will let you know.

[mpisat]

after checking everything, runondemand works for me, but my use case is different. I make a request to non existent stream

under paths: all: runOnDemand: node pull.js $RTSP_PATH $RTSP_PORT

this pulls the video, and pushes to the same stream name that I'm connected as a viewer localhost:8889/streamname

I believe this one caused the issues for me

runOnReady: ffmpeg -loglevel error -i rtsp://localhost:$RTSP_PORT/$RTSP_PATH -c:v libx264 -tune zerolatency -preset faster -b:v 2500k -bsf:v h264_mp4toannexb -g 30 -keyint_min 30 -profile:v baseline -level 3.0 -max_muxing_queue_size 1024 -f rtsp rtsp://localhost:$RTSP_PORT/ffmpeg/$RTSP_PATH

it was causing the infinite loop.

if you want to use runondemand method like me, use ffmpeg command similar to one above and pull video from origin/streamname perhaps, assuming you push them to mediamtx/origin/streamname and add runondemand for another path, where it can pull raw video, transcode, push to the non-existent stream you are trying to pull.

this should work with pre-defined streamnames, I'm not sure if we can catch all all streams under /origin or /live and apply different rules at this point.

[mpisat]

ok so I used this config:

paths:

"~^live/": runOnReady: ffmpeg -loglevel error -i rtsp://localhost:$RTSP_PORT/$RTSP_PATH -c:v libx264 -tune zerolatency -preset faster -b:v 2500k -bsf:v h264_mp4toannexb -g 30 -keyint_min 30 -profile:v baseline -level 3.0 -max_muxing_queue_size 1024 -f rtsp rtsp://localhost:$RTSP_PORT/ffmpeg/$RTSP_PATH

all:

and then I pushed a video

ffmpeg -re -i rtmp://localhost:19350/stream -c copy -f flv rtmp://localhost:19350/live/stream

mediamtx logs:

2023/06/29 06:05:58 INF [RTMP] [conn 127.0.0.1:57542] opened
2023/06/29 06:05:59 INF [RTSP] [conn 127.0.0.1:37580] opened
2023/06/29 06:05:59 INF [RTSP] [session 75d04b3d] created by 127.0.0.1:37580
2023/06/29 06:05:59 INF [RTSP] [session 75d04b3d] is publishing to path 'stream', with UDP, 2 tracks (H264, Opus)
2023/06/29 06:05:59 INF [RTMP] [conn 127.0.0.1:57542] is reading from path 'stream', 1 track (H264)
2023/06/29 06:06:00 INF [WebRTC] [session 5ea150a5] peer connection established, local candidate: host/udp/178.170.46.28/43488, remote candidate: prflx/udp/x.x.x.x/52542
2023/06/29 06:06:00 INF [WebRTC] [session 5ea150a5] is reading from path 'stream', 2 tracks (H264, Opus)
2023/06/29 06:06:03 INF [RTMP] [conn 127.0.0.1:57550] opened
2023/06/29 06:06:03 INF [path live/isabellaeva] runOnReady command started
2023/06/29 06:06:03 INF [RTMP] [conn 127.0.0.1:57550] is publishing to path 'live/stream', 1 track (H264)
2023/06/29 06:06:03 INF [RTSP] [conn 127.0.0.1:37584] opened
2023/06/29 06:06:03 INF [RTSP] [session 53469bef] created by 127.0.0.1:37584
2023/06/29 06:06:03 INF [RTSP] [session 53469bef] is reading from path 'live/stream', with UDP, 1 track (H264)
2023/06/29 06:06:05 INF [RTSP] [conn 127.0.0.1:32880] opened
2023/06/29 06:06:05 INF [RTSP] [session 730c181f] created by 127.0.0.1:32880
2023/06/29 06:06:05 INF [RTSP] [session 730c181f] is publishing to path 'ffmpeg/live/stream', with UDP, 1 track (H264)
2023/06/29 06:06:15 INF [RTSP] [conn 127.0.0.1:32880] closed (terminated)
2023/06/29 06:06:15 INF [RTSP] [session 730c181f] destroyed (terminated)
av_interleaved_write_frame(): Broken pipe
2023/06/29 06:06:15 INF [RTSP] [session 53469bef] destroyed (torn down by 127.0.0.1:37584)
2023/06/29 06:06:15 INF [RTSP] [conn 127.0.0.1:37584] closed (EOF)
2023/06/29 06:06:15 INF [path live/isabellaeva] runOnReady command exited with code 1

I tested with rtsp and rtmp, both same. ffmpeg version 4.4-static https://johnvansickle.com/ffmpeg/ Copyright (c) 2000-2021 the FFmpeg developers

[mpisat]

paths:

"~^live/": runOnReady: ffmpeg -loglevel error -i rtmp://localhost:19350/$RTSP_PATH -c:v libx264 -tune zerolatency -preset faster -b:v 2500k -bsf:v h264_mp4toannexb -g 30 -keyint_min 30 -profile:v baseline -level 3.0 -max_muxing_queue_size 1024 -f flv rtmp://localhost:19350/ffmpeg/$RTSP_PATH

also produces same error

[irg1008]

Do you have an example with HLS, not RSTP??