search

kaltura / nginx-vod-module

NGINX-based MP4 Repackager
GNU Affero General Public License v3.0
1.99k stars 439 forks source link

HLS/DASH with seperate audio and video files #1397

Closed sridhard closed 1 year ago

sridhard commented 1 year ago

Hi,

I am trying to setup kaltura nginx-vod-module. I have few queries for nginx-vod-module:

We have transcoded our files to multiple resolutions(720p/480p/360p). We have audio and video in different files. This means we have the following files: Video_720p.mp4, Video_480p.mp4, Video_360p.mp4, Audio.mp4. We want to setup adaptive bitrate with video track selection with these files.

I did not find any example for this setuo. can you please tell me how can we use audio and video stream from different files

I tried the below mapping file: { "sequences": [ { "clips": [ { "type": "source", "path": "/home/ubuntu/videos/a.mp4" } ] },
{ "clips": [ { "type": "source", "path": "/home/ubuntu/videos/hq.mp4" } ] }, { "clips": [ { "type": "source", "path": "/home/ubuntu/videos/mq.mp4" } ] } ] }

a.mp4 is audio stream. hq.mp4 and mq.mp4 are video streams. I am getting the below error

2022/10/29 08:19:09 [error] 21727#0: *20 ngx_http_vod_validate_streams: no matching streams were found, probably invalid segment index while getting mapping, client: 49.37.156.47, server: vod-server, request: "GET /dash/test.json/fragment-1-f1-v1-x3.m4s HTTP/1.1", host: "vodserver.learnyst.com", referrer: "https://shaka-player-demo.appspot.com/"

Thanks

erankor commented 1 year ago

The JSON looks wrong, it will play the first file, then the second one etc. you need to use multiple sequences instead, like this sample - https://github.com/kaltura/nginx-vod-module/#adaptive-set. Clips play one after the other, while sequences play at the same time. I would have expected a different error though - you did not provide the clip durations. You are probably testing it wrong due to caching, as I wrote on the other issue...

sridhard commented 1 year ago

@erankor sorry for the confusion. The JSON is not formatted properly. JSON is same as adaptive-set example. Below is the formated json

{
    "sequences": [
        {
            "clips": [{
                "type": "source",
                "path": "/home/ubuntu/videos/a.mp4"
            }]
        },
        {
            "clips": [{
                "type": "source",
                "path": "/home/ubuntu/videos/hq.mp4"
            }]
        },
        {
            "clips": [{
                "type": "source",
                "path": "/home/ubuntu/videos/mq.mp4"
            }]
        }
    ]
}

Here a.mp4 is audio file. hq.mp4 and mq.mp4 are video tracks with different resolutions. This is not working.

Consider the below JSON:

{
    "sequences": [
        {
            "clips": [{
                "type": "source",
                "path": "/home/ubuntu/videos/hq_a.mp4"
            }]
        },
        {
            "clips": [{
                "type": "source",
                "path": "/home/ubuntu/videos/mq.mp4"
            }]
        },
        {
            "clips": [{
                "type": "source",
                "path": "/home/ubuntu/videos/lq.mp4"
            }]
        }
    ]
}

here hq_a.mp4 has both video and audio track, mq.mp4 and lq.mp4 are only video tracks. This is working. I am able to do track selection and also audio is playing.

erankor commented 1 year ago

If the problem is only with HLS, then it makes sense, because with this JSON the returned m3u8 will have 1 EXT-X-STREAM-INF which is audio only, and 2 EXT-X-STREAM-INFs which are video only. The player will choose only one of them, so you should get either video-only or audio-only. I'd expect that DASH would play fine.

There are two ways to solve it -

  1. Add vod_hls_force_unmuxed_segments on - if you do that, the audio will be returned in an EXT-X-MEDIA tag, and the player will mux it with the video.
  2. You can use mixFilter to mux the audio + video on the server side, the JSON would look something like this - 
    {
"sequences": [
    {
        "clips": [
            {
                "type": "mixFilter",
                "sources": [
                    {
                        "type": "source",
                        "path": "/home/ubuntu/videos/a.mp4"
                    },
                    {
                        "type": "source",
                        "path": "/home/ubuntu/videos/mq.mp4"
                    }
                ]
            }
        ]
    },
    {
        "clips": [
            {
                "type": "mixFilter",
                "sources": [
                    {
                        "type": "source",
                        "path": "/home/ubuntu/videos/a.mp4"
                    },
                    {
                        "type": "source",
                        "path": "/home/ubuntu/videos/hq.mp4"
                    }
                ]
            }
        ]
    }       
]
}
sridhard commented 1 year ago

@erankor below is the result: (I have tested with new json and also restarted the server)

Test Case-1: hq.mp4, mq.mp4 are video files and a.mp4 is audio file

{
  "sequences": [
  {
    "clips": [
        {
          "type": "source",
          "path": "/home/ubuntu/videos/hq.mp4"
        }
    ]
  },
  {
    "clips": [
        {
          "type": "source",
          "path": "/home/ubuntu/videos/mq.mp4"
        }
    ]
  },
  {
    "clips": [
        {
          "type": "source",
          "path": "/home/ubuntu/videos/a.mp4"
        }
    ]
  }  
  ]
}

RESULT : Both DASH and HLS not working Dash Error: 2022/11/01 02:22:24 [error] 1680#0: 4 ngx_http_vod_validate_streams: no matching streams were found while reading media header, client: 49.37.156.47, server: vod-server, request: "GET /dash/test4.json/manifest.mpd HTTP/1.1", host: "vodserver.learnyst.com", referrer: "https://shaka-player-demo.appspot.com/" 2022/11/01 02:22:24 [error] 1680#0: 5 ngx_http_vod_validate_streams: no matching streams were found while getting mapping, client: 49.37.156.47, server: vod-server, request: "GET /dash/test4.json/manifest.mpd HTTP/1.1", host: "vodserver.learnyst.com", referrer: "https://shaka-player-demo.appspot.com/" HLS Error 2022/11/01 02:28:07 [error] 2332#0: 1 ngx_http_vod_validate_streams: no matching streams were found while reading media header, client: 49.37.156.47, server: vod-server, request: "GET /hls/test4.json/master.m3u8 HTTP/1.1", host: "vodserver.learnyst.com", referrer: "https://shaka-player-demo.appspot.com/" 2022/11/01 02:28:07 [error] 2332#0: 2 ngx_http_vod_validate_streams: no matching streams were found while getting mapping, client: 49.37.156.47, server: vod-server, request: "GET /hls/test4.json/master.m3u8 HTTP/1.1", host: "vodserver.learnyst.com" 2022/11/01 02:28:09 [error] 2332#0: *3 ngx_http_vod_validate_streams: no matching streams were found while getting mapping, client: 49.37.156.47, server: vod-server, request: "GET /hls/test4.json/master.m3u8 HTTP/1.1", host: "vodserver.learnyst.com", referrer: "https://shaka-player-demo.appspot.com/" Dash Debug Logs Link: https://coursecopy.s3.ap-south-1.amazonaws.com/logs/kaltura_nginx_debug.log

Test Case-2: hq.mp4, mq.mp4 are video files and a.mp4 is audio file

{
    "sequences": [
        {
            "clips": [
                {
                    "type": "mixFilter",
                    "sources": [
                        {
                            "type": "source",
                            "path": "/home/ubuntu/videos/a.mp4"
                        },
                        {
                            "type": "source",
                            "path": "/home/ubuntu/videos/mq.mp4"
                        }
                    ]
                }
            ]
        },
        {
            "clips": [
                {
                    "type": "mixFilter",
                    "sources": [
                        {
                            "type": "source",
                            "path": "/home/ubuntu/videos/a.mp4"
                        },
                        {
                            "type": "source",
                            "path": "/home/ubuntu/videos/hq.mp4"
                        }
                    ]
                }
            ]
        }       
    ]
}

RESULT : Failed with same error 2022/11/01 02:49:16 [error] 3488#0: 1 ngx_http_vod_validate_streams: no matching streams were found while reading media header, client: 49.37.156.47, server: vod-server, request: "GET /dash/test6.json/manifest.mpd HTTP/1.1", host: "vodserver.learnyst.com", referrer: "https://shaka-player-demo.appspot.com/" 2022/11/01 02:49:16 [error] 3488#0: 2 ngx_http_vod_validate_streams: no matching streams were found while getting mapping, client: 49.37.156.47, server: vod-server, request: "GET /dash/test6.json/manifest.mpd HTTP/1.1", host: "vodserver.learnyst.com", referrer: "https://shaka-player-demo.appspot.com/"

Debug Logs for DASH: https://coursecopy.s3.ap-south-1.amazonaws.com/logs/kaltura_nginx_debug-1.log

Note

Below JSON is working: hq_a.mp4 has both audio and video tracks. mq.mp4 has only video trach. I merged audio track to hq.mp4 using ffmpeg

{
  "sequences": [
  {
    "clips": [
        {
          "type": "source",
          "path": "/home/ubuntu/videos/hq_a.mp4"
        }
    ]
  },
  {
    "clips": [
        {
          "type": "source",
          "path": "/home/ubuntu/videos/mq.mp4"
        }
    ]
  }
  ]
}

a.mp4 link: https://coursecopy.s3.ap-south-1.amazonaws.com/logs/a.mp4 hq.mp4 link: https://coursecopy.s3.ap-south-1.amazonaws.com/logs/hq.mp4 mq.mp4 link: https://coursecopy.s3.ap-south-1.amazonaws.com/logs/mq.mp4

erankor commented 1 year ago

The problem is not the JSON, the problem is that all these files are fragmented MP4s - fragmented MP4 is not supported. You'd need to repackage them to regular MP4, in order to use them with this module, this can be done using ffmpeg - e.g. ffmpeg -i src.mp4 -c:v copy -c:a copy out.mp4.

sridhard commented 1 year ago

Thanks. It is working