search

shaka-project / shaka-streamer

A simple config-file based approach to preparing streaming media, based on FFmpeg and Shaka Packager.
https://shaka-project.github.io/shaka-streamer/
Apache License 2.0
198 stars 62 forks source link

DTS out of order with RTMP #42

Closed thellaindudhar closed 4 years ago

thellaindudhar commented 4 years ago

Shaka streamer with rtmp url :-

Stream #0:1: Video: h264 (High), yuv420p(progressive), 1280x720 [SAR 1:1 DAR 16:9], 3999 kb/s, 30 fps, 30 tbr, 1k tbn, 60 tbc

ffmpeg version 4.1 Copyright (c) 2000-2018 the FFmpeg developers
  built with gcc 5.5.0 (Ubuntu 5.5.0-12ubuntu1~16.04) 20171010
  configuration: --enable-gpl --enable-nonfree --enable-libx264 --enable-libmp3lame --enable-libfdk-aac --enable-libxml2 --enable-ffplay
  libavutil      56. 22.100 / 56. 22.100
  libavcodec     58. 35.100 / 58. 35.100
  libavformat    58. 20.100 / 58. 20.100
  libavdevice    58.  5.100 / 58.  5.100
  libavfilter     7. 40.101 /  7. 40.101
  libswscale      5.  3.100 /  5.  3.100
  libswresample   3.  3.100 /  3.  3.100
  libpostproc    55.  3.100 / 55.  3.100
[1105/181423:INFO:demuxer.cc(88)] Demuxer::Run() on file '/tmp/shaka-live-l9xznts7/28b66dc7-8a57-45ec-8498-3d0c5ab7bb06'.
[1105/181423:INFO:demuxer.cc(160)] Initialize Demuxer for file '/tmp/shaka-live-l9xznts7/28b66dc7-8a57-45ec-8498-3d0c5ab7bb06'.
[1105/181423:INFO:demuxer.cc(88)] Demuxer::Run() on file '/tmp/shaka-live-l9xznts7/bee4f5c6-c812-4d02-aa19-053c8b9dbcc8'.
[1105/181423:INFO:demuxer.cc(160)] Initialize Demuxer for file '/tmp/shaka-live-l9xznts7/bee4f5c6-c812-4d02-aa19-053c8b9dbcc8'.
Input #0, mpegts, from '/tmp/shaka-live-l9xznts7/c981326e-352a-4fff-9699-1ca7683c2eba':
  Duration: N/A, start: 1.400000, bitrate: N/A
  Program 1 
    Metadata:
      service_name    : Service01
      service_provider: FFmpeg
    Stream #0:0[0x100]: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p(progressive), 1280x720 [SAR 1:1 DAR 16:9], 30 fps, 30 tbr, 90k tbn, 60 tbc
    Stream #0:1[0x101]: Audio: aac (LC) ([15][0][0][0] / 0x000F), 44100 Hz, stereo, fltp, 117 kb/s
Input google/shaka-player#1, mpegts, from '/tmp/shaka-live-l9xznts7/b5a06f54-b5e4-4299-96c0-46f4ec9c39d6':
  Duration: N/A, start: 1.400000, bitrate: N/A
  Program 1 
    Metadata:
      service_name    : Service01
      service_provider: FFmpeg
    Stream google/shaka-player#1:0[0x100]: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p(progressive), 1280x720 [SAR 1:1 DAR 16:9], 30 fps, 30 tbr, 90k tbn, 60 tbc
    Stream google/shaka-player#1:1[0x101]: Audio: aac (LC) ([15][0][0][0] / 0x000F), 44100 Hz, stereo, fltp, 117 kb/s
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> h264 (libx264))
  Stream google/shaka-player#1:1 -> google/shaka-player#1:0 (aac (native) -> aac (native))
Press [q] to stop, [?] for help
[mpegts @ 0x21a5900] Thread message queue blocking; consider raising the thread_queue_size option (current value: 100)
[libx264 @ 0x21a7140] using SAR=1280/1281
[libx264 @ 0x21a7140] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
[libx264 @ 0x21a7140] profile Constrained Baseline, level 3.1
Output #0, mpegts, to '/tmp/shaka-live-l9xznts7/28b66dc7-8a57-45ec-8498-3d0c5ab7bb06':
  Metadata:
    encoder         : Lavf58.20.100
    Stream #0:0: Video: h264 (libx264), yuv420p, 854x480 [SAR 1280:1281 DAR 16:9], q=-1--1, 2000 kb/s, 30 fps, 90k tbn, 30 tbc
    Metadata:
      encoder         : Lavc58.35.100 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/2000000 buffer size: 0 vbv_delay: -1
[mpegts @ 0x225e1c0] Thread message queue blocking; consider raising the thread_queue_size option (current value: 100)
[mpegts @ 0x21dd500] frame size not set
Output google/shaka-player#1, mpegts, to '/tmp/shaka-live-l9xznts7/bee4f5c6-c812-4d02-aa19-053c8b9dbcc8':
  Metadata:
    encoder         : Lavf58.20.100
    Stream google/shaka-player#1:0: Audio: aac (LC), 44100 Hz, stereo, fltp, 64 kb/s
    Metadata:
      encoder         : Lavc58.35. libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/2000000 buffer size: 0 vbv_delay: -1
[mpegts @ 0x225e1c0] Thread message queue blocking; consider raising the thread_queue_size option (current value: 100)
[mpegts @ 0x21dd500] frame size not set
Output google/shaka-player#1, mpegts, to '/tmp/shaka-live-l9xznts7/bee4f5c6-c812-4d02-aa19-053c8b9dbcc8':
  Metadata:
    encoder         : Lavf58.20.100
    Stream google/shaka-player#1:0: Audio: aac (LC), 44100 Hz, stereo, fltp, 64 kb/s
    Metadata:
      encoder         : Lavc58.35.100 aac
[1105/181435:WARNING:es_parser_h26x.cc(334)] [MPEG-2 TS] PID 256 Possible GAP at dts 126000 with next sample at dts 129000 (difference 3000)
[1105/181437:WARNING:representation.cc(372)] Representation (id=1,codec='mp4a.40.2',frequency=44100,language='') Segments should not be overlapping. The new segment starts at 180312 but the previous segment ends at 180333.
[1105/181445:WARNING:representation.cc(362)] Representation (id=1,codec='mp4a.40.2',frequency=44100,language='') Found a gap of size 7 > kRoundingErrorGrace (5). The new segment starts at 1081008 but the previous segment ends at 1081001.
[1105/181512:WARNING:representation.cc(362)] Representation (id=1,codec='mp4a.40.2',frequency=44100,language='') Found a gap of size 6 > kRoundingErrorGrace (5). The new segment starts at 3601302 but the previous segment ends at 3601296.
[1105/181540:WARNING:representation.cc(362)] Representation (id=1,codec='mp4a.40.2',frequency=44100,language='') Found a gap of size 6 > kRoundingErrorGrace (5). The new segment starts at 5941873 but the previous segment ends at 5941867.
[1105/181609:WARNING:representation.cc(362)] Representation (id=1,codec='mp4a.40.2',frequency=44100,language='') Found a gap of size 7 > kRoundingErrorGrace (5). The new segment starts at 8280355 but the previous segment ends at 8280348.
[1105/181630:WARNING:representation.cc(362)] Representation (id=1,codec='mp4a.40.2',frequency=44100,language='') Found a gap of size 6 > kRoundingErrorGrace (5). The new segment starts at 10441203 but the previous segment ends at 10441197.
[1105/181644:WARNING:representation.cc(362)] Representation (id=1,codec='mp4a.40.2',frequency=44100,language='') Found a gap of size 6 > kRoundingErrorGrace (5). The new segment starts at 11521628 but the previous segment ends at 11521622.
[1105/181651:WARNING:representation.cc(362)] Representation (id=1,codec='mp4a.40.2',frequency=44100,language='') Found a gap of size 6 > kRoundingErrorGrace (5). The new segment starts at 12060795 but the previous segment ends at 12060789.

While playing the playback url:-

[dash @ 0x7ffaf0000940] DTS 4626000 < 4983000 out of order]
repeating the same segment util the segment disappears on the folder, later for every 2 seconds the video stops for 5 seconds up to last segment.

If i increase the seconds in time_Shift_Buffer_Depth then it will play the segments for that particular time,later same issue repeat.

Can any one guide me how to resolve the issue,what are the changes i need to do in the shaka-streamer

joeyparrish commented 4 years ago

We don't yet support RTMP in Shaka Streamer (see google/shaka-streamer#16), and I'm not sure we could reproduce this issue without access to your private stream. (I noticed a private IP range on the URL.)

It's possible that the "DTS out of order" error is a problem in the input. Where does this particular RTMP stream come from? What generates it?

Also, I'm going to transfer this issue to the streamer repo.

thellaindudhar commented 4 years ago

we are generating rtmp through node rtmp

joeyparrish commented 4 years ago

Can you please give us some way to reproduce your results? Some details on how you are generating the RTMP stream?

thellaindudhar commented 4 years ago

Thanks for the response. By using node-media-server (node.js) and ffmpeg converting the mp4 video file with (h264 video and audio aac codecs) to rtmp url.

node app.js ffmpeg -re -i sample.mp4 -c copy -f flv rtmp://localhost/live/commet

Using the rtmp url in shaka streamer

joeyparrish commented 4 years ago

Can you provide us with a more complete set of instructions to reproduce the issue? Please assume we know nothing about node-media-server, and please provide any source material you're starting with.

thellaindudhar commented 4 years ago

1)Creating a new folder nms , then i perform the steps as it is which was present in below link. ->This is the link where i follow the steps to generate the rtmp url with node.js:- click:- https://www.npmjs.com/package/node-media-server

2)Git version:-

mkdir nms
cd nms
git clone https://github.com/illuspas/Node-Media-Server
npm i
node app.js

npm version (recommended):-

mkdir nms
cd nms
npm install node-media-server

->create a new file app.js and open the file ,copy the below content to the app.js file:-

vi app.js
const NodeMediaServer = require('node-media-server');

const config = {
  rtmp: {
    port: 1935,
    chunk_size: 60000,
    gop_cache: true,
    ping: 30,
    ping_timeout: 60
  },
  http: {
    port: 8000,
    allow_origin: '*'
  }
};

var nms = new NodeMediaServer(config)
nms.run();

Then run the node app.js in one terminal and ffmpeg in another terminal

node app.js
ffmpeg -re -i sample.mp4 -c copy -f flv rtmp://localhost/live/commet

Like this way i am generating the rtmp url. Then inserting this rtmp url to "input_looped_file_config.yaml" for both audio and video. Make changes in the pipeline_live_config.yaml according to us

->vi pipeline_live_config.yaml 

streaming_mode: live
transcoder:
  # A list of resolutions to encode.
  resolutions:
   # - 720p
    - 480p
    #- 360p
    #- 240p
    #- 144p
  # The number of audio channels to output.
  channels: 2
  # The codecs to encode with.
  audio_codecs:
    - aac
  video_codecs:
    - h264
packager:
  # Manifest format (dash, hls, or both)
  manifest_format:
    - dash
  #  - hls
  # Length of each segment in seconds.
  segment_size: 2
  # Availability window, or the number of seconds a segment remains available.
  availability_window: 200
  # Presentation delay, or how far back from the edge the player should be.
  presentation_delay: 50
  # Update period, or how often the player should fetch a new manifest.
  update_period: 4

After that the below command line executed in that terminal

python3 shaka-streamer -i config_files/input_looped_file_config.yaml  -p config_files/pipeline_live_config.yaml  -o /var/www/html/dash/

All these operation is done in the ("UBUNTU LTS 16.04 OS")

joeyparrish commented 4 years ago

Thank you for the instructions. You still have not provided your sample.mp4 file, which could be key to reproducing your issue.

In particular, your command ffmpeg -re -i sample.mp4 -c copy -f flv rtmp://localhost/live/commet copies the input to FLV, which requires certain audio sample rates (44100, 22050, or 11025) and certain codecs (aac, for example). So not just any MP4 will do.

You also left out the contents of your input config file.

I've tried to reproduce your issue by recreating the missing pieces.

I created my own sample.mp4 from known-good content with:

ffmpeg \
    -t 60 -i Sintel.2010.720p.mkv \
    -ac 2 -ar 44100 \
    -c:v h264 -b:v 4M \
    -c:a aac -b:a 128k \
    sample.mp4

This is 60 seconds of Sintel, in stereo, 44.1kHz AAC, 720p h264, 4Mbps.

Your pipeline config is in an older format, so I'm guessing you're using v0.1 of the application. Please update to a newer build, either from source or with pip3 install --upgrade shaka-streamer.

In spite of the fact that we don't officially support RTMP yet (see #16), I've gotten this working without changing any of the Streamer code. I think I see the issue, and it's with your ffmpeg commands.

You're taking your sample.mp4 and sending it to RTMP, then asking Streamer to create a loop for you. For a networked input like this, you need to have the RTMP input be unending instead. So you need to loop that mp4 in your ffmpeg command.

ffmpeg -stream_loop -1 -re -i sample.mp4 -c copy -f flv rtmp://localhost/live/commet

Then, your pipeline config (updated to the current format) would be:

streaming_mode: live
resolutions:
  - 480p
audio_codecs:
  - aac
video_codecs:
  - h264
manifest_format:
  - dash
segment_size: 2
availability_window: 200
presentation_delay: 50
update_period: 4

And your input config would be (note the lack of input_type: looped_file):

inputs:
  - name: rtmp://localhost/live/commet
    media_type: video

  - name: rtmp://localhost/live/commet
    media_type: audio

This works for me, and plays fine in the latest version of Shaka Player.

Does this help?

thellaindudhar commented 4 years ago

Details of sample.mp4: $ mediainfo BangladeshInnings2nd.mp4

General Complete name : BangladeshInnings2nd.mp4 Format : MPEG-4 Format profile : Adobe Flash Codec ID : f4v (isom/mp42/m4v ) File size : 690 MiB Duration : 23mn 20s Overall bit rate mode : Variable Overall bit rate : 4 134 Kbps Encoded date : UTC 2019-05-17 17:55:14 Tagged date : UTC 2019-05-17 17:55:14

Video ID : 1 Format : AVC Format/Info : Advanced Video Codec Format profile : High@L3.1 Format settings, CABAC : Yes Format settings, ReFrames : 1 frame Format settings, GOP : M=1, N=60 Codec ID : avc1 Codec ID/Info : Advanced Video Coding Duration : 23mn 20s Bit rate mode : Constant Bit rate : 4 000 Kbps Width : 1 280 pixels Height : 720 pixels Display aspect ratio : 16:9 Frame rate mode : Variable Frame rate : 30.000 fps Minimum frame rate : 1.751 fps Maximum frame rate : 30.303 fps Color space : YUV Chroma subsampling : 4:2:0 Bit depth : 8 bits Scan type : Progressive Bits/(Pixel*Frame) : 0.145 Stream size : 668 MiB (97%) Title : WowzaStreamingEngine Language : English Encoded date : UTC 2019-05-17 17:55:14 Tagged date : UTC 2019-05-17 17:55:14

Audio ID : 2 Format : AAC Format/Info : Advanced Audio Codec Format profile : LC Codec ID : 40 Duration : 23mn 20s Bit rate mode : Variable Bit rate : 129 Kbps Channel(s) : 2 channels Channel positions : Front: L R Sampling rate : 44.1 KHz Frame rate : 43.066 fps (1024 spf) Compression mode : Lossy Stream size : 21.5 MiB (3%) Title : WowzaStreamingEngine Language : English Encoded date : UTC 2019-05-17 17:55:14 Tagged date

thellaindudhar commented 4 years ago

Thanks for your reply, but can i get some more detail information regarding of the issue and its solution

thellaindudhar commented 4 years ago

May i know what are the contents i left in the input_config files. If you don't mind can you please explain me in step by step method how you are approaching and what are the changes you made for resolving the issue and what are the changes i need to do to resolve the issue on my side.

thellaindudhar commented 4 years ago

shaka-streamer Here i am using the upgrade version of shaka-streamer but it is not taking the version of shaka-packager shaka-version can you help me how to resolve the issue

joeyparrish commented 4 years ago

We require Shaka Packager v2.1 or above. If you have a pre-release version built from source, you'll need to:

1) Verify that it is newer than the v2.1 tag 2) Get the latest Shaka Streamer source code from github 3) Run it with the option --skip_deps_check to bypass the version number check on Shaka Packager

The --skip_deps_check option will appear in Streamer v0.4, which has yet to be released on PyPI.

After that, you should use the commands and config files I showed you in my earlier comment. (ffmpeg -stream_loop -1 when streaming to the RTMP encoder, and use the input config I showed you without looped_file.)

If that doesn't work for you, you'll need to send us the actual MP4 source file you're using, not just the output of mediainfo on it. I can't reproduce a specific failure with mediainfo output alone.

Does this help?

thellaindudhar commented 4 years ago

While playing encrypted live+vod content in shakaplayer its not playing and throwing the issue :- Shaka Error MANIFEST.RESTRICTIONS_CANNOT_BE_MET ([object Object])

thellaindudhar commented 4 years ago

1 2 3

joeyparrish commented 4 years ago

I'm unable to reproduce your results. Please wait for the 0.4 release with documented RTMP support.

shaka-bot commented 4 years ago

@thellaindudhar Does this answer all your questions? Can we close the issue?

thellaindudhar commented 4 years ago

yes you can close the issue

joeyparrish commented 4 years ago

Thanks! Let us know if we can do anything else for you.