Open Zenexer opened 6 years ago
It seems all the data is downloaded, but the duration is wrong. Workaround: run ffmpeg -i <input.mp4> -c copy <output.mp4>
Edit: Okay, that doesn't work, but the data is definitely there.
Try using this flag: --hls-prefer-native Just downloaded it with that and it worked. Don't know why it doesn't work with FFMPEG though.
@HyperPlaneSubSpacePilot That log I provided is with --hls-prefer-native
. It doesn't fix the issue.
It seems that no matter what I do, the wrong time is recorded. The full video will still be there, but nothing can play past the first ad break.
Using 2018.04.25, I downloaded this TBS video using the --hls-prefer-native
flag and it worked; MPC shows the episode as 23 minutes and it plays through fine. I encounter the same errors without the flag though.
I am running into this issue downloading from Cartoon Network, another Tuner site. It only downloads the first 11 min until the first commercial break, but the file is 2GB, which makes me think it's all there.
I've tried --hls-prefer-natiive, --include-ads, --hls-use-mpegts, --keep-fragments
Fyi, this is the video I'm trying to download: https://www.cartoonnetwork.com/video/ben-10/innervasion-episode.html
I have this over and over in the event log: Non-monotonous DTS in output stream 0:0; previous: 60413327, current: 2726727; changing to 60413328. This may result in incorrect timestamps in the output file.
Anyone ever figure it out?
If you're able to stick with --prefer-ffmpeg
, the following youtube-dl option fixed the av_interleaved_write_frame()
error for me:
--postprocessor-args "-vsync drop"
[edit:] additional details here
Is there anything we can do to get this working? Or Provide the developers with anything?
you can start by providing a publically accessible URL(does not require an account) that has this problem.
This one doesn't require an account, but it is US only, so you may need a VPN.
https://www.tbs.com/shows/the-misery-index/season-1/episode-1/that-milk-must-have-turned-by-now
testing with HLS native downloader(with --hls-prefer-native
option) the full video(23 minutes) has been downloaded and it's playable with VLC 3.0.8 and MPV 0.30.
the manifest contains multiple discontinuity tags that are causing the problems with FFmpeg/LibAV:
the tags used by Turner are:
#EXT-X-DISCONTINUITY
#EXT-X-CUE-OUT:0
and even with the latest version of FFmpeg(4.2.1) the downloaded file is fully playable.
so either update FFmpeg to the latest version or use --hls-prefer-native
option.
youtube-dl https://www.tbs.com/shows/the-misery-index/season-1/episode-1/that-milk-must-have-turned-by-now --hls-prefer-native -vvv [debug] System config: [] [debug] User config: [] [debug] Custom config: [] [debug] Command-line args: [u'https://www.tbs.com/shows/the-misery-index/season-1/episode-1/that-milk-must-have-turned-by-now', u'--hls-prefer-native', u'-vvv'] [debug] Encodings: locale cp1252, fs mbcs, out cp437, pref cp1252 [debug] youtube-dl version 2019.11.05 [debug] Python version 2.7.16 (CPython) - Windows-10-10.0.17763 [debug] exe versions: ffmpeg 4.2.1, ffprobe 4.2.1 [debug] Proxy map: {} [TBS] that-milk-must-have-turned-by-now: Downloading webpage [TBS] f2e23805975ea07c4a8d10994ff20bf1dd672077: Downloading JSON metadata [TBS] f2e23805975ea07c4a8d10994ff20bf1dd672077: Downloading XML [TBS] f2e23805975ea07c4a8d10994ff20bf1dd672077: Downloading m3u8 information [debug] Default format spec: bestvideo+bestaudio/best [debug] Invoking downloader on u'https://tve.cdn.turner.com/tbs/c5f50ffe2f9fc66a6767df05e45721f6/layer7/layer7_bk.m3u8?hdntl=exp=1573811920~acl=%2ftbs%2fc5f50ffe2f9fc66a6767df05e45721f6%2f*~hmac=f3028d279b44254ecec28c4556d2de8253a70972390e2ab1f07b802e36c26bfe'
I ran it on 4.2.1 with hls-prefer-native, and i got a 8m video from it. A few other people are having the same issue. Might just have to wait and see if anyone can figure it out.
you didn't post the part when the download process is done(just to verify that the HLS Native Downloader is the one used to download the file). so, it would be better to put this part as well and also state what is the player you're using to play the file and its version. the quality I have downloaded is(i did check that it has the same discontinuities):
hls-880 mp4 640x360 880k , avc1.42001e, 29.97fps, mp4a.40.2
other differences such as the OS(Linux vs Window) and Python version(3.8.0 vs 2.7.16) should not be relevant. as I can't reproduce the problem, there is nothing that I can do to help.
I don't know if this helps, but I was able to get this working after reading these linked issues, including subtitles after running CCExtractor as well.
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['--write-all-thumbnails', '-f', 'bestvideo[height=1080]+bestaudio/bestvideo+bestaudio/best', '--add-metadata', '--hls-prefer-native', '--ap-mso', 'Charter_Direct', '--ap-username', 'PRIVATE', '--ap-password', 'PRIVATE', 'https://www.adultswim.com/videos/rick-and-morty/rattlestar-ricklactica', '--verbose']
[debug] Encodings: locale cp65001, fs mbcs, out cp65001, pref cp65001
[debug] youtube-dl version 2019.11.28
[debug] Python version 3.4.4 (CPython) - Windows-10-10.0.17763
[debug] exe versions: ffmpeg 4.2.1, ffprobe 4.2.1, rtmpdump 2.3
[debug] Proxy map: {}
[AdultSwim] rattlestar-ricklactica: Downloading JSON metadata
[AdultSwim] bd8e887924b70af7343008ccbf546a9d447ec2ca: Downloading JSON metadata
[AdultSwim] bd8e887924b70af7343008ccbf546a9d447ec2ca: Retrieving Media Token
[AdultSwim] bd8e887924b70af7343008ccbf546a9d447ec2ca: Downloading XML
[AdultSwim] bd8e887924b70af7343008ccbf546a9d447ec2ca: Downloading m3u8 information
[AdultSwim] AW7xVU2HadNs2hMNfuwC: Downloading thumbnail ...
[AdultSwim] AW7xVU2HadNs2hMNfuwC: Writing thumbnail to: Rick and Morty - Rattlestar Ricklactica-AW7xVU2HadNs2hMNfuwC.jpg
[debug] Invoking downloader on 'https://tve.cdn.turner.com/adultswim/0e3ed51de7f72d8b40fc0d13b8f753b3/layer7/layer7_bk.m3u8?hdntl=exp=1576553846~acl=%2fadultswim%2f0e3ed51de7f72d8b40fc0d13b8f753b3%2f*~hmac=eb769ff86bf66dd5c84de11de28cb91a6c2db2272a8d76273fb7feb4431c9fcb'
[hlsnative] Downloading m3u8 manifest
[hlsnative] Total fragments: 218
[download] Destination: Rick and Morty - Rattlestar Ricklactica-AW7xVU2HadNs2hMNfuwC.mp4
[download] 100% of 784.54MiB in 00:52
[debug] ffmpeg command line: ffprobe -show_streams "file:Rick and Morty - Rattlestar Ricklactica-AW7xVU2HadNs2hMNfuwC.mp4"
[ffmpeg] Fixing malformed AAC bitstream in "Rick and Morty - Rattlestar Ricklactica-AW7xVU2HadNs2hMNfuwC.mp4"
[debug] ffmpeg command line: ffmpeg -y -loglevel "repeat+info" -i "file:Rick and Morty - Rattlestar Ricklactica-AW7xVU2HadNs2hMNfuwC.mp4" -c copy -f mp4 "-bsf:a" aac_adtstoasc "file:Rick and Morty - Rattlestar Ricklactica-AW7xVU2HadNs2hMNfuwC.temp.mp4"
[ffmpeg] Adding metadata to 'Rick and Morty - Rattlestar Ricklactica-AW7xVU2HadNs2hMNfuwC.mp4'
[debug] ffmpeg command line: ffmpeg -y -loglevel "repeat+info" -i "file:Rick and Morty - Rattlestar Ricklactica-AW7xVU2HadNs2hMNfuwC.mp4" -i "file:Rick and Morty - Rattlestar Ricklactica-AW7xVU2HadNs2hMNfuwC.meta" -c copy -metadata "description=Lots of things in space broh. Snakes and sharp stuff. Watch this broh." -metadata "date=20191216" -metadata "comment=Lots of things in space broh. Snakes and sharp stuff. Watch this broh." -metadata "purl=https://www.adultswim.com/videos/rick-and-morty/rattlestar-ricklactica" -metadata "title=Rick and Morty - Rattlestar Ricklactica" -map_metadata 1 "file:Rick and Morty - Rattlestar Ricklactica-AW7xVU2HadNs2hMNfuwC.temp.mp4"
This downloads the file correctly. Then I ran CCExtractor (GUI, I'm lazy) to dump an SRT conversion of the embedded EIA_608 captions. With iTunes files, ffmpeg can read them because they're a separate stream, but Turner has them embedded in the video stream.
Here's the input/output if you're interested?
ccextractorwin.exe --gui_mode_reports -out=srt -bom -latin1 -trim "F:\youtube-dl\Rick and Morty - Rattlestar Ricklactica-AW7xVU2HadNs2hMNfuwC.mp4"
CCExtractor 0.87, Carlos Fernandez Sanz, Volker Quetschke.
Teletext portions taken from Petr Kutalek's telxcc
--------------------------------------------------------------------------
Input: F:\youtube-dl\Rick and Morty - Rattlestar Ricklactica-AW7xVU2HadNs2hMNfuwC.mp4
[Extract: 1] [Stream mode: Autodetect]
[Program : Auto ] [Hauppage mode: No] [Use MythTV code: Auto]
[Timing mode: Auto] [Debug: No] [Buffer input: Yes]
[Use pic_order_cnt_lsb for H.264: No] [Print CC decoder traces: No]
[Target format: .srt] [Encoding: Latin-1] [Delay: 0] [Trim lines: Yes]
[Add font color data: Yes] [Add font typesetting: Yes]
[Convert case: No] [Video-edit join: No]
[Extraction start time: not set (from start)]
[Extraction end time: not set (to end)]
[Live stream: No] [Clock frequency: 90000]
[Teletext page: Autodetect]
[Start credits text: None]
[Quantisation-mode: CCExtractor's internal function]
-----------------------------------------------------------------
Opening file: F:\youtube-dl\Rick and Morty - Rattlestar Ricklactica-AW7xVU2HadNs2hMNfuwC.mp4
Detected MP4 box with name: ftyp
Detected MP4 box with name: free
Detected MP4 box with name: mdat
File seems to be a MP4
Analyzing data with GPAC (MP4 library)
Opening 'F:\youtube-dl\Rick and Morty - Rattlestar Ricklactica-AW7xVU2HadNs2hMNfuwC.mp4': ok
Track 1, type=vide subtype=avc1
Track 2, type=soun subtype=MPEG
MP4: found 2 tracks: 1 avc and 0 cc
Processing track 1, type=vide subtype=avc1
Changed fps using NAL to: 29.970000
XDS: ContentAdvisory: US TV Parental Guidelines. Age Rating: TV-Y (All Children)
XDS:
Processing track 2, type=soun subtype=MPEG
Closing media: ok
Found 1 AVC track(s). Found no dedicated CC track(s).
Total frames time: 00:21:41:768 (39014 frames at 29.97fps)
Min PTS: 00:00:00:066
Max PTS: 00:21:41:868
Length: 00:21:41:802
Done, processing time = 5 seconds
Issues? Open a ticket here
https://github.com/CCExtractor/ccextractor/issues
F:\youtube-dl>ren "Rick and Morty - Rattlestar Ricklactica-AW7xVU2HadNs2hMNfuwC.jpg" cover_wide.jpg
The above step isn't really necessary, but I use mkv and want the thumbnail. The "cover.jpg" referenced below is just the iTunes Season 4 cover. So if you were so inclined you could drop -attach cover.jpg -metadata:s:t mimetype=image/jpeg -attach cover_wide.jpg -metadata:s:t mimetype=image/jpeg
from the command below.
F:\youtube-dl>ffmpeg -i "Rick and Morty - Rattlestar Ricklactica-AW7xVU2HadNs2hMNfuwC.mp4" -i "Rick and Morty - Rattlestar Ricklactica-AW7xVU2HadNs2hMNfuwC.srt" -metadata:s:s:0 language=eng -c copy -vsync cfr -attach cover.jpg -metadata:s:t mimetype=image/jpeg -attach cover_wide.jpg -metadata:s:t mimetype=image/jpeg "Rick and Morty - S04E05 - Rattlestar Ricklactica.mkv"
ffmpeg version 4.2.1 Copyright (c) 2000-2019 the FFmpeg developers
built with gcc 9.1.1 (GCC) 20190807
configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libdav1d --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvorbis --enable-libvo-amrwbenc --enable-libmysofa --enable-libspeex --enable-libxvid --enable-libaom --enable-libmfx --enable-amf --enable-ffnvcodec --enable-cuvid --enable-d3d11va --enable-nvenc --enable-nvdec --enable-dxva2 --enable-avisynth --enable-libopenmpt
libavutil 56. 31.100 / 56. 31.100
libavcodec 58. 54.100 / 58. 54.100
libavformat 58. 29.100 / 58. 29.100
libavdevice 58. 8.100 / 58. 8.100
libavfilter 7. 57.100 / 7. 57.100
libswscale 5. 5.100 / 5. 5.100
libswresample 3. 5.100 / 3. 5.100
libpostproc 55. 5.100 / 55. 5.100
[mov,mp4,m4a,3gp,3g2,mj2 @ 0000026af977aac0] Referenced QT chapter track not found
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'Rick and Morty - Rattlestar Ricklactica-AW7xVU2HadNs2hMNfuwC.mp4':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
title : Rick and Morty - Rattlestar Ricklactica
date : 20191216
encoder : Lavf58.29.100
comment : Lots of things in space broh. Snakes and sharp stuff. Watch this broh.
description : Lots of things in space broh. Snakes and sharp stuff. Watch this broh.
Duration: 00:21:41.77, start: 0.000000, bitrate: 4947 kb/s
Chapter #0:0: start 0.000000, end 782.014000
Metadata:
title :
Chapter #0:1: start 782.014000, end 1301.769000
Metadata:
title :
Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], Closed Captions, 4812 kb/s, 29.97 fps, 29.97 tbr, 90k tbn, 59.94 tbc (default)
Metadata:
handler_name : VideoHandler
Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 125 kb/s (default)
Metadata:
handler_name : SoundHandler
Input #1, srt, from 'Rick and Morty - Rattlestar Ricklactica-AW7xVU2HadNs2hMNfuwC.srt':
Duration: N/A, bitrate: N/A
Stream #1:0: Subtitle: subrip
Output #0, matroska, to 'Rick and Morty - S04E05 - Rattlestar Ricklactica.mkv':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
title : Rick and Morty - Rattlestar Ricklactica
date : 20191216
description : Lots of things in space broh. Snakes and sharp stuff. Watch this broh.
comment : Lots of things in space broh. Snakes and sharp stuff. Watch this broh.
encoder : Lavf58.29.100
Chapter #0:0: start 0.000000, end 782.014000
Metadata:
title :
Chapter #0:1: start 782.014000, end 1301.769000
Metadata:
title :
Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 4812 kb/s, 29.97 fps, 29.97 tbr, 1k tbn, 90k tbc (default)
Metadata:
handler_name : VideoHandler
Stream #0:1(und): Audio: aac (LC) ([255][0][0][0] / 0x00FF), 44100 Hz, stereo, fltp, 125 kb/s (default)
Metadata:
handler_name : SoundHandler
Stream #0:2(eng): Subtitle: subrip
Stream #0:3: Attachment: none
Metadata:
filename : cover.jpg
mimetype : image/jpeg
Stream #0:4: Attachment: none
Metadata:
filename : cover_wide.jpg
mimetype : image/jpeg
Stream mapping:
Stream #0:0 -> #0:0 (copy)
Stream #0:1 -> #0:1 (copy)
Stream #1:0 -> #0:2 (copy)
File cover.jpg -> Stream #0:3
File cover_wide.jpg -> Stream #0:4
Press [q] to stop, [?] for help
frame=39014 fps=19459 q=-1.0 Lsize= 785685kB time=00:21:41.72 bitrate=4944.5kbits/s speed= 649x
video:764764kB audio:19957kB subtitle:16kB other streams:0kB global headers:0kB muxing overhead: 0.120896%
Not sure if -vsync cfr
is necessary or not, but as far as I can tell, 1. It's not interlaced anyways, 2. Adult Swim/Cartoon Network "broadcast" and upload online at 29.97p. I think the 59.94 time base tag that ends up dropping from the output video stream is only present in the source for ads' sake?
This gives me an MKV with:
There isn't a way to have youtube-dl do the last two steps, is there? With or without the thumbs.
I have tried the solutions presented in this bug and I am still having this issue on
Issue seems to be still there. I tried to download this: Adultswim but I only manage to get a 11:39min long video file. The only error I see is Non-monotonous DTS in output stream 0:1; previous. --postprocessor-args "-vsync drop" is not helping and -use_wallclock_as_timestamps 1 is also not working.
Please follow the guide below
x
into all the boxes [ ] relevant to your issue (like this:[x]
)Make sure you are using the latest version: run
youtube-dl --version
and ensure your version is 2018.04.03. If it's not, read this FAQ entry and update. Issues with outdated version will be rejected.Before submitting an issue make sure you have:
What is the purpose of your issue?
The following sections concretize particular purposed issues, you can erase any section (the contents between triple ---) not applicable to your issue
If the purpose of this issue is a bug report, site support request or you are not completely sure provide the full verbose output as follows:
Add the
-v
flag to your command line you run youtube-dl with (youtube-dl -v <your command line>
), copy the whole output and insert it here. It should look similar to one below (replace it with your log inserted between triple ```):If the purpose of this issue is a site support request please provide all kinds of example URLs support for which should be included (replace following example URLs by yours):
Note that youtube-dl does not support sites dedicated to copyright infringement. In order for site support request to be accepted all provided example URLs should not violate any copyrights.
Description of your issue, suggested solution and other information
Attempting to download from TBS because their site is garbage and takes forever to get working. It used to work fine. Now I'm having trouble getting it to download past the first commercial. If I use ffmpeg, I'm guaranteed to only get up to the first commercial. Other issues suggest using
--hls-prefer-native
, so I switched to that. I still end up with just the first segment (764.22MiB), but now I getERROR: av_interleaved_write_frame(): Invalid argument
on top of that. The only solution I can find for that issue is to use--prefer-ffmpeg
(#15010, #9631), which won't work since I'll only get the first segment (up until the ads).Someone else seems to be encountering the same issue: https://github.com/rg3/youtube-dl/issues/15395#issuecomment-360715386 I get a very similar (enormous) log if I use ffmpeg. For
seg-0
URLs, it works fine. Forseg-1
and beyond:The end of the ffmpeg log, in case it's of any use:
I tried
--include-ads
, as well; no luck.The URL I provided as an example doesn't require MSO auth. For URLs in the same series that require MSO auth, I'm encountering #16084 as well, which should be easier to fix.