search

ytdl-org / youtube-dl

Command-line program to download videos from YouTube.com and other video sites
http://ytdl-org.github.io/youtube-dl/
The Unlicense
131.21k stars 9.93k forks source link

Conversion failed because of missing aac_adtstoasc/FFmpegFixupM3u8PP #10644

Open johnpatch opened 7 years ago

johnpatch commented 7 years ago

When only a single format uses the m3u8 protocol, the fixup fails to be applied.

Example failing URL: http://www.tf1.fr/tmc/quotidien-avec-yann-barthes/videos/didier-deschamps-souhaite-bonne-chance-a-quotidien.html

johnpatch commented 7 years ago

Duplicate of #9913

adamvoss commented 7 years ago

CBS appears to be affected too: 

youtube-dl http://www.cbs.com/shows/superior-donuts/video/n606KxiDW9XuHS2oGXGt3Ps8jNvqwe_j/superior-donuts-pilot/

Fails with:

[mp4 @ 0x92f6a0] Malformed AAC bitstream detected: use the audio bitstream filter 'aac_adtstoasc' to fix it ('-bsf:a aac_adtstoasc' option with ffmpeg) av_interleaved_write_frame(): Operation not permitted

Setting --merge-output-format mkv as suggested at https://github.com/rg3/youtube-dl/issues/9913#issuecomment-250731749 did not have any effect.

$ youtube-dl --version 2017.02.04.1

harlanb commented 7 years ago

Failing for ABC, CBS and NBC for several days. Running current version.

Output: $ youtube-dl -v --prefer-ffmpeg http://www.nbc.com/timeless/video/the-lost-generation/3463403 [debug] System config: [] [debug] User config: [] [debug] Custom config: [] [debug] Command-line args: [u'-v', u'--prefer-ffmpeg', u'http://www.nbc.com/timeless/video/the-lost-generation/3463403'] [debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8 [debug] youtube-dl version 2017.02.07 [debug] Python version 2.7.12 - Linux-4.4.0-38-generic-x86_64-with-LinuxMint-18-sarah [debug] exe versions: avconv 2.8.10-0ubuntu0.16.04.1, avprobe 2.8.10-0ubuntu0.16.04.1, ffmpeg N-81864-geb60256, ffprobe N-81864-geb60256 [debug] Proxy map: {} [NBC] 3463403: Downloading webpage [ThePlatform] 3463403: Downloading webpage [ThePlatform] 3463403: Downloading SMIL data [ThePlatform] 3463403: Downloading m3u8 information [ThePlatform] 3463403: Downloading JSON metadata [debug] Invoking downloader on u'https://nbcmpx-vh.akamaihd.net/i/video/285/614/170201_3463403_The_Lost_Generation_mpx_,1696,1296,896,696,496,306,150,240,64,.mp4.csmil/index_0_av.m3u8?null=0&id=AgAY2bxgaFumPsKNnFi2h8%2fdSaVcpFy2cUhp5cgbdg+3vgWLk2nnCMYe60oqkxM+ZN5LW5abDVCJXQ%3d%3d&hdntl=exp=1486741314~acl=%2fi%2fvideo%2f285%2f614%2f170201_3463403_The_Lost_Generation_mpx_*~data=hdntl~hmac=4d8ba89cdd01e22da66ebfd02f6018d237a1439d64208a782273b2dabcf41aba' [download] Destination: The Lost Generation-3463403.mp4 [debug] ffmpeg command line: ffmpeg -y -headers 'Accept-Language: en-us,en;q=0.5 Accept-Encoding: gzip, deflate Accept: text/html,application/xhtml+xml,application/xml;q=0.9,/;q=0.8 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20150101 Firefox/47.0 (Chrome) Accept-Charset: ISO-8859-1,utf-8;q=0.7,;q=0.7 Cookie: alid=+3Or5CftYD1n/ioaUXYlag== ' -i 'https://nbcmpx-vh.akamaihd.net/i/video/285/614/170201_3463403_The_Lost_Generation_mpx_,1696,1296,896,696,496,306,150,240,64,.mp4.csmil/index_0_av.m3u8?null=0&id=AgAY2bxgaFumPsKNnFi2h8%2fdSaVcpFy2cUhp5cgbdg+3vgWLk2nnCMYe60oqkxM+ZN5LW5abDVCJXQ%3d%3d&hdntl=exp=1486741314~acl=%2fi%2fvideo%2f285%2f614%2f170201_3463403_The_Lost_Generation_mpx_~data=hdntl~hmac=4d8ba89cdd01e22da66ebfd02f6018d237a1439d64208a782273b2dabcf41aba' -c copy -f mp4 'file:The Lost Generation-3463403.mp4.part' ffmpeg version N-81864-geb60256 Copyright (c) 2000-2016 the FFmpeg developers built with gcc 5.4.0 (Ubuntu 5.4.0-6ubuntu1~16.04.2) 20160609 configuration: --prefix=/home/harlan/ffmpeg_build --pkg-config-flags=--static --extra-cflags=-I/home/harlan/ffmpeg_build/include --extra-ldflags=-L/home/harlan/ffmpeg_build/lib --bindir=/home/harlan/bin --enable-gpl --enable-libass --enable-libfdk-aac --enable-libfreetype --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --enable-openssl --enable-nonfree libavutil 55. 32.100 / 55. 32.100 libavcodec 57. 60.101 / 57. 60.101 libavformat 57. 51.102 / 57. 51.102 libavdevice 57. 0.102 / 57. 0.102 libavfilter 6. 63.100 / 6. 63.100 libswscale 4. 1.100 / 4. 1.100 libswresample 2. 2.100 / 2. 2.100 libpostproc 54. 0.100 / 54. 0.100 Input #0, hls,applehttp, from 'https://nbcmpx-vh.akamaihd.net/i/video/285/614/170201_3463403_The_Lost_Generation_mpx_,1696,1296,896,696,496,306,150,240,64,.mp4.csmil/index_0_av.m3u8?null=0&id=AgAY2bxgaFumPsKNnFi2h8%2fdSaVcpFy2cUhp5cgbdg+3vgWLk2nnCMYe60oqkxM+ZN5LW5abDVCJXQ%3d%3d&hdntl=exp=1486741314~acl=%2fi%2fvideo%2f285%2f614%2f170201_3463403_The_Lost_Generation_mpx_*~data=hdntl~hmac=4d8ba89cdd01e22da66ebfd02f6018d237a1439d64208a782273b2dabcf41aba': Duration: 00:43:14.99, start: 0.100511, bitrate: 0 kb/s Program 0 Metadata: variant_bitrate : 0 Stream #0:0: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p, 1280x720, Closed Captions, 23.98 fps, 23.98 tbr, 90k tbn, 47.95 tbc Metadata: variant_bitrate : 0 Stream #0:1: Audio: aac (LC) ([15][0][0][0] / 0x000F), 44100 Hz, stereo, fltp Metadata: variant_bitrate : 0 Output #0, mp4, to 'file:The Lost Generation-3463403.mp4.part': Metadata: encoder : Lavf57.51.102 Stream #0:0: Video: h264 (High) ([33][0][0][0] / 0x0021), yuv420p, 1280x720, q=2-31, 23.98 fps, 23.98 tbr, 90k tbn, 90k tbc Metadata: variant_bitrate : 0 Stream #0:1: Audio: aac (LC) ([64][0][0][0] / 0x0040), 44100 Hz, stereo Metadata: variant_bitrate : 0 Stream mapping: Stream #0:0 -> #0:0 (copy) Stream #0:1 -> #0:1 (copy) Press [q] to stop, [?] for help [mp4 @ 0x33b5580] Malformed AAC bitstream detected: use the audio bitstream filter 'aac_adtstoasc' to fix it ('-bsf:a aac_adtstoasc' option with ffmpeg) av_interleaved_write_frame(): Operation not permitted frame= 1 fps=0.0 q=-1.0 Lsize= 2kB time=00:00:00.00 bitrate=27552.0kbits/s speed= 3.6x
video:1kB audio:0kB subtitle:0kB other streams:0kB global headers:1kB muxing overhead: 74.645027% Conversion failed!

ERROR: ffmpeg exited with code 1 File "/usr/lib/python2.7/runpy.py", line 174, in _run_module_as_main "main", fname, loader, pkg_name) File "/usr/lib/python2.7/runpy.py", line 72, in _run_code exec code in run_globals File "/usr/local/bin/youtube-dl/main.py", line 19, in youtube_dl.main() File "/usr/local/bin/youtube-dl/youtube_dl/init.py", line 452, in main _real_main(argv) File "/usr/local/bin/youtube-dl/youtube_dl/init.py", line 442, in _real_main retcode = ydl.download(all_urls) File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 1796, in download url, force_generic_extractor=self.params.get('force_generic_extractor', False)) File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 707, in extract_info return self.process_ie_result(ie_result, download, extra_info) File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 778, in process_ie_result new_result, download=download, extra_info=extra_info) File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 753, in process_ie_result return self.process_video_result(ie_result, download=download) File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 1440, in process_video_result self.process_info(new_info) File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 1704, in process_info success = dl(filename, info_dict) File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 1646, in dl return fd.download(name, info) File "/usr/local/bin/youtube-dl/youtube_dl/downloader/common.py", line 353, in download return self.real_download(filename, info_dict) File "/usr/local/bin/youtube-dl/youtube_dl/downloader/external.py", line 44, in real_download self.get_basename(), retval)) File "/usr/local/bin/youtube-dl/youtube_dl/downloader/common.py", line 162, in report_error self.ydl.report_error(*args, **kargs) File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 570, in report_error self.trouble(error_message, tb) File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 532, in trouble tb_data = traceback.format_list(traceback.extract_stack())

$ youtube-dl --version 2017.02.07

remitamine commented 7 years ago

@harlanb, @vossad01 the problem should be fixed with version 2017.02.10.

harlanb commented 7 years ago

CBS and NBC are working. Thank You very much!

mrplePurple commented 7 years ago

This persists for me in version 2017.02.24.1

youtube-dl --ignore-config http://www.nbc.com/law-and-order-special-victims-unit/video/no-surrender/3437476 -v
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'--ignore-config', u'http://www.nbc.com/law-and-order-special-victims-unit/video/no-surrender/3437476', u'-v']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2017.02.24.1
[debug] Python version 2.7.12 - Linux-4.4.0-64-generic-x86_64-with-Ubuntu-16.04-xenial
[debug] exe versions: avconv 2.8.11-0ubuntu0.16.04.1, avprobe 2.8.11-0ubuntu0.16.04.1, ffmpeg 2.8.11-0ubuntu0.16.04.1, ffprobe 2.8.11-0ubuntu0.16.04.1, rtmpdump 2.4
[debug] Proxy map: {}
[NBC] 3437476: Downloading webpage
[ThePlatform] 3437476: Downloading SMIL data
[ThePlatform] 3437476: Downloading m3u8 information
[ThePlatform] 3437476: Downloading JSON metadata
[debug] Invoking downloader on u'https://nbcmpx-vh.akamaihd.net/i/video/215/459/161207_3437476_No_Surrender_mpx_,1696,1296,896,696,496,306,150,240,64,.mp4.csmil/index_0_av.m3u8?null=0&id=AgBiwWcMFz%2fi5IpOsFg2tCw9CeVtOhjyxYSdZDR9P2tTWLTMQvHFcZnTpoxhs8TRFsWcNTQl0KqzvA%3d%3d&hdntl=exp=1488035850~acl=%2fi%2fvideo%2f215%2f459%2f161207_3437476_No_Surrender_mpx_*~data=hdntl~hmac=23ea47450975f4ae262be324306b52d8584da9b3782ec4f8f4981aaee4842971'
[download] Destination: No Surrender-3437476.mp4
[debug] avconv command line: avconv -y -headers 'Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip, deflate
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20150101 Firefox/47.0 (Chrome)
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Cookie: _alid_=QK14UHYNVSdn0PrNHrYktg==
' -i 'https://nbcmpx-vh.akamaihd.net/i/video/215/459/161207_3437476_No_Surrender_mpx_,1696,1296,896,696,496,306,150,240,64,.mp4.csmil/index_0_av.m3u8?null=0&id=AgBiwWcMFz%2fi5IpOsFg2tCw9CeVtOhjyxYSdZDR9P2tTWLTMQvHFcZnTpoxhs8TRFsWcNTQl0KqzvA%3d%3d&hdntl=exp=1488035850~acl=%2fi%2fvideo%2f215%2f459%2f161207_3437476_No_Surrender_mpx_*~data=hdntl~hmac=23ea47450975f4ae262be324306b52d8584da9b3782ec4f8f4981aaee4842971' -c copy -f mp4 'file:No Surrender-3437476.mp4.part'
ffmpeg version 2.8.11-0ubuntu0.16.04.1 Copyright (c) 2000-2017 the FFmpeg developers
  built with gcc 5.4.0 (Ubuntu 5.4.0-6ubuntu1~16.04.4) 20160609
  configuration: --prefix=/usr --extra-version=0ubuntu0.16.04.1 --build-suffix=-ffmpeg --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --cc=cc --cxx=g++ --enable-gpl --enable-shared --disable-stripping --disable-decoder=libopenjpeg --disable-decoder=libschroedinger --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmodplug --enable-libmp3lame --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-librtmp --enable-libschroedinger --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxvid --enable-libzvbi --enable-openal --enable-opengl --enable-x11grab --enable-libdc1394 --enable-libiec61883 --enable-libzmq --enable-frei0r --enable-libx264 --enable-libopencv
  libavutil      54. 31.100 / 54. 31.100
  libavcodec     56. 60.100 / 56. 60.100
  libavformat    56. 40.101 / 56. 40.101
  libavdevice    56.  4.100 / 56.  4.100
  libavfilter     5. 40.101 /  5. 40.101
  libavresample   2.  1.  0 /  2.  1.  0
  libswscale      3.  1.101 /  3.  1.101
  libswresample   1.  2.101 /  1.  2.101
  libpostproc    53.  3.100 / 53.  3.100
Input #0, hls,applehttp, from 'https://nbcmpx-vh.akamaihd.net/i/video/215/459/161207_3437476_No_Surrender_mpx_,1696,1296,896,696,496,306,150,240,64,.mp4.csmil/index_0_av.m3u8?null=0&id=AgBiwWcMFz%2fi5IpOsFg2tCw9CeVtOhjyxYSdZDR9P2tTWLTMQvHFcZnTpoxhs8TRFsWcNTQl0KqzvA%3d%3d&hdntl=exp=1488035850~acl=%2fi%2fvideo%2f215%2f459%2f161207_3437476_No_Surrender_mpx_*~data=hdntl~hmac=23ea47450975f4ae262be324306b52d8584da9b3782ec4f8f4981aaee4842971':
  Duration: 00:40:45.03, start: 0.100511, bitrate: 0 kb/s
  Program 0 
    Metadata:
      variant_bitrate : 0
    Stream #0:0: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p, 1280x720, Closed Captions, 23.98 fps, 23.98 tbr, 90k tbn, 47.95 tbc
    Stream #0:1: Audio: aac (LC) ([15][0][0][0] / 0x000F), 44100 Hz, stereo, fltp, 5 kb/s
[mp4 @ 0x138a1c0] Codec for stream 0 does not use global headers but container format requires global headers
[mp4 @ 0x138a1c0] Codec for stream 1 does not use global headers but container format requires global headers
Output #0, mp4, to 'file:No Surrender-3437476.mp4.part':
  Metadata:
    encoder         : Lavf56.40.101
    Stream #0:0: Video: h264 ([33][0][0][0] / 0x0021), yuv420p, 1280x720, q=2-31, 23.98 fps, 23.98 tbr, 90k tbn, 90k tbc
    Stream #0:1: Audio: aac ([64][0][0][0] / 0x0040), 44100 Hz, stereo, 5 kb/s
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
  Stream #0:1 -> #0:1 (copy)
Press [q] to stop, [?] for help
[mp4 @ 0x138a1c0] Malformed AAC bitstream detected: use the audio bitstream filter 'aac_adtstoasc' to fix it ('-bsf:a aac_adtstoasc' option with ffmpeg)
av_interleaved_write_frame(): Operation not permitted
frame=    1 fps=0.0 q=-1.0 Lsize=       2kB time=00:00:00.02 bitrate= 593.3kbits/s    
video:1kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 79.749481%
Conversion failed!

ERROR: ffmpeg exited with code 1
  File "/usr/lib/python2.7/runpy.py", line 174, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/usr/local/bin/youtube-dl/__main__.py", line 19, in <module>
    youtube_dl.main()
  File "/usr/local/bin/youtube-dl/youtube_dl/__init__.py", line 457, in main
    _real_main(argv)
  File "/usr/local/bin/youtube-dl/youtube_dl/__init__.py", line 447, in _real_main
    retcode = ydl.download(all_urls)
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 1883, in download
    url, force_generic_extractor=self.params.get('force_generic_extractor', False))
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 772, in extract_info
    return self.process_ie_result(ie_result, download, extra_info)
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 851, in process_ie_result
    new_result, download=download, extra_info=extra_info)
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 826, in process_ie_result
    return self.process_video_result(ie_result, download=download)
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 1527, in process_video_result
    self.process_info(new_info)
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 1791, in process_info
    success = dl(filename, info_dict)
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 1733, in dl
    return fd.download(name, info)
  File "/usr/local/bin/youtube-dl/youtube_dl/downloader/common.py", line 353, in download
    return self.real_download(filename, info_dict)
  File "/usr/local/bin/youtube-dl/youtube_dl/downloader/external.py", line 44, in real_download
    self.get_basename(), retval))
  File "/usr/local/bin/youtube-dl/youtube_dl/downloader/common.py", line 162, in report_error
    self.ydl.report_error(*args, **kargs)
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 589, in report_error
    self.trouble(error_message, tb)
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 551, in trouble
    tb_data = traceback.format_list(traceback.extract_stack())

However running with --hls-prefer-native allows the file to be downloaded.

remitamine commented 7 years ago

@mrplePurple you're problem is different, it happened because avconv is symlinked to ffmpeg(#8720).

mrplePurple commented 7 years ago

You were right! Removed /usr/bin/avconv and it works like a dream. Thanks!

nahoj commented 6 years ago

I have the same problem or a similar one using youtube-dl 2017.10.15.1 with ffmpeg 2.4.13 to download the same TF1 video that johnpatch initially gave. Adding -bsf:a aac_adtstoasc to the ffmpeg command solves it.

Calling youtube-dl with --hls-prefer-native doesn't help, but --merge-output-format mkv works.

$ youtube-dl "http://www.tf1.fr/tmc/quotidien-avec-yann-barthes/videos/didier-deschamps-souhaite-bonne-chance-a-quotidien.html" -v
...
[debug] ffmpeg command line: ffmpeg -y -i 'file:Didier Deschamps souhaite bonne chance à Quotidien-13245448.fdash-video=2493346.mp4' -i 'file:Didier Deschamps souhaite bonne chance à Quotidien-13245448.fhls-68.mp4' -c copy -map 0:v:0 -map 1:a:0 'file:Didier Deschamps souhaite bonne chance à Quotidien-13245448.temp.mp4'
ERROR: Conversion failed!
...

$ ffmpeg -y -i 'file:Didier Deschamps souhaite bonne chance à Quotidien-13245448.fdash-video=2493346.mp4' -i 'file:Didier Deschamps souhaite bonne chance à Quotidien-13245448.fhls-68.mp4' -c copy -map 0:v:0 -map 1:a:0 'file:Didier Deschamps souhaite bonne chance à Quotidien-13245448.temp.mp4'
...
[mp4 @ 0x1863d00] Malformed AAC bitstream detected: use the audio bitstream filter 'aac_adtstoasc' to fix it ('-bsf:a aac_adtstoasc' option with ffmpeg)
av_interleaved_write_frame(): Operation not permitted
...

$ ffmpeg -y -i 'file:Didier Deschamps souhaite bonne chance à Quotidien-13245448.fdash-video=2493346.mp4' -i 'file:Didier Deschamps souhaite bonne chance à Quotidien-13245448.fhls-68.mp4' -c copy -map 0:v:0 -map 1:a:0 -bsf:a aac_adtstoasc 'file:Didier Deschamps souhaite bonne chance à Quotidien-13245448.temp.mp4'
...
[works fine]
francisrod01 commented 6 years ago

@nahoj for me it was the opposite of what you said, as below:

~$ youtube-dl https://youtu.be/xxxxxxxxx --hls-prefer-native