search

yt-dlp / yt-dlp

A feature-rich command-line audio/video downloader
https://discord.gg/H5MNcFW63r
The Unlicense
83.93k stars 6.54k forks source link

ffmpeg not selected as downloader when not in PATH #2191

Closed huubert closed 2 years ago

huubert commented 2 years ago

Checklist

Description

Hi,

I'm trying to download a livestream, but it doesn't work. It still works fine with youtube-dl, but not with yt-dlp. I suspect the problem is that yt-dlp keeps using hlsnative as the external downloader, even when I clearly specify to use ffmpeg. Because of this, it only downloads the first chunk of video and stops.

This is the youtube-dl command that works fine "youtube-dl.exe" --hls-use-mpegts --ffmpeg-location "ffmpeg-2021-12-30-git-12f21849e5-essentials_build\bin" "same_url_as_in_the_log"

Any help would be appreciated

Verbose log

[debug] Command-line config: ['--verbose', '--downloader', 'm3u8:ffmpeg', '--ffmpeg-location', 'ffmpeg-2021-12-30-git-12f21849e5-essentials_build\\bin', 'https://video-auth2.iol.pt/live_tvi/live_tvi/playlist.m3u8?wmsAuthSign=redacted_id']
[debug] Encodings: locale cp1252, fs utf-8, out cp1252 (No ANSI), err cp1252 (No ANSI), pref cp1252
[debug] yt-dlp version 2021.12.27 [6223f67] (win_exe)
[debug] Python version 3.8.10 (CPython 64bit) - Windows-7-6.1.7601-SP1
[debug] exe versions: ffmpeg 2021-12-30-git-12f21849e5-essentials_build-www.gyan.dev (setts), ffprobe 2021-12-30-git-12f21849e5-essentials_build-www.gyan.dev
[debug] Optional libraries: Cryptodome, mutagen, sqlite, websockets
[debug] Proxy map: {}
[debug] [generic] Extracting URL: https://video-auth2.iol.pt/live_tvi/live_tvi/playlist.m3u8?wmsAuthSign=redacted_id
[generic] playlist: Requesting header
[debug] Identified a direct video link
[generic] playlist: Downloading m3u8 information
[debug] Formats sorted by: hasvid, ie_pref, lang, quality, res, fps, hdr:12(7), vcodec:vp9.2(10), acodec, filesize, fs_approx, tbr, vbr, abr, asr, proto, vext, aext, hasaud, source, id
[debug] Default format spec: bestvideo*+bestaudio/best
[info] playlist: Downloading 1 format(s): 1885
[debug] Invoking downloader on "https://video-auth2.iol.pt/live_tvi/live_tvi/edge_servers/tvi-720_passthrough/chunks.m3u8?nimblesessionid=redacted_id&wmsAuthSign=redacted_id"
[hlsnative] Downloading m3u8 manifest
[hlsnative] Total fragments: 6
[download] Destination: playlist [playlist].mp4

[download]   0.0% of ~11.39MiB at 21.28KiB/s ETA 09:08
[download]   0.0% of ~11.39MiB at 62.50KiB/s ETA 03:06
[download]   0.1% of ~11.39MiB at 145.83KiB/s ETA 01:19
[download]   0.1% of ~11.39MiB at 306.11KiB/s ETA 00:38
[download]   0.3% of ~11.39MiB at 619.97KiB/s ETA 00:18
[download]   0.5% of ~11.39MiB at  1.14MiB/s ETA 00:09 
[download]   1.1% of ~11.39MiB at  2.00MiB/s ETA 00:05
[download]   2.2% of ~11.39MiB at  2.96MiB/s ETA 00:03
[download]   4.4% of ~11.39MiB at  4.02MiB/s ETA 00:02
[download]   8.8% of ~11.39MiB at  5.23MiB/s ETA 00:01
[download]  16.7% of ~11.39MiB at  6.59MiB/s ETA 00:01
[download]  16.7% of ~11.39MiB at  6.57MiB/s ETA 00:01 (frag 1/6)
[download]  14.3% of ~13.29MiB at 17.54KiB/s ETA 00:02 (frag 1/6)
[download]  14.3% of ~13.29MiB at 51.72KiB/s ETA 00:02 (frag 1/6)
[download]  14.3% of ~13.29MiB at 120.68KiB/s ETA 00:02 (frag 1/6)
[download]  14.4% of ~13.29MiB at 254.22KiB/s ETA 00:02 (frag 1/6)
[download]  14.5% of ~13.29MiB at 516.64KiB/s ETA 00:02 (frag 1/6)
[download]  14.7% of ~13.29MiB at 999.94KiB/s ETA 00:02 (frag 1/6)
[download]  15.2% of ~13.29MiB at  1.72MiB/s ETA 00:02 (frag 1/6) 
[download]  16.2% of ~13.29MiB at  2.59MiB/s ETA 00:01 (frag 1/6)
[download]  18.0% of ~13.29MiB at  3.81MiB/s ETA 00:01 (frag 1/6)
[download]  21.8% of ~13.29MiB at  4.99MiB/s ETA 00:01 (frag 1/6)
[download]  29.3% of ~13.29MiB at  6.53MiB/s ETA 00:01 (frag 1/6)
[download]  33.3% of ~13.29MiB at  7.13MiB/s ETA 00:01 (frag 1/6)
[download]  33.3% of ~13.29MiB at  7.11MiB/s ETA 00:01 (frag 2/6)
[download]  32.8% of ~13.50MiB at 18.52KiB/s ETA 00:01 (frag 2/6)
[download]  32.9% of ~13.50MiB at 54.54KiB/s ETA 00:01 (frag 2/6)
[download]  32.9% of ~13.50MiB at 127.27KiB/s ETA 00:01 (frag 2/6)
[download]  32.9% of ~13.50MiB at 267.84KiB/s ETA 00:01 (frag 2/6)
[download]  33.1% of ~13.50MiB at 543.83KiB/s ETA 00:01 (frag 2/6)
[download]  33.3% of ~13.50MiB at  1.03MiB/s ETA 00:01 (frag 2/6) 
[download]  33.7% of ~13.50MiB at  1.80MiB/s ETA 00:01 (frag 2/6)
[download]  34.7% of ~13.50MiB at  2.77MiB/s ETA 00:01 (frag 2/6)
[download]  36.5% of ~13.50MiB at  3.22MiB/s ETA 00:01 (frag 2/6)
[download]  40.2% of ~13.50MiB at  3.96MiB/s ETA 00:01 (frag 2/6)
[download]  47.6% of ~13.50MiB at  5.25MiB/s ETA 00:01 (frag 2/6)
[download]  50.0% of ~13.50MiB at  5.57MiB/s ETA 00:01 (frag 2/6)
[download]  50.0% of ~13.50MiB at  5.56MiB/s ETA 00:01 (frag 3/6)
[download]  49.2% of ~13.73MiB at 18.18KiB/s ETA 00:01 (frag 3/6)
[download]  49.2% of ~13.73MiB at 53.57KiB/s ETA 00:01 (frag 3/6)
[download]  49.2% of ~13.73MiB at 122.80KiB/s ETA 00:01 (frag 3/6)
[download]  49.3% of ~13.73MiB at 263.14KiB/s ETA 00:01 (frag 3/6)
[download]  49.4% of ~13.73MiB at 534.45KiB/s ETA 00:01 (frag 3/6)
[download]  49.6% of ~13.73MiB at 1016.07KiB/s ETA 00:01 (frag 3/6)
[download]  50.1% of ~13.73MiB at  1.63MiB/s ETA 00:01 (frag 3/6)  
[download]  51.0% of ~13.73MiB at  2.47MiB/s ETA 00:01 (frag 3/6)
[download]  52.8% of ~13.73MiB at  3.49MiB/s ETA 00:01 (frag 3/6)
[download]  56.5% of ~13.73MiB at  4.71MiB/s ETA 00:00 (frag 3/6)
[download]  63.7% of ~13.73MiB at  6.29MiB/s ETA 00:00 (frag 3/6)
[download]  66.7% of ~13.73MiB at  6.63MiB/s ETA 00:00 (frag 3/6)
[download]  66.7% of ~13.73MiB at  6.61MiB/s ETA 00:00 (frag 4/6)
[download]  67.2% of ~13.62MiB at 18.52KiB/s ETA 00:00 (frag 4/6)
[download]  67.2% of ~13.62MiB at 54.54KiB/s ETA 00:00 (frag 4/6)
[download]  67.2% of ~13.62MiB at 127.27KiB/s ETA 00:00 (frag 4/6)
[download]  67.3% of ~13.62MiB at 267.84KiB/s ETA 00:00 (frag 4/6)
[download]  67.4% of ~13.62MiB at 543.83KiB/s ETA 00:00 (frag 4/6)
[download]  67.6% of ~13.62MiB at  1.01MiB/s ETA 00:00 (frag 4/6) 
[download]  68.1% of ~13.62MiB at  1.68MiB/s ETA 00:00 (frag 4/6)
[download]  69.0% of ~13.62MiB at  2.65MiB/s ETA 00:00 (frag 4/6)
[download]  70.8% of ~13.62MiB at  3.81MiB/s ETA 00:00 (frag 4/6)
[download]  74.5% of ~13.62MiB at  4.97MiB/s ETA 00:00 (frag 4/6)
[download]  81.9% of ~13.62MiB at  6.47MiB/s ETA 00:00 (frag 4/6)
[download]  83.3% of ~13.62MiB at  6.73MiB/s ETA 00:00 (frag 4/6)
[download]  83.3% of ~13.62MiB at  6.71MiB/s ETA 00:00 (frag 5/6)
[download]  83.6% of ~13.58MiB at 18.52KiB/s ETA 00:00 (frag 5/6)
[download]  83.6% of ~13.58MiB at 54.54KiB/s ETA 00:00 (frag 5/6)
[download]  83.6% of ~13.58MiB at 127.27KiB/s ETA 00:00 (frag 5/6)
[download]  83.7% of ~13.58MiB at 267.84KiB/s ETA 00:00 (frag 5/6)
[download]  83.8% of ~13.58MiB at 543.83KiB/s ETA 00:00 (frag 5/6)
[download]  84.1% of ~13.58MiB at  1.01MiB/s ETA 00:00 (frag 5/6) 
[download]  84.5% of ~13.58MiB at  1.65MiB/s ETA 00:00 (frag 5/6)
[download]  85.4% of ~13.58MiB at  2.68MiB/s ETA 00:00 (frag 5/6)
[download]  87.3% of ~13.58MiB at  3.42MiB/s ETA 00:00 (frag 5/6)
[download]  91.0% of ~13.58MiB at  4.04MiB/s ETA 00:00 (frag 5/6)
[download]  98.3% of ~13.58MiB at  5.18MiB/s ETA 00:00 (frag 5/6)
[download] 100% of ~13.58MiB at  5.15MiB/s ETA 00:00 (frag 5/6)  
[download] 100% of ~13.58MiB at  5.14MiB/s ETA 00:00 (frag 6/6)
[download] 100% of 13.58MiB in 00:02                           
[debug] ffprobe command line: "ffmpeg-2021-12-30-git-12f21849e5-essentials_build\bin\ffprobe" -hide_banner -show_format -show_streams -print_format json "file:playlist [playlist].mp4"
[FixupM3u8] Fixing MPEG-TS in MP4 container of "playlist [playlist].mp4"
[debug] ffmpeg command line: "ffmpeg-2021-12-30-git-12f21849e5-essentials_build\bin\ffmpeg" -y -loglevel "repeat+info" -i "file:playlist [playlist].mp4" -c copy -map 0 -dn -f mp4 "-bsf:a" aac_adtstoasc -movflags "+faststart" "file:playlist [playlist].temp.mp4"
pukkandan commented 2 years ago

Could you try adding ffmpeg to PATH (or same folder as yt-dlp) instead of using --ffmpeg-location

huubert commented 2 years ago

Copied ffmpeg.exe to the same path as yt-dlp.exe and it works fine now. Thanks! I guess it's a bug though?

By the way, this is the same site as #2134. I don't think it needs a specific extractor. The site uses jwplayer and will only work with a portuguese IP address. The link points to a m3u8 like in the log I included above, which can be easily identified with Dev Tools -> Network.

pukkandan commented 2 years ago

Yeah, the issue is this:

https://github.com/yt-dlp/yt-dlp/blob/767f999b53705cdeda1b5bcca360aa1fc9c88135/yt_dlp/downloader/external.py#L322-L326

which can be easily identified with Dev Tools -> Network.

Adding extractor is so you dont have to do that

huubert commented 2 years ago

Adding extractor is so you dont have to do that

Anything I should do to help or do you have enough info?

Another thing: the mp4.part file wasn't playable until I added --hls-use-mpegts to the command. It seems it's not being applied by default like it says in --help, even though it is a livestream.

pukkandan commented 2 years ago

When you pass m3u8 directly, yt-dlp can't reliably detect whether it is a live stream or not. It would work of it were a supported site

huubert commented 2 years ago

How a vod link looks like: https://streaming-vod3.iol.pt/vod/_definst_/6/0/3/d/smil:61c3ad0b0cf21a10a415603d-L/playlist.m3u8?wmsAuthSign=redacted_id

For vods, no options need to be set. Just yt-dlp.exe "link" will work. For lives: yt-dlp.exe --hls-use-mpegts --downloader "m3u8:ffmpeg" "url"

That digit in the url will vary: streaming-vod3.iol.pt video-auth2.iol.pt

bashonly commented 2 years ago

When you pass m3u8 directly, yt-dlp can't reliably detect whether it is a live stream or not. It would work of it were a supported site

Could yt-dlp try looking for #EXT-X-ENDLIST at the end of the m3u8 file, and if it is not found, then handle the playlist as a livestream?

pukkandan commented 2 years ago

This also affects --download-sections now:

yt-dlp-daily "https://www.youtube.com/watch?v=ci6ZtPAN0PM" --download-sections "*00:10-00:20" --no-update -f 248 --verbose
[debug] Command-line config: ['https://www.youtube.com/watch?v=ci6ZtPAN0PM', '--download-sections', '*00:10-00:20', '--no-update', '-f', '248', '--verbose']
[debug] User config "C:\Users\REDACTED\AppData\Roaming\yt-dlp\config.txt": ['--ffmpeg-location', 'C:\\Users\\REDACTED\\ffmpeg\\bin', '-P', 'C:\\Users\\REDACTED\\youtube.dl', '--update', '--audio-quality', '0', '--write-subs', '--write-auto-subs', '--embed-subs', '--compat-options', 'no-keep-subs,no-live-chat']
[debug] Encodings: locale cp1252, fs utf-8, pref cp1252, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version 2022.06.18.43044 [61b4932] (win32_exe)
[debug] ** This build is unofficial daily builds, provided for ease of use.
[debug] ** Please do not ask for any support.
[debug] Compatibility options: no-keep-subs, no-live-chat
[debug] Python version 3.8.10 (CPython 64bit) - Windows-10-10.0.19043-SP0
[debug] Checking exe version: "C:\Users\REDACTED\ffmpeg\bin\ffmpeg" -bsfs
[debug] Checking exe version: "C:\Users\REDACTED\ffmpeg\bin\ffprobe" -bsfs
[debug] exe versions: ffmpeg N-106498-g854615adf2-20220405 (setts), ffprobe N-106498-g854615adf2-20220405
[debug] Optional libraries: Cryptodome-3.14.1, brotli-1.0.9, certifi-2022.06.15, mutagen-1.45.1, sqlite3-2.6.0, websockets-10.3
[debug] Proxy map: {}
[debug] [youtube] Extracting URL: https://www.youtube.com/watch?v=ci6ZtPAN0PM
[youtube] ci6ZtPAN0PM: Downloading webpage
[youtube] ci6ZtPAN0PM: Downloading android player API JSON
[debug] Sort order given by extractor: quality, res, fps, hdr:12, source, codec:vp9.2, lang, proto
[debug] Formats sorted by: hasvid, ie_pref, quality, res, fps, hdr:12(7), source, vcodec:vp9.2(10), acodec, lang, proto, filesize, fs_approx, tbr, vbr, abr, asr, vext, aext, hasaud, id
[info] ci6ZtPAN0PM: Downloading 1 format(s): 248
[info] ci6ZtPAN0PM: Downloading 1 time ranges: 10-20
ERROR: This format cannot be partially downloaded. Aborting
  File "yt_dlp\__main__.py", line 16, in <module>
  File "yt_dlp\__init__.py", line 897, in main
  File "yt_dlp\__init__.py", line 889, in _real_main
  File "yt_dlp\YoutubeDL.py", line 3235, in download
  File "yt_dlp\YoutubeDL.py", line 3211, in wrapper
  File "yt_dlp\YoutubeDL.py", line 1413, in extract_info
  File "yt_dlp\YoutubeDL.py", line 1422, in wrapper
  File "yt_dlp\YoutubeDL.py", line 1506, in __extract_info
  File "yt_dlp\YoutubeDL.py", line 1563, in process_ie_result
  File "yt_dlp\YoutubeDL.py", line 2616, in process_video_result
  File "yt_dlp\YoutubeDL.py", line 2995, in process_info
  File "yt_dlp\YoutubeDL.py", line 962, in report_error
  File "yt_dlp\YoutubeDL.py", line 894, in trouble
someziggyman commented 2 years ago

Youtube clips download also fails with: ERROR: You have requested downloading the video partially, but ffmpeg is not installed. Aborting

Test link: https://youtube.com/clip/Ugkx7Jls-tHrB-KPmJ8etrfZ7CCvJK7_9I-1

Log: ./yt-dlp -vU --ffmpeg-location ffmpeg "https://youtube.com/clip/Ugkx7Jls-tHrB-KPmJ8etrfZ7CCvJK7_9I-1" [debug] Command-line config: ['-vU', '--ffmpeg-location', 'ffmpeg', 'https://youtube.com/clip/Ugkx7Jls-tHrB-KPmJ8etrfZ7CCvJK7_9I-1'] [debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8, error utf-8, screen utf-8 [debug] yt-dlp version 2022.07.18 [135f05ef6] (zip) [debug] Plugins: ['SamplePluginIE', 'SamplePluginPP'] [debug] Python 3.9.13 (CPython 64bit) - macOS-12.4-arm64-arm-64bit [debug] Checking exe version: ffmpeg -bsfs [debug] Checking exe version: /yt-dlp/avconv -bsfs [debug] Checking exe version: /yt-dlp/ffprobe -bsfs [debug] Checking exe version: /yt-dlp/avprobe -bsfs [debug] exe versions: none [debug] Optional libraries: sqlite3-2.6.0 [debug] Proxy map: {} [debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest Latest version: 2022.07.18, Current version: 2022.07.18 yt-dlp is up to date (2022.07.18) [debug] [youtube:clip] Extracting URL: https://youtube.com/clip/Ugkx7Jls-tHrB-KPmJ8etrfZ7CCvJK7_9I-1 [youtube:clip] Ugkx7Jls-tHrB-KPmJ8etrfZ7CCvJK7_9I-1: Downloading webpage [debug] [youtube] Extracting URL: https://www.youtube.com/watch?v=VGk3tY4yP7k [youtube] VGk3tY4yP7k: Downloading webpage [youtube] VGk3tY4yP7k: Downloading android player API JSON [debug] Sort order given by extractor: quality, res, fps, hdr:12, source, codec:vp9.2, lang, proto [debug] Formats sorted by: hasvid, ie_pref, quality, res, fps, hdr:12(7), source, vcodec:vp9.2(10), acodec, lang, proto, filesize, fs_approx, tbr, vbr, abr, asr, vext, aext, hasaud, id [debug] Default format spec: best/bestvideo+bestaudio [info] Ugkx7Jls-tHrB-KPmJ8etrfZ7CCvJK7_9I-1: Downloading 1 format(s): 18 ERROR: You have requested downloading the video partially, but ffmpeg is not installed. Aborting File "/opt/homebrew/Cellar/python@3.9/3.9.13_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/runpy.py", line 197, in _run_module_as_main return _run_code(code, main_globals, None, File "/opt/homebrew/Cellar/python@3.9/3.9.13_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/runpy.py", line 87, in _run_code exec(code, run_globals) File "./yt-dlp/main.py", line 17, in yt_dlp.main() File "./yt-dlp/yt_dlp/init.py", line 940, in main _exit(variadic(_real_main(argv))) File "./yt-dlp/yt_dlp/init.py", line 932, in _real_main return ydl.download(all_urls) File "./yt-dlp/yt_dlp/YoutubeDL.py", line 3298, in download self.__download_wrapper(self.extract_info)( File "./yt-dlp/yt_dlp/YoutubeDL.py", line 3274, in wrapper res = func(args, kwargs) File "./yt-dlp/yt_dlp/YoutubeDL.py", line 1424, in extract_info return self.__extract_info(url, self.get_info_extractor(ie_key), download, extra_info, process) File "./yt-dlp/yt_dlp/YoutubeDL.py", line 1433, in wrapper return func(self, *args, *kwargs) File "./yt-dlp/yt_dlp/YoutubeDL.py", line 1530, in __extract_info return self.process_ie_result(ie_result, download, extra_info) File "./yt-dlp/yt_dlp/YoutubeDL.py", line 1640, in process_ie_result return self.process_ie_result( File "./yt-dlp/yt_dlp/YoutubeDL.py", line 1587, in process_ie_result ie_result = self.process_video_result(ie_result, download=download) File "./yt-dlp/yt_dlp/YoutubeDL.py", line 2678, in process_video_result self.process_info(new_info) File "./yt-dlp/yt_dlp/YoutubeDL.py", line 3056, in process_info self.report_error(f'{msg}. Aborting') File "./yt-dlp/yt_dlp/YoutubeDL.py", line 973, in report_error self.trouble(f'{self._format_err("ERROR:", self.Styles.ERROR)} {message}', args, kwargs) File "./yt-dlp/yt_dlp/YoutubeDL.py", line 905, in trouble tb_data = traceback.format_list(traceback.extract_stack())

pukkandan commented 2 years ago

The issue is that --ffmpeg-location is not passed through to the "downloader selection" code. Solving this "properly" requires a lot of changes. However, since we no longer have to worry about Py3.6, this can be fixed using contextvars

someziggyman commented 2 years ago

The issue is that --ffmpeg-location is not passed through to the "downloader selection" code. Solving this "properly" requires a lot of changes. However, since we no longer have to worry about Py3.6, this can be fixed using contextvars

Always thought --ffmpeg-location is specified to be used in whatever location/ use case necessary. So, was surprised I was wrong. BTW, how can ffprobe path be specified? It seems to me, I came across some link recently, where yt-dlp asked be for ffprobe (I might be a bit wrong here). I forgot about the case and never came back to it, but while we are at it, decided to ask. Apologies, if that's a bit off topic here.

pukkandan commented 2 years ago

It uses the same path as --ffmpeg-location

someziggyman commented 2 years ago

It uses the same path as --ffmpeg-location

so syntax should be yt-dlp --ffmpeg-location ffmpegPath --ffmpeg-location ffprobePath blabla?

pukkandan commented 2 years ago

No, we just look for it in the same location as whatever --ffmpeg-location was given

kawashirov commented 7 months ago

Just figured out, passing --ffmpeg-location to binary or module (python -m yt_dlp) works.

But when importing yt_dlp as python library and using yt_dlp.YoutubeDL(ydl_opts) or something, even if you provide 'ffmpeg_location' to ydl_opts, it will not work.

Turns out, FFmpegPostProcessor._ffmpeg_location.set(opts.ffmpeg_location) workaround being applied only when module being run through __main__.

It can be workarounded manually by ydl_opts.FFmpegPostProcessor._ffmpeg_location.set(...) but bruh I spent two hours to figure that out 🙈