search

ytdl-org / youtube-dl

Command-line program to download videos from YouTube.com and other video sites
http://ytdl-org.github.io/youtube-dl/
The Unlicense
131.39k stars 9.96k forks source link

youtube_dl.jsinterp.JSInterpreter.Exception: slice must be applied on a list in: 'a.slice(0,0) #32896

Closed xihajun closed 1 month ago

xihajun commented 1 month ago

Checklist

Verbose log

Latest version: 2024.08.02, Current version: 2024.08.02
youtube-dl is up to date (2024.08.02)
[youtube] LG9G4aA28rU: Downloading webpage
[youtube] LG9G4aA28rU: Downloading player b12cc44b
WARNING: [youtube] Falling back to generic n function search
WARNING: [youtube] LG9G4aA28rU: Unable to decode n-parameter: expect download to be blocked or throttled (slice must be applied on a list in: 'a.slice(0,0)'; please report this issue on https://github.com/ytdl-org/youtube-dl/issues , using the appropriate issue template. Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose option and include the complete output. Traceback (most recent call last):
  File "/__w/speech2text/speech2text/youtube/./youtube-dl/youtube_dl/extractor/youtube.py", line 1646, in _decrypt_nsig
    ret = extract_nsig(jsi, func_code)(n)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/__w/speech2text/speech2text/youtube/./youtube-dl/youtube_dl/extractor/youtube.py", line 1722, in extract_nsig
    ret = func([s])
          ^^^^^^^^^
  File "/__w/speech2text/speech2text/youtube/./youtube-dl/youtube_dl/jsinterp.py", line 1122, in resf
    ret, should_abort = self.interpret_statement(code.replace('\n', ' '), var_stack, allow_recursion - 1)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/__w/speech2text/speech2text/youtube/./youtube-dl/youtube_dl/jsinterp.py", line 249, in interpret_statement
  File "/__w/speech2text/speech2text/youtube/./youtube-dl/youtube_dl/extractor/youtube.py", line 1646, in _decrypt_nsig
    ret = extract_nsig(jsi, func_code)(n)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/__w/speech2text/speech2text/youtube/./youtube-dl/youtube_dl/extractor/youtube.py", line 1722, in extract_nsig
    ret = func([s])
          ^^^^^^^^^
  File "/__w/speech2text/speech2text/youtube/./youtube-dl/youtube_dl/jsinterp.py", line 1122, in resf
    ret, should_abort = self.interpret_statement(code.replace('\n', ' '), var_stack, allow_recursion - 1)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/__w/speech2text/speech2text/youtube/./youtube-dl/youtube_dl/jsinterp.py", line 249, in interpret_statement
    ret, should_ret = f(self, stmt, local_vars, allow_recursion, *args, **kwargs)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/__w/speech2text/speech2text/youtube/./youtube-dl/youtube_dl/jsinterp.py", line 495, in interpret_statement
    ret, should_return = self.interpret_statement(sub_stmt, local_vars, allow_recursion)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/__w/speech2text/speech2text/youtube/./youtube-dl/youtube_dl/jsinterp.py", line 249, in interpret_statement
    ret, should_ret = f(self, stmt, local_vars, allow_recursion, *args, **kwargs)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/__w/speech2text/speech2text/youtube/./youtube-dl/youtube_dl/jsinterp.py", line 711, in interpret_statement
    ret, should_abort = self.interpret_statement(sub_expr, local_vars, allow_recursion)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/__w/speech2text/speech2text/youtube/./youtube-dl/youtube_dl/jsinterp.py", line 249, in interpret_statement
    ret, should_ret = f(self, stmt, local_vars, allow_recursion, *args, **kwargs)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/__w/speech2text/speech2text/youtube/./youtube-dl/youtube_dl/jsinterp.py", line 750, in interpret_statement
    local_vars[m.group('out')] = self._operator(
                                 ^^^^^^^^^^^^^^^
  File "/__w/speech2text/speech2text/youtube/./youtube-dl/youtube_dl/jsinterp.py", line 441, in _operator
    right_val = self.interpret_expression(right_expr, local_vars, allow_recursion)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/__w/speech2text/speech2text/youtube/./youtube-dl/youtube_dl/jsinterp.py", line 1017, in interpret_expression
    ret, should_return = self.interpret_statement(expr, local_vars, allow_recursion)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/__w/speech2text/speech2text/youtube/./youtube-dl/youtube_dl/jsinterp.py", line 249, in interpret_statement
    ret, should_ret = f(self, stmt, local_vars, allow_recursion, *args, **kwargs)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/__w/speech2text/speech2text/youtube/./youtube-dl/youtube_dl/jsinterp.py", line 1001, in interpret_statement
    return eval_method(variable, member), should_return
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/__w/speech2text/speech2text/youtube/./youtube-dl/youtube_dl/jsinterp.py", line 883, in eval_method
    argvals = [
              ^
  File "/__w/speech2text/speech2text/youtube/./youtube-dl/youtube_dl/jsinterp.py", line 884, in <listcomp>
    self.interpret_expression(v, local_vars, allow_recursion)
  File "/__w/speech2text/speech2text/youtube/./youtube-dl/youtube_dl/jsinterp.py", line 1017, in interpret_expression
    ret, should_return = self.interpret_statement(expr, local_vars, allow_recursion)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/__w/speech2text/speech2text/youtube/./youtube-dl/youtube_dl/jsinterp.py", line 249, in interpret_statement
    ret, should_ret = f(self, stmt, local_vars, allow_recursion, *args, **kwargs)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/__w/speech2text/speech2text/youtube/./youtube-dl/youtube_dl/jsinterp.py", line 1001, in interpret_statement
    return eval_method(variable, member), should_return
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/__w/speech2text/speech2text/youtube/./youtube-dl/youtube_dl/jsinterp.py", line 928, in eval_method
    assertion(isinstance(obj, list), 'must be applied on a list')
  File "/__w/speech2text/speech2text/youtube/./youtube-dl/youtube_dl/jsinterp.py", line [85](https://github.com/xihajun/speech2text/actions/runs/10256086421/job/28374451507#step:4:86)1, in assertion
    raise self.Exception('{memb} {msg}'.format(**locals()), expr=expr)
youtube_dl.jsinterp.JSInterpreter.Exception: slice must be applied on a list in: 'a.slice(0,0)'; please report this issue on https://github.com/ytdl-org/youtube-dl/issues , using the appropriate issue template. Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose option and include the complete output.

Description

WRITE DESCRIPTION HERE

bashonly commented 1 month ago

related: https://github.com/yt-dlp/yt-dlp/commit/bb8bf1db993f59752d20b73b861bd55e40cf0e31, https://github.com/yt-dlp/yt-dlp/commit/c86891eb9434b4d7eec426d38c0c625b5e13cb2f

dirkf commented 1 month ago

Slicing patch for yt-dl, where we can't say fn(*args, rest), and also handling JS .slice(n) -> Py [slice(n, None)]:

--- old/youtube-dl/youtube_dl/jsinterp.py
+++ new/youtube-dl/youtube_dl/jsinterp.py
@@ -925,9 +925,11 @@
                     obj.reverse()
                     return obj
                 elif member == 'slice':
-                    assertion(isinstance(obj, list), 'must be applied on a list')
-                    assertion(len(argvals) == 1, 'takes exactly one argument')
-                    return obj[argvals[0]:]
+                    assertion(isinstance(obj, (list, compat_str)), 'must be applied on a list or string')
+                    assertion(len(argvals) <= 2, 'takes between 0 and 2 arguments')
+                    if len(argvals) < 2:
+                        argvals += (None,)
+                    return obj[slice(*argvals)]
                 elif member == 'splice':
                     assertion(isinstance(obj, list), 'must be applied on a list')
                     assertion(argvals, 'takes one or more arguments')
1000i100 commented 1 month ago

it look like i have the same issue :

youtube-dl --verbose --extract-audio --audio-format mp3 https://www.youtube.com/playlist\?list\=PLAn_LH8R51dnof424PSSK5eQ0BWBCJLRo

[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['--verbose', '--extract-audio', '--audio-format', 'mp3', 'https://www.youtube.com/playlist?list=PLAn_LH8R51dnof424PSSK5eQ0BWBCJLRo']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, pref UTF-8
[debug] youtube-dl version 2021.12.17
[debug] Git HEAD: 71223bff3
[debug] Python 3.11.2 (CPython x86_64 64bit) - Linux-6.1.0-12-amd64-x86_64-with-glibc2.36 - OpenSSL 3.0.11 19 Sep 2023 - glibc 2.36
[debug] exe versions: ffmpeg 5.1.4-0, ffprobe 5.1.4-0
[debug] Proxy map: {}
[youtube:tab] PLAn_LH8R51dnof424PSSK5eQ0BWBCJLRo: Downloading webpage
[download] Downloading playlist: Vidéos publiées
[youtube:tab] Downloading page 1
[youtube:tab] playlist Vidéos publiées: Downloading 8 videos
[download] Downloading video 1 of 8
[youtube] 9JU8LTyEwBg: Downloading webpage
WARNING: [youtube] 9JU8LTyEwBg: Unable to decode n-parameter: expect download to be blocked or throttled (slice must be applied on a list in: 'a.slice(0,0)'; please report this issue on https://github.com/ytdl-org/youtube-dl/issues , using the appropriate issue template. Make sure you are using the latest version; see  https://github.com/ytdl-org/youtube-dl/#user-content-installation  on how to update. Be sure to call youtube-dl with the --verbose option and include the complete output. Traceback (most recent call last):
  File "/home/1000i100/zsh-snap/youtube-dl/youtube_dl/extractor/youtube.py", line 1646, in _decrypt_nsig
    ret = extract_nsig(jsi, func_code)(n)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/1000i100/zsh-snap/youtube-dl/youtube_dl/extractor/youtube.py", line 1722, in extract_nsig
    ret = func([s])
          ^^^^^^^^^
  File "/home/1000i100/zsh-snap/youtube-dl/youtube_dl/jsinterp.py", line 1122, in resf
    ret, should_abort = self.interpret_statement(code.replace('\n', ' '), var_stack, allow_recursion - 1)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/1000i100/zsh-snap/youtube-dl/youtube_dl/jsinterp.py", line 249, in interpret_statement
    ret, should_ret = f(self, stmt, local_vars, allow_recursion, *args, **kwargs)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/1000i100/zsh-snap/youtube-dl/youtube_dl/jsinterp.py", line 495, in interpret_statement
    ret, should_return = self.interpret_statement(sub_stmt, local_vars, allow_recursion)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/1000i100/zsh-snap/youtube-dl/youtube_dl/jsinterp.py", line 249, in interpret_statement
    ret, should_ret = f(self, stmt, local_vars, allow_recursion, *args, **kwargs)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/1000i100/zsh-snap/youtube-dl/youtube_dl/jsinterp.py", line 711, in interpret_statement
    ret, should_abort = self.interpret_statement(sub_expr, local_vars, allow_recursion)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/1000i100/zsh-snap/youtube-dl/youtube_dl/jsinterp.py", line 249, in interpret_statement
    ret, should_ret = f(self, stmt, local_vars, allow_recursion, *args, **kwargs)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/1000i100/zsh-snap/youtube-dl/youtube_dl/jsinterp.py", line 750, in interpret_statement
    local_vars[m.group('out')] = self._operator(
                                 ^^^^^^^^^^^^^^^
  File "/home/1000i100/zsh-snap/youtube-dl/youtube_dl/jsinterp.py", line 441, in _operator
    right_val = self.interpret_expression(right_expr, local_vars, allow_recursion)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/1000i100/zsh-snap/youtube-dl/youtube_dl/jsinterp.py", line 1017, in interpret_expression
    ret, should_return = self.interpret_statement(expr, local_vars, allow_recursion)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/1000i100/zsh-snap/youtube-dl/youtube_dl/jsinterp.py", line 249, in interpret_statement
    ret, should_ret = f(self, stmt, local_vars, allow_recursion, *args, **kwargs)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/1000i100/zsh-snap/youtube-dl/youtube_dl/jsinterp.py", line 1001, in interpret_statement
    return eval_method(variable, member), should_return
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/1000i100/zsh-snap/youtube-dl/youtube_dl/jsinterp.py", line 883, in eval_method
    argvals = [
              ^
  File "/home/1000i100/zsh-snap/youtube-dl/youtube_dl/jsinterp.py", line 884, in <listcomp>
    self.interpret_expression(v, local_vars, allow_recursion)
  File "/home/1000i100/zsh-snap/youtube-dl/youtube_dl/jsinterp.py", line 1017, in interpret_expression
    ret, should_return = self.interpret_statement(expr, local_vars, allow_recursion)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/1000i100/zsh-snap/youtube-dl/youtube_dl/jsinterp.py", line 249, in interpret_statement
    ret, should_ret = f(self, stmt, local_vars, allow_recursion, *args, **kwargs)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/1000i100/zsh-snap/youtube-dl/youtube_dl/jsinterp.py", line 1001, in interpret_statement
    return eval_method(variable, member), should_return
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/1000i100/zsh-snap/youtube-dl/youtube_dl/jsinterp.py", line 928, in eval_method
    assertion(isinstance(obj, list), 'must be applied on a list')
  File "/home/1000i100/zsh-snap/youtube-dl/youtube_dl/jsinterp.py", line 851, in assertion
    raise self.Exception('{memb} {msg}'.format(**locals()), expr=expr)
youtube_dl.jsinterp.JSInterpreter.Exception: slice must be applied on a list in: 'a.slice(0,0)'; please report this issue on https://github.com/ytdl-org/youtube-dl/issues , using the appropriate issue template. Make sure you are using the latest version; see  https://github.com/ytdl-org/youtube-dl/#user-content-installation  on how to update. Be sure to call youtube-dl with the --verbose option and include the complete output.
)
WARNING: [youtube] 9JU8LTyEwBg: Unable to decode n-parameter: expect download to be blocked or throttled (slice must be applied on a list in: 'a.slice(0,0)'; please report this issue on https://github.com/ytdl-org/youtube-dl/issues , using the appropriate issue template. Make sure you are using the latest version; see  https://github.com/ytdl-org/youtube-dl/#user-content-installation  on how to update. Be sure to call youtube-dl with the --verbose option and include the complete output. Traceback (most recent call last):
  File "/home/1000i100/zsh-snap/youtube-dl/youtube_dl/extractor/youtube.py", line 1646, in _decrypt_nsig
    ret = extract_nsig(jsi, func_code)(n)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/1000i100/zsh-snap/youtube-dl/youtube_dl/extractor/youtube.py", line 1722, in extract_nsig
    ret = func([s])
          ^^^^^^^^^
  File "/home/1000i100/zsh-snap/youtube-dl/youtube_dl/jsinterp.py", line 1122, in resf
    ret, should_abort = self.interpret_statement(code.replace('\n', ' '), var_stack, allow_recursion - 1)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/1000i100/zsh-snap/youtube-dl/youtube_dl/jsinterp.py", line 249, in interpret_statement
    ret, should_ret = f(self, stmt, local_vars, allow_recursion, *args, **kwargs)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/1000i100/zsh-snap/youtube-dl/youtube_dl/jsinterp.py", line 495, in interpret_statement
    ret, should_return = self.interpret_statement(sub_stmt, local_vars, allow_recursion)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/1000i100/zsh-snap/youtube-dl/youtube_dl/jsinterp.py", line 249, in interpret_statement
    ret, should_ret = f(self, stmt, local_vars, allow_recursion, *args, **kwargs)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/1000i100/zsh-snap/youtube-dl/youtube_dl/jsinterp.py", line 711, in interpret_statement
    ret, should_abort = self.interpret_statement(sub_expr, local_vars, allow_recursion)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/1000i100/zsh-snap/youtube-dl/youtube_dl/jsinterp.py", line 249, in interpret_statement
    ret, should_ret = f(self, stmt, local_vars, allow_recursion, *args, **kwargs)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/1000i100/zsh-snap/youtube-dl/youtube_dl/jsinterp.py", line 750, in interpret_statement
    local_vars[m.group('out')] = self._operator(
                                 ^^^^^^^^^^^^^^^
  File "/home/1000i100/zsh-snap/youtube-dl/youtube_dl/jsinterp.py", line 441, in _operator
    right_val = self.interpret_expression(right_expr, local_vars, allow_recursion)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/1000i100/zsh-snap/youtube-dl/youtube_dl/jsinterp.py", line 1017, in interpret_expression
    ret, should_return = self.interpret_statement(expr, local_vars, allow_recursion)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/1000i100/zsh-snap/youtube-dl/youtube_dl/jsinterp.py", line 249, in interpret_statement
    ret, should_ret = f(self, stmt, local_vars, allow_recursion, *args, **kwargs)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/1000i100/zsh-snap/youtube-dl/youtube_dl/jsinterp.py", line 1001, in interpret_statement
    return eval_method(variable, member), should_return
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/1000i100/zsh-snap/youtube-dl/youtube_dl/jsinterp.py", line 883, in eval_method
    argvals = [
              ^
  File "/home/1000i100/zsh-snap/youtube-dl/youtube_dl/jsinterp.py", line 884, in <listcomp>
    self.interpret_expression(v, local_vars, allow_recursion)
  File "/home/1000i100/zsh-snap/youtube-dl/youtube_dl/jsinterp.py", line 1017, in interpret_expression
    ret, should_return = self.interpret_statement(expr, local_vars, allow_recursion)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/1000i100/zsh-snap/youtube-dl/youtube_dl/jsinterp.py", line 249, in interpret_statement
    ret, should_ret = f(self, stmt, local_vars, allow_recursion, *args, **kwargs)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/1000i100/zsh-snap/youtube-dl/youtube_dl/jsinterp.py", line 1001, in interpret_statement
    return eval_method(variable, member), should_return
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/1000i100/zsh-snap/youtube-dl/youtube_dl/jsinterp.py", line 928, in eval_method
    assertion(isinstance(obj, list), 'must be applied on a list')
  File "/home/1000i100/zsh-snap/youtube-dl/youtube_dl/jsinterp.py", line 851, in assertion
    raise self.Exception('{memb} {msg}'.format(**locals()), expr=expr)
youtube_dl.jsinterp.JSInterpreter.Exception: slice must be applied on a list in: 'a.slice(0,0)'; please report this issue on https://github.com/ytdl-org/youtube-dl/issues , using the appropriate issue template. Make sure you are using the latest version; see  https://github.com/ytdl-org/youtube-dl/#user-content-installation  on how to update. Be sure to call youtube-dl with the --verbose option and include the complete output.
)
[debug] Invoking downloader on 'https://rr4---sn-p5qs7nsk.googlevideo.com/videoplayback?expire=1722995087&ei=L32yZpepCrTUp-oPjeCCgAo&ip=2a0c%3Ae300%3A4%3A12%3A0%3A0%3A0%3A0&id=o-ANSGSgS0BF3mcstvhEIw5X93ocnEkJyLyy6gHDUIw8Ni&itag=251&source=youtube&requiressl=yes&xpc=EgVo2aDSNQ%3D%3D&mh=Gf&mm=31%2C26&mn=sn-p5qs7nsk%2Csn-vgqsrnld&ms=au%2Conr&mv=m&mvi=4&pl=46&bui=AXc671JpzbgSJRUKsSAWaqfBxlecdeYV7twJAg4TNc_d9GPXZ5JxKfq0t4cNm7YnQesbeU1gUHrObHM_&spc=Mv1m9iBQEI8kyHFOR07_36ngpFw4ErLshQ4TvgwzpvwMgrVSODtA8fintb05&vprv=1&svpuc=1&xtags=drc%3D1&mime=audio%2Fwebm&ns=bMCSC4WKgeuS6toxlOfLz6MQ&rqh=1&gir=yes&clen=30241293&dur=1766.501&lmt=1657252092559436&mt=1722973032&fvip=1&keepalive=yes&c=WEB&sefc=1&txp=5311224&n=qObFmM3nDXVWO2Dy&sparams=expire%2Cei%2Cip%2Cid%2Citag%2Csource%2Crequiressl%2Cxpc%2Cbui%2Cspc%2Cvprv%2Csvpuc%2Cxtags%2Cmime%2Cns%2Crqh%2Cgir%2Cclen%2Cdur%2Clmt&sig=AJfQdSswRQIhAOxCCQp0t5BwWGhIJCSha2owNcY3LFzqTqckRRsnRFehAiB-PdASbd9WWTxCHsae02pjC45pgv-WOAQeb_70ejotKw%3D%3D&lsparams=mh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl&lsig=AGtxev0wRAIgXwQpxjTl0psjG_Rmcm2gh7sBUcmyV3losdV0VtimoJgCIB8yI5A2NMk4ouGQ5FUSL9StBD0_A7eb0iH3WmUMOWrk'
[dashsegments] Total fragments: 3
[download] Destination: Scientisme ... une science cléricale  Postulat #1.07-9JU8LTyEwBg.webm
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 1 (attempt 1 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 1 (attempt 2 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 1 (attempt 3 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 1 (attempt 4 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 1 (attempt 5 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 1 (attempt 6 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 1 (attempt 7 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 1 (attempt 8 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 1 (attempt 9 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 1 (attempt 10 of 10)...
ERROR: giving up after 10 fragment retries
  File "/home/1000i100/zsh-snap/youtube-dl/bin/youtube-dl", line 6, in <module>
    youtube_dl.main()
  File "/home/1000i100/zsh-snap/youtube-dl/youtube_dl/__init__.py", line 477, in main
    _real_main(argv)
  File "/home/1000i100/zsh-snap/youtube-dl/youtube_dl/__init__.py", line 467, in _real_main
    retcode = ydl.download(all_urls)
  File "/home/1000i100/zsh-snap/youtube-dl/youtube_dl/YoutubeDL.py", line 2253, in download
    res = self.extract_info(
  File "/home/1000i100/zsh-snap/youtube-dl/youtube_dl/YoutubeDL.py", line 868, in extract_info
    return self.__extract_info(url, ie, download, extra_info, process)
  File "/home/1000i100/zsh-snap/youtube-dl/youtube_dl/YoutubeDL.py", line 875, in wrapper
    return func(self, *args, **kwargs)
  File "/home/1000i100/zsh-snap/youtube-dl/youtube_dl/YoutubeDL.py", line 982, in __extract_info
    return self.process_ie_result(ie_result, download, extra_info)
  File "/home/1000i100/zsh-snap/youtube-dl/youtube_dl/YoutubeDL.py", line 1072, in process_ie_result
    return self.__process_playlist(ie_result, download)
  File "/home/1000i100/zsh-snap/youtube-dl/youtube_dl/YoutubeDL.py", line 1206, in __process_playlist
    entry_result = self.__process_iterable_entry(entry, download, extra)
  File "/home/1000i100/zsh-snap/youtube-dl/youtube_dl/YoutubeDL.py", line 875, in wrapper
    return func(self, *args, **kwargs)
  File "/home/1000i100/zsh-snap/youtube-dl/youtube_dl/YoutubeDL.py", line 1215, in __process_iterable_entry
    return self.process_ie_result(
  File "/home/1000i100/zsh-snap/youtube-dl/youtube_dl/YoutubeDL.py", line 1020, in process_ie_result
    return self.extract_info(ie_result['url'],
  File "/home/1000i100/zsh-snap/youtube-dl/youtube_dl/YoutubeDL.py", line 868, in extract_info
    return self.__extract_info(url, ie, download, extra_info, process)
  File "/home/1000i100/zsh-snap/youtube-dl/youtube_dl/YoutubeDL.py", line 875, in wrapper
    return func(self, *args, **kwargs)
  File "/home/1000i100/zsh-snap/youtube-dl/youtube_dl/YoutubeDL.py", line 982, in __extract_info
    return self.process_ie_result(ie_result, download, extra_info)
  File "/home/1000i100/zsh-snap/youtube-dl/youtube_dl/YoutubeDL.py", line 1016, in process_ie_result
    return self.process_video_result(ie_result, download=download)
  File "/home/1000i100/zsh-snap/youtube-dl/youtube_dl/YoutubeDL.py", line 1860, in process_video_result
    self.process_info(new_info)
  File "/home/1000i100/zsh-snap/youtube-dl/youtube_dl/YoutubeDL.py", line 138, in wrapper
    return func(self, *args, **kwargs)
  File "/home/1000i100/zsh-snap/youtube-dl/youtube_dl/YoutubeDL.py", line 2158, in process_info
    success = dl(filename, info_dict)
  File "/home/1000i100/zsh-snap/youtube-dl/youtube_dl/YoutubeDL.py", line 2088, in dl
    return fd.download(name, new_info)
  File "/home/1000i100/zsh-snap/youtube-dl/youtube_dl/downloader/common.py", line 380, in download
    return self.real_download(filename, info_dict)
  File "/home/1000i100/zsh-snap/youtube-dl/youtube_dl/downloader/dash.py", line 78, in real_download
    self.report_error('giving up after %s fragment retries' % count)
  File "/home/1000i100/zsh-snap/youtube-dl/youtube_dl/downloader/common.py", line 175, in report_error
    self.ydl.report_error(*args, **kargs)
  File "/home/1000i100/zsh-snap/youtube-dl/youtube_dl/YoutubeDL.py", line 678, in report_error
    self.trouble(*args, **kwargs)
  File "/home/1000i100/zsh-snap/youtube-dl/youtube_dl/YoutubeDL.py", line 630, in trouble
    tb_data = traceback.format_list(traceback.extract_stack())
iactix commented 1 month ago

Will this keep breaking every two days because youtube combats it, or is this just sloppy code when something expects a list but it happens to not be one?

xihajun commented 1 month ago

Will this keep breaking every two days because youtube combats it, or is this just sloppy code when something expects a list but it happens to not be one?

want to know that as well, is there any stable solution? does google provide any api to download?

tansy commented 1 month ago

Will this keep breaking every two days because youtube combats it?

Good question. And answer we can only guess, and the guess is yes.

GregoriusT commented 1 month ago

it works more often now but I am still getting this dang 403 error on some videos

[debug] System config: []
[debug] User config: ['--no-mtime', '--match-filter', '!is_live', '--retries', 'infinite', '--fragment-retries', '3', '--skip-unavailable-fragments', '--restrict-filenames', '-i', '-o', '/home/gregorius/home/pending/videos/%(title)s___%(id)s.webm', '-f', '(bestvideo[height<=360]+worstaudio/best[height<=360])[protocol!=http_dash_segments][container!^=dash]', '--console-title', '--hls-prefer-native', '--no-cache-dir', '--http-chunk-size', '100M', '--cookies', '/home/gregorius/home/scripts/video/youtube-dl-cookies']
[debug] Custom config: []
[debug] Command-line args: ['https://www.youtube.com/watch?v=lrrS6a7jr3A', '-vf', '(242+249/242+250/242+171/242+251)/(243+249/243+250/243+171/243+251)/18', '--no-playlist', '-o', '/home/gregorius/home/scripts/video/TEST_NORMAL_%(title)s___%(id)s.webm']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, pref UTF-8
[debug] youtube-dl version 2021.12.17
[debug] Single file build
[debug] Python 3.10.12 (CPython x86_64 64bit) - Linux-5.15.0-112-generic-x86_64-with-glibc2.35 - OpenSSL 3.0.2 15 Mar 2022 - glibc 2.35
[debug] exe versions: ffmpeg 4.4.2, ffprobe 4.4.2, rtmpdump 2.4
[debug] Proxy map: {}
[youtube] lrrS6a7jr3A: Downloading webpage
[youtube] Downloading just video lrrS6a7jr3A because of --no-playlist
[youtube] lrrS6a7jr3A: Downloading player b12cc44b
[debug] [youtube] Decrypted nsig NILeF1oRQoLyCWV3 => 9Bd0xlQNg01rRw
[debug] [youtube] Decrypted nsig M0LcH457IghNxyM- => 3DSpEhXsYQ0ptw
[debug] Invoking downloader on 'https://rr2---sn-4g5e6nz7.googlevideo.com/videoplayback?expire=1723004797&ei=HaOyZrL0EoO06dsPtNKiqAs&ip=47.69.169.114&id=o-AM4a1IMnptO1Iz4H78Txu8VHqMRohVbUiOGUDJ1Ri6Ez&itag=242&aitags=133%2C134%2C135%2C136%2C160%2C242%2C243%2C244%2C247%2C278%2C298%2C299%2C302%2C303&source=youtube&requiressl=yes&xpc=EgVo2aDSNQ%3D%3D&mh=kU&mm=31%2C29&mn=sn-4g5e6nz7%2Csn-4g5edn6r&ms=au%2Crdu&mv=m&mvi=2&pl=19&initcwndbps=2112500&bui=AXc671KR1ZUGhA58Uz7iAf6xvr5jWfZMQWXrpERlhKg0FUVpXvmQ73e4s-yIv_seM3raukI6awIDof0v&spc=Mv1m9sVkQ9EsUroFKWvu-kQF5C4PHBWO6HImy4UID4u_BUmGoIQDFYt-Kgt5&vprv=1&svpuc=1&mime=video%2Fwebm&ns=QVOAow5E_bhy1VHLa4hZq-YQ&rqh=1&gir=yes&clen=5399293&dur=632.298&lmt=1722408844169274&mt=1722982853&fvip=4&keepalive=yes&c=WEB&sefc=1&txp=630F224&n=3DSpEhXsYQ0ptw&sparams=expire%2Cei%2Cip%2Cid%2Caitags%2Csource%2Crequiressl%2Cxpc%2Cbui%2Cspc%2Cvprv%2Csvpuc%2Cmime%2Cns%2Crqh%2Cgir%2Cclen%2Cdur%2Clmt&sig=AJfQdSswRQIgOz1YyVveoHpBPbirJ8Ey906Zk9KAZjAMC_F8NGCVZYQCIQDL8Uc5GdzRrzeHH29nBdwrGi5m0v8mqg1PPJwxPzgQiA%3D%3D&lsparams=mh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl%2Cinitcwndbps&lsig=AGtxev0wRQIge1V4gX3DxGUh9laN-oa4EW2QVQm4oPg9LU5vH6kv5gcCIQCL2PgVf3t6gq0F3UOy9wVESaO5qIs0Hml0FQQ3CIEaog%3D%3D'
[dashsegments] Total fragments: 1
[download] Destination: /home/gregorius/home/scripts/video/TEST_NORMAL_New_Chess_Website_And_It_Looks_GOOD___lrrS6a7jr3A.f242.webm
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 1 (attempt 1 of 3)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 1 (attempt 2 of 3)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 1 (attempt 3 of 3)...
ERROR: giving up after 3 fragment retries
  File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/home/gregorius/home/scripts/video/youtube-dl-normal/youtube-dl/__main__.py", line 19, in <module>
    youtube_dl.main()
  File "/home/gregorius/home/scripts/video/youtube-dl-normal/youtube-dl/youtube_dl/__init__.py", line 477, in main
    _real_main(argv)
  File "/home/gregorius/home/scripts/video/youtube-dl-normal/youtube-dl/youtube_dl/__init__.py", line 467, in _real_main
    retcode = ydl.download(all_urls)
  File "/home/gregorius/home/scripts/video/youtube-dl-normal/youtube-dl/youtube_dl/YoutubeDL.py", line 2253, in download
    res = self.extract_info(
  File "/home/gregorius/home/scripts/video/youtube-dl-normal/youtube-dl/youtube_dl/YoutubeDL.py", line 868, in extract_info
    return self.__extract_info(url, ie, download, extra_info, process)
  File "/home/gregorius/home/scripts/video/youtube-dl-normal/youtube-dl/youtube_dl/YoutubeDL.py", line 875, in wrapper
    return func(self, *args, **kwargs)
  File "/home/gregorius/home/scripts/video/youtube-dl-normal/youtube-dl/youtube_dl/YoutubeDL.py", line 982, in __extract_info
    return self.process_ie_result(ie_result, download, extra_info)
  File "/home/gregorius/home/scripts/video/youtube-dl-normal/youtube-dl/youtube_dl/YoutubeDL.py", line 1016, in process_ie_result
    return self.process_video_result(ie_result, download=download)
  File "/home/gregorius/home/scripts/video/youtube-dl-normal/youtube-dl/youtube_dl/YoutubeDL.py", line 1860, in process_video_result
    self.process_info(new_info)
  File "/home/gregorius/home/scripts/video/youtube-dl-normal/youtube-dl/youtube_dl/YoutubeDL.py", line 138, in wrapper
    return func(self, *args, **kwargs)
  File "/home/gregorius/home/scripts/video/youtube-dl-normal/youtube-dl/youtube_dl/YoutubeDL.py", line 2152, in process_info
    partial_success = dl(fname, new_info)
  File "/home/gregorius/home/scripts/video/youtube-dl-normal/youtube-dl/youtube_dl/YoutubeDL.py", line 2088, in dl
    return fd.download(name, new_info)
  File "/home/gregorius/home/scripts/video/youtube-dl-normal/youtube-dl/youtube_dl/downloader/common.py", line 380, in download
    return self.real_download(filename, info_dict)
  File "/home/gregorius/home/scripts/video/youtube-dl-normal/youtube-dl/youtube_dl/downloader/dash.py", line 78, in real_download
    self.report_error('giving up after %s fragment retries' % count)
  File "/home/gregorius/home/scripts/video/youtube-dl-normal/youtube-dl/youtube_dl/downloader/common.py", line 175, in report_error
    self.ydl.report_error(*args, **kargs)
  File "/home/gregorius/home/scripts/video/youtube-dl-normal/youtube-dl/youtube_dl/YoutubeDL.py", line 678, in report_error
    self.trouble(*args, **kwargs)
  File "/home/gregorius/home/scripts/video/youtube-dl-normal/youtube-dl/youtube_dl/YoutubeDL.py", line 630, in trouble
    tb_data = traceback.format_list(traceback.extract_stack())

[debug] Invoking downloader on 'https://rr2---sn-4g5e6nz7.googlevideo.com/videoplayback?expire=1723004797&ei=HaOyZrL0EoO06dsPtNKiqAs&ip=47.69.169.114&id=o-AM4a1IMnptO1Iz4H78Txu8VHqMRohVbUiOGUDJ1Ri6Ez&itag=249&source=youtube&requiressl=yes&xpc=EgVo2aDSNQ%3D%3D&mh=kU&mm=31%2C29&mn=sn-4g5e6nz7%2Csn-4g5edn6r&ms=au%2Crdu&mv=m&mvi=2&pl=19&initcwndbps=2112500&bui=AXc671KR1ZUGhA58Uz7iAf6xvr5jWfZMQWXrpERlhKg0FUVpXvmQ73e4s-yIv_seM3raukI6awIDof0v&spc=Mv1m9sVkQ9EsUroFKWvu-kQF5C4PHBWO6HImy4UID4u_BUmGoIQDFYt-Kgt5&vprv=1&svpuc=1&mime=audio%2Fwebm&ns=QVOAow5E_bhy1VHLa4hZq-YQ&rqh=1&gir=yes&clen=3985647&dur=632.321&lmt=1722408902825500&mt=1722982853&fvip=4&keepalive=yes&c=WEB&sefc=1&txp=6308224&n=3DSpEhXsYQ0ptw&sparams=expire%2Cei%2Cip%2Cid%2Citag%2Csource%2Crequiressl%2Cxpc%2Cbui%2Cspc%2Cvprv%2Csvpuc%2Cmime%2Cns%2Crqh%2Cgir%2Cclen%2Cdur%2Clmt&sig=AJfQdSswRAIgRIyoqirrzgBT2IRSNseX8zPz5OZZ72Z-Swkid6whh9sCIG1heux_m5W8goi8EekmFe6vy9xXaEqUaaKYEX84qZPu&lsparams=mh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl%2Cinitcwndbps&lsig=AGtxev0wRQIge1V4gX3DxGUh9laN-oa4EW2QVQm4oPg9LU5vH6kv5gcCIQCL2PgVf3t6gq0F3UOy9wVESaO5qIs0Hml0FQQ3CIEaog%3D%3D'
[dashsegments] Total fragments: 1
[download] Destination: /home/gregorius/home/scripts/video/TEST_NORMAL_New_Chess_Website_And_It_Looks_GOOD___lrrS6a7jr3A.f249.webm
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 1 (attempt 1 of 3)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 1 (attempt 2 of 3)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 1 (attempt 3 of 3)...
ERROR: giving up after 3 fragment retries
  File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/home/gregorius/home/scripts/video/youtube-dl-normal/youtube-dl/__main__.py", line 19, in <module>
    youtube_dl.main()
  File "/home/gregorius/home/scripts/video/youtube-dl-normal/youtube-dl/youtube_dl/__init__.py", line 477, in main
    _real_main(argv)
  File "/home/gregorius/home/scripts/video/youtube-dl-normal/youtube-dl/youtube_dl/__init__.py", line 467, in _real_main
    retcode = ydl.download(all_urls)
  File "/home/gregorius/home/scripts/video/youtube-dl-normal/youtube-dl/youtube_dl/YoutubeDL.py", line 2253, in download
    res = self.extract_info(
  File "/home/gregorius/home/scripts/video/youtube-dl-normal/youtube-dl/youtube_dl/YoutubeDL.py", line 868, in extract_info
    return self.__extract_info(url, ie, download, extra_info, process)
  File "/home/gregorius/home/scripts/video/youtube-dl-normal/youtube-dl/youtube_dl/YoutubeDL.py", line 875, in wrapper
    return func(self, *args, **kwargs)
  File "/home/gregorius/home/scripts/video/youtube-dl-normal/youtube-dl/youtube_dl/YoutubeDL.py", line 982, in __extract_info
    return self.process_ie_result(ie_result, download, extra_info)
  File "/home/gregorius/home/scripts/video/youtube-dl-normal/youtube-dl/youtube_dl/YoutubeDL.py", line 1016, in process_ie_result
    return self.process_video_result(ie_result, download=download)
  File "/home/gregorius/home/scripts/video/youtube-dl-normal/youtube-dl/youtube_dl/YoutubeDL.py", line 1860, in process_video_result
    self.process_info(new_info)
  File "/home/gregorius/home/scripts/video/youtube-dl-normal/youtube-dl/youtube_dl/YoutubeDL.py", line 138, in wrapper
    return func(self, *args, **kwargs)
  File "/home/gregorius/home/scripts/video/youtube-dl-normal/youtube-dl/youtube_dl/YoutubeDL.py", line 2152, in process_info
    partial_success = dl(fname, new_info)
  File "/home/gregorius/home/scripts/video/youtube-dl-normal/youtube-dl/youtube_dl/YoutubeDL.py", line 2088, in dl
    return fd.download(name, new_info)
  File "/home/gregorius/home/scripts/video/youtube-dl-normal/youtube-dl/youtube_dl/downloader/common.py", line 380, in download
    return self.real_download(filename, info_dict)
  File "/home/gregorius/home/scripts/video/youtube-dl-normal/youtube-dl/youtube_dl/downloader/dash.py", line 78, in real_download
    self.report_error('giving up after %s fragment retries' % count)
  File "/home/gregorius/home/scripts/video/youtube-dl-normal/youtube-dl/youtube_dl/downloader/common.py", line 175, in report_error
    self.ydl.report_error(*args, **kargs)
  File "/home/gregorius/home/scripts/video/youtube-dl-normal/youtube-dl/youtube_dl/YoutubeDL.py", line 678, in report_error
    self.trouble(*args, **kwargs)
  File "/home/gregorius/home/scripts/video/youtube-dl-normal/youtube-dl/youtube_dl/YoutubeDL.py", line 630, in trouble
    tb_data = traceback.format_list(traceback.extract_stack())
dirkf commented 1 month ago

There are two targets here, basically because yt-dl is by design not Chrome, nor even Firefox.

  1. We need to extract the code from the YT JS player that runs the n-parameter challenge. Essentially, like our code, the YT code has to get the n query parameter value out of the URL, pass it the challenge function, and set it back in the URL. From time to time YT changes the way that code is called, which makes it difficult to find reliably. In the commit above I reworked the search pattern, but it is always going to be easy for a YT intern or LLM action to produce another circumlocution that we don't recognise. From the previous update, we also have a fallback that tries to find the challenge code directly rather than finding its caller. This would have worked with all (I think) the previous players and so should be more resilient.

  2. We need to execute the extracted challenge code without a browser (or external JS platform, Deno say). The program has a (HN-worthy) toy JS interpreter that relies heavily on the similarities between JS and Python. This has gradually acquired more, and more accurate, functionality, but generalisations beyond what any player has required before are more only likely to have been implemented if they're easy.

The issue here is a good example. The interpreter has long supported the JS .slice() method (to make a subsequence), but (a) only for instances of Array (modelled by list) and (b) only with a single argument from. The problem player broke the assumptions on which the implementation relied. This is not so much sloppy coding as coding that was as good as it had to be when it wasn't being paid for (obviously not as good as the code delivered by ClownStrike, say).

To implement the method (more) properly, we have to observe the following:

In this case, more of the fix is about fixing the original assumptions than fixing the slicing.

skjsjslee commented 1 month ago

I suppose the empire is striking back lately. Thank you for the fix.

dirkf commented 1 month ago

Also thanks to @seproDev for the original yt-dlp fix and accompanying extensive test suite.

Rekrullurker commented 1 month ago

I'm using the latest nightly build and I'm getting the 403 error on various videos. Strangely, it always seems to happen on the videos I actually WANT to download, while pretty much every other video I try just as a test, works fine.

I've compared the formats available, tried downloading specific formats, etc, and there doesn't seem to be any pattern to it.

Right now, it seems completely incapable of downloading these two URLs;

https://www.youtube.com/watch?v=_qMlE4XHs4c https://www.youtube.com/watch?v=oH6Q_2f0aCI

Here's the part I don't get: The last time I updated the version of yt-dlp (backported to my old system by nicolaasjan, was on 7/31/24, and that version has no trouble downloading the same videos. In fact, I don't think I've seen a single 403 error while using it.

I prefer youtube-dl though. Mostly because it runs faster.

dirkf commented 1 month ago

As always, try some different network connection.

Despite finding the poToken experiment (the ignoring ... bit is a lie), I seem to be able to get those two videos. One relevant change in this WIP is this 

-                'clientVersion': '2.20201021.03.00',
+                'clientVersion': '2.20220801.00.00',

but we are getting the web data from the web page, not the API. Also, isn't 2022... (the Web client version from current yt-dlp) quite a long internet-time ago?

python3 -m youtube_dl -v --test '_qMlE4XHs4c' ```console $ python3 -m youtube_dl -v --test '_qMlE4XHs4c' [debug] System config: ['--prefer-ffmpeg'] [debug] User config: [] [debug] Custom config: [] [debug] Command-line args: ['-v', '--test', '_qMlE4XHs4c'] [debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8 [debug] youtube-dl version 2021.12.17 [debug] Git HEAD: c5098961b [debug] Python 3.5.2 (CPython i686 32bit) - Linux-4.4.0-210-generic-i686-with-Ubuntu-16.04-xenial - OpenSSL 1.0.2n 7 Dec 2017 - glibc 2.9 [debug] exe versions: avconv 4.3, avprobe 4.3, ffmpeg 4.3, ffprobe 4.3 [debug] Proxy map: {} [youtube] _qMlE4XHs4c: Downloading webpage WARNING: [youtube] Webpage contains broken formats (poToken experiment detected). Ignoring initial player response [debug] [youtube] Decrypted nsig _Jnzi9V_qgvpmLOkY17f => tGYq8UIxZctKSQ [debug] [youtube] Decrypted nsig S1B1gt9aLz-hOttzdFib => ugRJsjwTRHVPsQ [debug] Default format spec: bestvideo+bestaudio/best [debug] Invoking downloader on 'https://rr5---sn-cu-aigsl.googlevideo.com/videoplayback?svpuc=1&id=o-APV8DC6p2x_7E-nixviWVHn8dUd8daHTMaoKwohuFrp5&mv=m&sefc=1&mt=1723112713&mm=31%2C29&lmt=1722810022449989&itag=137&gir=yes&expire=1723134755&requiressl=yes&ns=pN9no3PuzE8KRcXLwbY8vpcQ&pl=25&mime=video%2Fmp4&ip=46.208.6.25&n=ugRJsjwTRHVPsQ&sparams=expire%2Cei%2Cip%2Cid%2Caitags%2Csource%2Crequiressl%2Cxpc%2Cbui%2Cspc%2Cvprv%2Csvpuc%2Cmime%2Cns%2Crqh%2Cgir%2Cclen%2Cdur%2Clmt&initcwndbps=1821250&mn=sn-cu-aigsl%2Csn-cu-c9id&c=WEB&bui=AXc671LwNQve8HarSKWW-cuZ8mQabFPU1TZhOgzTt4rxpbP81iUU-uT9Zk16XOFJn_awmT6P2DrSIe0M&mh=ua&mvi=5&spc=Mv1m9u0LCjUQSo3aYtcmyAV2rTTjGFrpMEmkXLPNvXsnJqEFe3-nPabgGdgt&xpc=EgVo2aDSNQ%3D%3D&lsparams=mh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl%2Cinitcwndbps&fvip=1&dur=2315.566&keepalive=yes&source=youtube&vprv=1&clen=301152641&lsig=AGtxev0wRgIhAKJ1k4YMouIdQoOx3aVcjfSP0P5HizcpkD-x7gp2pEGyAiEA9-H27z8sJZelkZ5UNdIVt5Wdh86YPidaftEqTAz77Xo%3D&rqh=1&ei=w560Zq_2FM6UhcIPk9nbgAM&txp=4432434&sig=AJfQdSswRAIgSktQ9xpdLPBMzR51teJUECyCxm8_9PdM-KJVARJGzkUCIDUwnCiMM1aUFFtXZdkkwDLGCRcXSkGtI_cPCXg2tF-k&ms=au%2Crdu&aitags=133%2C134%2C135%2C136%2C137%2C160%2C242%2C243%2C244%2C247%2C248%2C278' [dashsegments] Total fragments: 1 [download] Destination: First Time Watching NEAR DARK Reaction... I LOVED IT-_qMlE4XHs4c.f137.mp4 [download] 100% of 10.00KiB in 00:00 [debug] Invoking downloader on 'https://rr5---sn-cu-aigsl.googlevideo.com/videoplayback?svpuc=1&id=o-APV8DC6p2x_7E-nixviWVHn8dUd8daHTMaoKwohuFrp5&mv=m&sefc=1&mt=1723112713&mm=31%2C29&lmt=1722809423829645&itag=140&rqh=1&gir=yes&expire=1723134755&requiressl=yes&ns=pN9no3PuzE8KRcXLwbY8vpcQ&pl=25&mime=audio%2Fmp4&ip=46.208.6.25&n=ugRJsjwTRHVPsQ&sparams=expire%2Cei%2Cip%2Cid%2Citag%2Csource%2Crequiressl%2Cxpc%2Cbui%2Cspc%2Cvprv%2Csvpuc%2Cxtags%2Cmime%2Cns%2Crqh%2Cgir%2Cclen%2Cdur%2Clmt&initcwndbps=1821250&mn=sn-cu-aigsl%2Csn-cu-c9id&c=WEB&bui=AXc671LwNQve8HarSKWW-cuZ8mQabFPU1TZhOgzTt4rxpbP81iUU-uT9Zk16XOFJn_awmT6P2DrSIe0M&mh=ua&mvi=5&spc=Mv1m9u0LCjUQSo3aYtcmyAV2rTTjGFrpMEmkXLPNvXsnJqEFe3-nPabgGdgt&xpc=EgVo2aDSNQ%3D%3D&lsparams=mh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl%2Cinitcwndbps&fvip=1&dur=2315.633&keepalive=yes&source=youtube&vprv=1&clen=37477193&lsig=AGtxev0wRgIhAKJ1k4YMouIdQoOx3aVcjfSP0P5HizcpkD-x7gp2pEGyAiEA9-H27z8sJZelkZ5UNdIVt5Wdh86YPidaftEqTAz77Xo%3D&xtags=drc%3D1&ei=w560Zq_2FM6UhcIPk9nbgAM&txp=4432434&sig=AJfQdSswRQIhAN9Igizeyd0hfaIM4ATv8dismLlOYevhO7fhvkRmGhF1AiApSqimXprt2NYDjeVTF4uAZua3Gd7DBdusfblgiBjv-A%3D%3D&ms=au%2Crdu' [dashsegments] Total fragments: 1 [download] Destination: First Time Watching NEAR DARK Reaction... I LOVED IT-_qMlE4XHs4c.m4a.f140-drc.m4a [download] 100% of 10.00KiB in 00:00 [ffmpeg] Merging formats into "First Time Watching NEAR DARK Reaction... I LOVED IT-_qMlE4XHs4c.mp4" [debug] ffmpeg command line: ffmpeg -y -loglevel repeat+info -i 'file:First Time Watching NEAR DARK Reaction... I LOVED IT-_qMlE4XHs4c.f137.mp4' -i 'file:First Time Watching NEAR DARK Reaction... I LOVED IT-_qMlE4XHs4c.m4a.f140-drc.m4a' -c copy -map 0:v:0 -map 1:a:0 'file:First Time Watching NEAR DARK Reaction... I LOVED IT-_qMlE4XHs4c.temp.mp4' Deleting original file First Time Watching NEAR DARK Reaction... I LOVED IT-_qMlE4XHs4c.f137.mp4 (pass -k to keep) Deleting original file First Time Watching NEAR DARK Reaction... I LOVED IT-_qMlE4XHs4c.m4a.f140-drc.m4a (pass -k to keep) $ ```
python -m youtube_dl -v --test 'oH6Q_2f0aCI' ```console $ python -m youtube_dl -v --test 'oH6Q_2f0aCI' [debug] System config: [u'--prefer-ffmpeg'] [debug] User config: [] [debug] Custom config: [] [debug] Command-line args: [u'-v', u'--test', u'oH6Q_2f0aCI'] [debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8 [debug] youtube-dl version 2021.12.17 [debug] Git HEAD: c5098961b [debug] Python 2.7.18 (CPython i686 32bit) - Linux-4.4.0-210-generic-i686-with-Ubuntu-16.04-xenial - OpenSSL 1.1.1w 11 Sep 2023 - glibc 2.15 [debug] exe versions: avconv 4.3, avprobe 4.3, ffmpeg 4.3, ffprobe 4.3 [debug] Proxy map: {} [youtube] oH6Q_2f0aCI: Downloading webpage WARNING: [youtube] Webpage contains broken formats (poToken experiment detected). Ignoring initial player response [debug] [youtube] Decrypted nsig ryGtrmgBQzMA4hpxTGgV => 6BSFgxaHA4sZHA [debug] [youtube] Decrypted nsig 10qSpw4dzAR36rSm2EUR => 9qmux2cRH3oDgw [debug] Default format spec: bestvideo+bestaudio/best [debug] Invoking downloader on u'https://rr3---sn-cu-aigss.googlevideo.com/videoplayback?sparams=expire%2Cei%2Cip%2Cid%2Caitags%2Csource%2Crequiressl%2Cxpc%2Cbui%2Cspc%2Cvprv%2Csvpuc%2Cmime%2Cns%2Crqh%2Cgir%2Cclen%2Cdur%2Clmt&ei=UaC0Zu7NI4uJp-oPzrfGoQU&ip=46.208.6.25&clen=294572127&spc=Mv1m9gR0Jkubnn7cp25OS0kCZ-OoxNn7GMiKHf3QkaqTHm1-SZ3hmVw3rrfS&id=o-AGtdISe5CoefE4gubBP4iDSBVvPPoXt_pyTXYzxJzaZS&txp=4432434&svpuc=1&aitags=133%2C134%2C135%2C136%2C137%2C160%2C242%2C243%2C244%2C247%2C248%2C278&gir=yes&xpc=EgVo2aDSNQ%3D%3D&requiressl=yes&keepalive=yes&source=youtube&mv=m&sig=AJfQdSswRQIhAMvSVuaQsNUsI8hgdFG9CEPoO3Y9N-iFs3j7dsuE1PIfAiBEdtNkjt0SU8A4AQGvSpgVAXdawTjHiDRQTU6rCN94og%3D%3D&dur=2026.023&ns=dT9veM7VjtTk-DIqsru4sT8Q&initcwndbps=1583750&vprv=1&lsig=AGtxev0wRQIgd0kaBb7vPsefKsIVE7XUgxBp6bdaHdJxV4YAM-tYH_MCIQDeQGcIU9WM3bs8NmaLarvv2AtMNnpXN2V3754Zmcab6w%3D%3D&lsparams=mh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl%2Cinitcwndbps&lmt=1722539947587762&c=WEB&sefc=1&bui=AXc671LC-gdKMR_Q2JTF3Hikuey3NB-rFJ8IBIko0DwpDzLXntY-lODLdhdnvF1rreM1ix54_vNjvtp9&mime=video%2Fmp4&fvip=3&rqh=1&itag=137&mm=31%2C29&mn=sn-cu-aigss%2Csn-cu-c9id&mh=DT&n=9qmux2cRH3oDgw&mt=1723113190&expire=1723135153&pl=25&ms=au%2Crdu&mvi=3' [dashsegments] Total fragments: 1 [download] Destination: BUFFY the VAMPIRE SLAYER already has me HOOKED.-oH6Q_2f0aCI.f137.mp4 [download] 100% of 10.00KiB in 00:00 [debug] Invoking downloader on u'https://rr3---sn-cu-aigss.googlevideo.com/videoplayback?sparams=expire%2Cei%2Cip%2Cid%2Citag%2Csource%2Crequiressl%2Cxpc%2Cbui%2Cspc%2Cvprv%2Csvpuc%2Cxtags%2Cmime%2Cns%2Crqh%2Cgir%2Cclen%2Cdur%2Clmt&ei=UaC0Zu7NI4uJp-oPzrfGoQU&ip=46.208.6.25&clen=32790612&spc=Mv1m9gR0Jkubnn7cp25OS0kCZ-OoxNn7GMiKHf3QkaqTHm1-SZ3hmVw3rrfS&id=o-AGtdISe5CoefE4gubBP4iDSBVvPPoXt_pyTXYzxJzaZS&txp=4432434&svpuc=1&gir=yes&xpc=EgVo2aDSNQ%3D%3D&requiressl=yes&keepalive=yes&source=youtube&mv=m&sig=AJfQdSswRgIhAJjANxLl3CmnVaTc2eSaYTP247ho1NqGzoBytkHXqhu0AiEA78CJ9mB5nb-ebDD2BcrvfjglKMzhKBIUMevlIqG23Ps%3D&xtags=drc%3D1&dur=2026.080&ns=dT9veM7VjtTk-DIqsru4sT8Q&initcwndbps=1583750&vprv=1&lsig=AGtxev0wRQIgd0kaBb7vPsefKsIVE7XUgxBp6bdaHdJxV4YAM-tYH_MCIQDeQGcIU9WM3bs8NmaLarvv2AtMNnpXN2V3754Zmcab6w%3D%3D&lsparams=mh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl%2Cinitcwndbps&lmt=1722538658500076&c=WEB&sefc=1&bui=AXc671LC-gdKMR_Q2JTF3Hikuey3NB-rFJ8IBIko0DwpDzLXntY-lODLdhdnvF1rreM1ix54_vNjvtp9&mime=audio%2Fmp4&fvip=3&rqh=1&itag=140&mm=31%2C29&mn=sn-cu-aigss%2Csn-cu-c9id&mh=DT&n=9qmux2cRH3oDgw&mt=1723113190&expire=1723135153&pl=25&ms=au%2Crdu&mvi=3' [dashsegments] Total fragments: 1 [download] Destination: BUFFY the VAMPIRE SLAYER already has me HOOKED.-oH6Q_2f0aCI.m4a.f140-drc.m4a [download] 100% of 10.00KiB in 00:00 [ffmpeg] Merging formats into "BUFFY the VAMPIRE SLAYER already has me HOOKED.-oH6Q_2f0aCI.mp4" [debug] ffmpeg command line: ffmpeg -y -loglevel 'repeat+info' -i 'file:BUFFY the VAMPIRE SLAYER already has me HOOKED.-oH6Q_2f0aCI.f137.mp4' -i 'file:BUFFY the VAMPIRE SLAYER already has me HOOKED.-oH6Q_2f0aCI.m4a.f140-drc.m4a' -c copy -map '0:v:0' -map '1:a:0' 'file:BUFFY the VAMPIRE SLAYER already has me HOOKED.-oH6Q_2f0aCI.temp.mp4' Deleting original file BUFFY the VAMPIRE SLAYER already has me HOOKED.-oH6Q_2f0aCI.f137.mp4 (pass -k to keep) Deleting original file BUFFY the VAMPIRE SLAYER already has me HOOKED.-oH6Q_2f0aCI.m4a.f140-drc.m4a (pass -k to keep) $ ```
bashonly commented 1 month ago

@dirkf --test is not sufficient to check for these 403 errors. If the videoplayback URL needs a pot param, the first 10KB will download just fine; the 403 error is encountered shortly after 1MB has been downloaded (as noted in the description of https://github.com/yt-dlp/yt-dlp/pull/10456)

Rekrullurker commented 1 month ago

Call me dumb, but I'm not sure what you mean by trying different network connections.

As for "testing", what I meant was that I picked various other videos and tried downloading them as a test, and it seems to work with videos that I don't care about, but when I actually WANT to download a video, all I get are 403 errors.

For the record, I still can't download the two URLs in my previous post. Here is the result of using --verbose;

[debug] System config: [] [debug] User config: [] [debug] Custom config: [] [debug] Command-line args: ['--verbose', 'https://www.youtube.com/watch?v=_qMlE4XHs4c'] [debug] Encodings: locale cp1252, fs mbcs, out cp437, pref cp1252 [debug] youtube-dl version 2024.08.07 [c5098961b] (single file build) [debug] This version was built from the latest master code at https://github.com/ytdl-org/youtube-dl. [debug] For support, visit the main site. [debug] Python 3.4.4 (CPython x86 32bit) - Windows-XP-5.1.2600-SP3 - OpenSSL 1.0.2d 9 Jul 2015 [debug] exe versions: ffmpeg N-105053-gc417616-Reino [debug] Proxy map: {} [youtube] qMlE4XHs4c: Downloading webpage [debug] [youtube] Decrypted nsig p7c-ppSqRGulPB3uZ => q4J74EK8zIJ5tw [debug] [youtube] Decrypted nsig 4EUTmcUZzj5wawLAs => emC4TGBZfoSBzw [debug] Default format spec: bestvideo+bestaudio/best [debug] Invoking downloader on 'https://rr4---sn-j5caxoxu-i2oe.googlevideo.com/v ideoplayback?itag=137&source=youtube&fvip=5&id=o-AD8XJh7GWC2O36vbyC3DE8mjva8ueR --YXoH1OjGOLt&mt=1723339023&txp=4432434&mv=m&pl=21&c=WEB&vprv=1&ei=GRO4Zr6KFJ-Lk ucP052Y4QM&lsparams=mh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpcm2cms%2Cpl%2Cinitcwndbps&cl en=301152641&ns=_1ekMbUw5eoRefcUHhmYgEAQ&initcwndbps=1501250&lsig=AGtxev0wRQIhAJ LR0NWH1t1KArDu49H3aBLQR73goRoH75ijLOstpSXsAiAzWFYSFOpXWKlsqLVg9kxlXB0KdvZrqqFHa2 YDW5JFlw%3D%3D&sefc=1&dur=2315.566&gir=yes&keepalive=yes&sparams=expire%2Cei%2Ci p%2Cid%2Caitags%2Csource%2Crequiressl%2Cxpc%2Cbui%2Cspc%2Cvprv%2Csvpuc%2Cmime%2C ns%2Crqh%2Cgir%2Cclen%2Cdur%2Clmt&xpc=EgVo2aDSNQ%3D%3D&svpuc=1&rqh=1&pcm2cms=yes &mh=ua&mime=video%2Fmp4&expire=1723361145&ip=69.114.104.125&n=emC4TGBZfoSBzw&lmt =1722810022449989&mn=sn-j5caxoxu-i2oe%2Csn-ab5sznzy&requiressl=yes&sig=AJfQdSswR AIgGE1uwRF5IMJecXdLgEMUU12JPSz4jdc0YLWXGwrgim8CIDCw0Wj0voplo-mz8wYD3a9dzIdsBY9vj JjrpYo_rfc-&mvi=4&mm=31%2C29&spc=Mv1m9iY3F8wSwlgtwfG2IwHJCfLqa7bgFVU5EHJ_T86JGMG sbKW_18saGNu4&bui=AXc671KyW2kUFAfbgfPWl-xPo_nkX7mLlnoSPbhGVAcHasiNhY6iYU1VO7TGHU aCe-B66r-mqxtpkx7E&ms=au%2Crdu&aitags=133%2C134%2C135%2C136%2C137%2C160%2C242%2C 243%2C244%2C247%2C248%2C278' [dashsegments] Total fragments: 29 [download] Destination: First Time Watching NEAR DARK Reaction... I LOVED IT-_qM lE4XHs4c.f137.mp4 [download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 1 (attempt 1 of 10)... [download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 1 (attempt 2 of 10)... [download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 1 (attempt 3 of 10)... [download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 1 (attempt 4 of 10)... [download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 1 (attempt 5 of 10)... [download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 1 (attempt 6 of 10)... [download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 1 (attempt 7 of 10)... [download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 1 (attempt 8 of 10)... [download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 1 (attempt 9 of 10)... [download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 1 (attempt 10 of 10)... ERROR: giving up after 10 fragment retries File "main.py", line 19, in File "D:\a\ytdl-nightly\ytdl-nightly\youtube_dl__init.py", line 477, in main File "D:\a\ytdl-nightly\ytdl-nightly\youtube_dl__init__.py", line 467, in _real_main File "D:\a\ytdl-nightly\ytdl-nightly\youtube_dl\YoutubeDL.py", line 2258, in download File "D:\a\ytdl-nightly\ytdl-nightly\youtube_dl\YoutubeDL.py", line 872, in extract_info File "D:\a\ytdl-nightly\ytdl-nightly\youtube_dl\YoutubeDL.py", line 879, in wrapper File "D:\a\ytdl-nightly\ytdl-nightly\youtube_dl\YoutubeDL.py", line 986, in extract_info File "D:\a\ytdl-nightly\ytdl-nightly\youtube_dl\YoutubeDL.py", line 1020, in process_ie_result File "D:\a\ytdl-nightly\ytdl-nightly\youtube_dl\YoutubeDL.py", line 1864, in process_video_result File "D:\a\ytdl-nightly\ytdl-nightly\youtube_dl\YoutubeDL.py", line 142, in wrapper File "D:\a\ytdl-nightly\ytdl-nightly\youtube_dl\YoutubeDL.py", line 2156, in process_info File "D:\a\ytdl-nightly\ytdl-nightly\youtube_dl\YoutubeDL.py", line 2092, in dl File "D:\a\ytdl-nightly\ytdl-nightly\youtube_dl\downloader\common.py", line 380, in download File "D:\a\ytdl-nightly\ytdl-nightly\youtube_dl\downloader\dash.py", line 78, in real_download File "D:\a\ytdl-nightly\ytdl-nightly\youtube_dl\downloader\common.py", line 175, in report_error File "D:\a\ytdl-nightly\ytdl-nightly\youtube_dl\YoutubeDL.py", line 682, in report_error File "D:\a\ytdl-nightly\ytdl-nightly\youtube_dl\YoutubeDL.py", line 634, in trouble

dirkf commented 1 month ago

Indeed, the problem is presumably #32905 and independent of connection type.