yt-dlp / yt-dlp

A feature-rich command-line audio/video downloader
https://discord.gg/H5MNcFW63r
The Unlicense
90.69k stars 7.04k forks source link

reset sleep counter upon success #7241

Open chrizilla opened 1 year ago

chrizilla commented 1 year ago

DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

Checklist

Provide a description that is worded well enough to be understood

Exponential --retry-sleep is very useful, but doesn't quite work as I would have expected.

Let's look at the following example:

With

  --retry-sleep exp=1
  --retry-sleep http,fragment,file_access,extractor:exp=1

I got this:

[debug] Invoking http downloader on "https://proxy-032.ix7.dailymotion.com/sec(qXn33KyUlfq1gBmyc-Vtn7WOnmfpWc2h4zlCmbIRyVfWmyXVr1Yubw8pXa6aMGBGzf_WOyTMewCSuCNbSmm4Bg)/video/389/351/491153983_mp4_h264_aac_fhd.mp4"
[debug] File locking is not supported. Proceeding without locking
[download] Destination: c:\hp\-\#YT\2021-09-24 737 Max: Boeings Todesmaschinen\737 Max: Boeings Todesmaschinen [dailymotion.x84f4u7] {http-1080-1}.mp4
[download]   3.4% of    1.23GiB at  181.39KiB/s ETA 01:54:24[download] Got error: Downloaded 45554816 bytes, expected 1320524734 bytes. Retrying (1/inf)...
Sleeping 1.00 seconds ...
[download] Destination: c:\hp\-\#YT\2021-09-24 737 Max: Boeings Todesmaschinen\737 Max: Boeings Todesmaschinen [dailymotion.x84f4u7] {http-1080-1}.mp4
[download]   3.6% of    1.23GiB at   33.71KiB/s ETA 10:14:36[download] Got error: Downloaded 47466880 bytes, expected 1320524734 bytes. Retrying (2/inf)...
Sleeping 2.00 seconds ...
[download] Destination: c:\hp\-\#YT\2021-09-24 737 Max: Boeings Todesmaschinen\737 Max: Boeings Todesmaschinen [dailymotion.x84f4u7] {http-1080-1}.mp4
[download]   3.7% of    1.23GiB at   48.28KiB/s ETA 07:08:41[download] Got error: Downloaded 48927488 bytes, expected 1320524734 bytes. Retrying (3/inf)...
Sleeping 4.00 seconds ...
[download] Destination: c:\hp\-\#YT\2021-09-24 737 Max: Boeings Todesmaschinen\737 Max: Boeings Todesmaschinen [dailymotion.x84f4u7] {http-1080-1}.mp4
[download]   4.0% of    1.23GiB at   83.73KiB/s ETA 04:06:23[download] Got error: Downloaded 53007104 bytes, expected 1320524734 bytes. Retrying (4/inf)...
Sleeping 8.00 seconds ...
[download] Destination: c:\hp\-\#YT\2021-09-24 737 Max: Boeings Todesmaschinen\737 Max: Boeings Todesmaschinen [dailymotion.x84f4u7] {http-1080-1}.mp4
[download]   4.2% of    1.23GiB at   56.75KiB/s ETA 06:02:52[download] Got error: Downloaded 55312384 bytes, expected 1320524734 bytes. Retrying (5/inf)...
Sleeping 16.00 seconds ...
[download] Destination: c:\hp\-\#YT\2021-09-24 737 Max: Boeings Todesmaschinen\737 Max: Boeings Todesmaschinen [dailymotion.x84f4u7] {http-1080-1}.mp4
[download]   4.2% of    1.23GiB at   20.16KiB/s ETA 17:01:06[download] Got error: Downloaded 55695452 bytes, expected 1320524734 bytes. Retrying (6/inf)...
Sleeping 32.00 seconds ...
[download] Destination: c:\hp\-\#YT\2021-09-24 737 Max: Boeings Todesmaschinen\737 Max: Boeings Todesmaschinen [dailymotion.x84f4u7] {http-1080-1}.mp4
[download]   4.4% of    1.23GiB at   49.86KiB/s ETA 06:51:55[download] Got error: Downloaded 58753024 bytes, expected 1320524734 bytes. Retrying (7/inf)...
Sleeping 64.00 seconds ...
[download] Destination: c:\hp\-\#YT\2021-09-24 737 Max: Boeings Todesmaschinen\737 Max: Boeings Todesmaschinen [dailymotion.x84f4u7] {http-1080-1}.mp4
[download]   4.7% of    1.23GiB at   42.24KiB/s ETA 08:04:52[download] Got error: Downloaded 62200960 bytes, expected 1320524734 bytes. Retrying (8/inf)...
Sleeping 128.00 seconds ...
[download] Destination: c:\hp\-\#YT\2021-09-24 737 Max: Boeings Todesmaschinen\737 Max: Boeings Todesmaschinen [dailymotion.x84f4u7] {http-1080-1}.mp4
[download]   4.8% of    1.23GiB at   61.47KiB/s ETA 05:32:46[download] Got error: Downloaded 63733760 bytes, expected 1320524734 bytes. Retrying (9/inf)...
Sleeping 256.00 seconds ...
[download] Destination: c:\hp\-\#YT\2021-09-24 737 Max: Boeings Todesmaschinen\737 Max: Boeings Todesmaschinen [dailymotion.x84f4u7] {http-1080-1}.mp4
[download]   5.2% of    1.23GiB at   71.38KiB/s ETA 04:45:29[download] Got error: Downloaded 68585856 bytes, expected 1320524734 bytes. Retrying (10/inf)...
Sleeping 512.00 seconds ...
[download] Destination: c:\hp\-\#YT\2021-09-24 737 Max: Boeings Todesmaschinen\737 Max: Boeings Todesmaschinen [dailymotion.x84f4u7] {http-1080-1}.mp4
[download]   5.5% of    1.23GiB at   70.18KiB/s ETA 04:49:30[download] Got error: Downloaded 72155136 bytes, expected 1320524734 bytes. Retrying (11/inf)...
Sleeping 1024.00 seconds ...
[download] Destination: c:\hp\-\#YT\2021-09-24 737 Max: Boeings Todesmaschinen\737 Max: Boeings Todesmaschinen [dailymotion.x84f4u7] {http-1080-1}.mp4
[download]   5.6% of    1.23GiB at   44.26KiB/s ETA 07:38:27[download] Got error: Downloaded 73812352 bytes, expected 1320524734 bytes. Retrying (12/inf)...
Sleeping 2048.00 seconds ...
[download] Destination: c:\hp\-\#YT\2021-09-24 737 Max: Boeings Todesmaschinen\737 Max: Boeings Todesmaschinen [dailymotion.x84f4u7] {http-1080-1}.mp4
[download]   5.7% of    1.23GiB at   65.34KiB/s ETA 05:10:10[download] Got error: Downloaded 75338496 bytes, expected 1320524734 bytes. Retrying (13/inf)...
Sleeping 4096.00 seconds ...

I don't think this makes much sense.

If an error occurs (such as above), --retry-sleep sends yt-dlp to sleep, ok. Now, when yt-dlp wakes up from sleep and

rationale

IMO the reason for exponential sleep time is to avoid continuously bumping into the same error with linear sleep time (i.e. trying to download every 2 seconds does not help when the server is currently down). Exponential sleep time is there sleep longer and longer and longer ... until the error goes away. But when the error has gone away, the counter should be reset, because it otherwise hampers the download unnecessarily (as shown in the example above).

Provide verbose output that clearly demonstrates the problem

Complete Verbose Output

full log ```shell [debug] Command-line config: ['https://www.dailymotion.com/video/x84f4u7'] [debug] Portable config "c:\hp\-\!portable\yt-dlp\yt-dlp.conf": ['--verbose', '--downloader-args', 'ffmpeg:-v 8 -stats -allowed_extensions ALL', '--no-playlist', '--format-sort', 'hasvid,ie_pref,quality,res,hdr,codec,size,br,asr,ext,fps,hasaud,lang,source,proto,id', '-F', '-f', 'bestvideo*+bestaudio/best', '-f-', '--paths', 'c:\\hp\\-\\#YT', '--download-archive', 'history.log', '--ffmpeg-location', '.\\helpertools\\FFmpeg\\bin', '--no-simulate', '--no-quiet', '--progress', '--write-info-json', '--write-description', '--write-annotations', '--write-playlist-metafiles', '--get-comments', '--write-link', '--write-url-link', '--write-desktop-link', '--no-overwrites', '--no-post-overwrites', '--merge-output-format', 'mkv', '--remux-video', 'mkv', '--list-subs', '--write-subs', '--write-auto-subs', '--all-subs', '--sub-format', 'all/ttml/vtt/best', '--sub-langs', 'all', '--extractor-args', 'youtube:skip=translated_subs', '--write-thumbnail', '--write-all-thumbnails', '--embed-metadata', '--embed-chapters', '--no-embed-info-json', '--no-check-certificate', '--retries', 'inf', '--file-access-retries', 'inf', '--fragment-retries', 'inf', '--no-skip-unavailable-fragments', '--retry-sleep', 'exp=1', '--retry-sleep', 'http,fragment,file_access,extractor:exp=1', '--geo-bypass', '--audio-multistreams', '--print', 'pre_process:\r\n========================================================= %(epoch>%Y-%m-%d [%H:%M:%S])s ========== stage 1: pre_process =================================================================\r\n', '--print', 'after_filter:\r\n========================================================= %(epoch>%Y-%m-%d [%H:%M:%S])s ========== stage 2: after_filter ================================================================\r\n', '--print', 'video:\r\n========================================================= %(epoch>%Y-%m-%d [%H:%M:%S])s ========== stage 3: video =======================================================================\r\n', '--print', 'before_dl:\r\n========================================================= %(epoch>%Y-%m-%d [%H:%M:%S])s ========== stage 4: before_dl ===================================================================\r\n', '--print', 'post_process:\r\n========================================================= %(epoch>%Y-%m-%d [%H:%M:%S])s ========== stage 5: post_process ================================================================\r\n', '--print', 'after_move:\r\n========================================================= %(epoch>%Y-%m-%d [%H:%M:%S])s ========== stage 6: after_move ==================================================================\r\n', '--print', 'after_video:\r\n========================================================= %(epoch>%Y-%m-%d [%H:%M:%S])s ========== stage 7: after_video =================================================================\r\n', '--print', 'playlist:\r\n========================================================= %(epoch>%Y-%m-%d [%H:%M:%S])s ========== stage 8: playlist ====================================================================\r\n', '--output-na-placeholder', '~~', '--parse-metadata', '%(is_live)s:(?PTrue)', '--parse-metadata', '%(was_live)s:(?PTrue)', '--replace-in-metadata', 'video:format_id', '_?-program_audio_0-[^+]+', '', '--alias', 'print-custom-file', '--print-to-file {} ".\\%(playlist_title,playlist|).100B%(playlist_title.101,playlist.101&…|)s\\%(playlist_index|)02d%(playlist_index& = |)s%(release_timestamp>%Y-%m-%d,release_date>%Y-%m-%d,modified_date>%Y-%m-%d,modified_timestamp>%Y-%m-%d,upload_date>%Y-%m-%d,upload_timestamp>%Y-%m-%d,timestamp>%Y-%m-%d,release_year|#nodate#)s %(alt_title,title).80B%(alt_title.81,title.81&…|)s\\\\"{}', '--alias', 'set-output', '-o {}".\\%(playlist_title,playlist|).100B%(playlist_title.101,playlist.101&…|)s\\%(playlist_index|)02d%(playlist_index& = |)s%(release_timestamp>%Y-%m-%d,release_date>%Y-%m-%d,modified_date>%Y-%m-%d,modified_timestamp>%Y-%m-%d,upload_date>%Y-%m-%d,upload_timestamp>%Y-%m-%d,timestamp>%Y-%m-%d,release_year|#nodate#)s %(alt_title,title).80B%(alt_title.81,title.81&…|)s\\\\"{}', '--alias', 'set-pl_output', '-o {}".\\%(playlist_title,playlist|).100B%(playlist_title.101,playlist.101&…|)s\\\\"{}', '--set-output', '', '%(title).120B%(title.121B&…|)s [%(extractor)s.%(islive&live.|)s%(id).80B%(id.81&…|)s] {%(format_id).60B%(format_id.61&…|)s}.%(ext)s', '--set-output', 'subtitle:', 'subs\\%(title).120B%(title.121B&…|)s [%(extractor)s.%(islive&live.|)s%(id).80B%(id.81&…|)s] {%(format_id).60B%(format_id.61&…|)s}.%(ext)s', '--set-output', 'infojson:', 'tech\\metadata.%(ext)s', '--set-output', 'link:', 'tech\\link.%(ext)s', '--set-output', 'description:', '!nfo\\synopsis.%(ext)s', '--set-output', 'annotation:', '!nfo\\annotations.%(ext)s', '--set-output', 'thumbnail:', 'thumbs\\thumb.%(ext)s', '--set-pl_output', 'pl_infojson:', 'playlist metadata.%(ext)s', '--set-pl_output', 'pl_description:', 'playlist description.%(ext)s', '--set-pl_output', 'pl_thumbnail:', 'folder.%(ext)s', '--set-pl_output', 'pl_video:', '%(playlist_title,playlist|).120B%(playlist_title.121,playlist.121&…|)s [%(extractor)s.%(islive&live.|)s%(id).50B%(id.51&…)s] {%(format_id)s}.%(ext)s', '--print-custom-file', 'after_video:%(comments|)j', '!nfo\\%(comment_count|[aborted])s comments.json', '--print-custom-file', '%(chapters|)#j', '!nfo\\chapters.json', '--print-custom-file', '%(requested_formats)#j', 'tech\\AV info.json', '--parse-metadata', 'video::(?P)', '--parse-metadata', 'video::(?P)', '--parse-metadata', 'video::(?P)', '--parse-metadata', 'before_dl:filename:(?P.*\\\\)', '--parse-metadata', 'before_dl:filename:(?P(?<=c:\\\\hp\\\\-\\\\#YT\\\\).*\\\\)', '--exec', 'pre_process:IF EXIST %(download_dir)q CD %(download_dir)q && IF EXIST !nfo ren !nfo !nfo.old & IF EXIST tech ren tech tech.old', '--exec', 'after_move:.\\helpertools\\FFmpeg\\bin\\ffprobe -hide_banner -show_streams -show_format %(filepath)q -of json=c=0 -o "%(download_dir)s\\tech\\FFprobe results.json"', '--exec', 'after_move:echo. & ..\\curl\\bin\\curl.exe --insecure "%(webpage_url,original_url)s" -o "%(download_dir)s\\!nfo\\home.html"', '--print-custom-file', '%(playlist&--------------- PLAYLIST -------------------------------|)s%(playlist&\r\nplaylist_title: |)s%(playlist_title|)s%(playlist&\r\nplaylist: |)s%(playlist|)s%(playlist&\r\nplaylist_id: |)s%(playlist_id|)s%(playlist&\r\nplaylist uploader: |)s%(playlist_uploader|)s%(playlist&\r\nplaylist uploader id: |)s%(playlist_uploader_id|)s%(playlist&\r\nplaylist index: |)s%(playlist_index&{} / |)s%(playlist_count|)s%(playlist&\r\nplaylist autonumber: |)s%(playlist_autonumber&{} / |)s%(n_entries|)s%(playlist&\r\n---------------------------------------------------------\r\n|)sfilename: %(filename|)s\r\n---------------- TITLE ----------------------------------\r\ntitle: %(title|)s\r\nfulltitle: %(fulltitle|)s\r\nalt title: %(alt_title|)s\r\ncategories: %(categories|)l\r\ntags: %(tags|)l\r\n---------------- WWW ------------------------------------\r\nid: %(id|)s\r\ndisplay id: %(display_id|)s\r\noriginal url: %(original_url|)s\r\nwebpage url: %(webpage_url|)s\r\nprotocol: %(protocol|)s\r\nextractor: %(extractor|)s\r\nextractor key: %(extractor_key|)s\r\n---------------- TIME ----------------------------------\r\nlocation: %(location|)s\r\nlive status: %(live_status|)s (islive:%(islive&yes|no)s, waslive:%(waslive&yes|no)s)\r\nreleased: %(release_timestamp>%Y-%m-%d %H:%M,release_date>%Y-%m-%d,release_year|)s\r\nlast modified: %(modified_timestamp>%Y-%m-%d %H:%M,modified_date>%Y-%m-%d|)s\r\nuploaded: %(upload_timestamp>%Y-%m-%d %H:%M,upload_date>%Y-%m-%d|)s\r\nbecame available: %(timestamp>%Y-%m-%d %H:%M|)s\r\ndownload start: %(epoch>%Y-%m-%d %H:%M:%S)s UTC\r\n runtime: %(duration>%H:%M:%S,duration_string|)19s\r\nstart time: %(start_time>%H:%M:%S|)19s\r\n end time: %(end_time>%H:%M:%S|)19s\r\n---------------- A/V -----------------------------------\r\nformat id: %(format_id|)s\r\nformat: %(format|)s\r\nformat note: %(format_note|)s\r\nhas DRM: %(has_drm,_has_drm|-)s / %(requested_formats.0.has_drm|)s / %(requested_formats.1.has_drm|)s\r\naudio bitrate: %(abr|)4d KBit/s\r\nvideo bitrate: %(vbr|)4d KBit/s\r\na/v bitrate: %(tbr|)4d KBit/s\r\nfilesize: %(filesize&{:,} bytes|)s%(filesize_approx&~{:,} bytes|)s\r\next: %(ext|)s%(container& / {}|)s\r\n--------------- VIDEO ----------------------------------\r\nvcodec: %(vcodec|)s\r\nresolution: %(width|)s*%(height|)s\r\naspect ratio: %(aspect_ratio|)s\r\nstretched ratio: %(stretched_ratio|)s\r\nframe rate: %(fps|)s fps\r\ndynamic range: %(dynamic_range|)s\r\n--------------- AUDIO ----------------------------------\r\nacodec: %(acodec|)s\r\nasr: %(asr|)s Hz\r\naudio channels: %(audio_channels|)s\r\nlanguage: %(language&{} |)s%(requested_formats.0.language&{} |)s%(requested_formats.1.language&{} |)s%(requested_formats.2.language&{} |)s%(requested_formats.3.language&{} |)s%(requested_formats.4.language&{} |)s%(requested_formats.5.language&{} |)s\r\n-------------- web 2.0 ---------------------------------\r\naverage rating: %(average_rating|)s\r\nage limit: %(age_limit|)s\r\nviewcount: %(view_count&{} views|)16s\r\nconcurrent views: %(concurrent_view_count&{} views|)16s\r\nlikes: %(like_count&{} likes|)16s%(dislike_count& / {} dislikes|)s\r\nreposts: %(repost_count&{} reposts|)18s\r\ncomments: %(comment_count&{} comments|)19s\r\n-------------- source ----------------------------------\r\ncreator: %(creator|)s\r\nuploader: %(uploader|)s\r\nuploader id: %(uploader_id|)s\r\nchannel: %(channel|)s\r\nchannel id: %(channel_id|)s\r\nchannel followers: %(channel_follower_count&{} followers|)s\r\navailability: %(availability|)s\r\nlicense: %(license|)s\r\nURLs:\r\n%(urls,url|)s', 'tech\\factsheet.nfo'] [debug] Encodings: locale cp1252, fs utf-8, pref cp1252, out utf-8, error utf-8, screen utf-8 [debug] yt-dlp version nightly@2023.05.26.130334 [66468bbf4] (win_exe) [debug] Python 3.8.10 (CPython AMD64 64bit) - Windows-10-10.0.19045-SP0 (OpenSSL 1.1.1k 25 Mar 2021) [debug] exe versions: ffmpeg N-110185-gb564ad8eac-20230407 (setts), ffprobe N-110185-gb564ad8eac-20230407 [debug] Optional libraries: Cryptodome-3.18.0, brotli-1.0.9, certifi-2023.05.07, mutagen-1.46.0, sqlite3-2.6.0, websockets-11.0.3 [debug] Proxy map: {} [debug] Loaded 1822 extractors [debug] Loading archive file 'history.log' [dailymotion] Extracting URL: https://www.dailymotion.com/video/x84f4u7 [dailymotion] Downloading Access Token [dailymotion] x84f4u7: Downloading media JSON metadata [dailymotion] x84f4u7: Downloading metadata JSON [dailymotion] x84f4u7: Downloading m3u8 information [debug] Sort order given by user: hasvid, ie_pref, quality, res, hdr, codec, size, br, asr, ext, fps, hasaud, lang, source, proto, id [debug] Formats sorted by: hasvid, ie_pref, quality, res, hdr, vcodec, acodec, filesize, fs_approx, tbr, vbr, abr, asr, vext, aext, fps, hasaud, lang, source, proto, id, channels ========================================================= 2023-05-29 [19:14:51] ========== stage 1: pre_process ================================================================= [debug] Searching for '(?PTrue)' in '%(is_live)s' [MetadataParser] Could not interpret '%(is_live)s' as '(?PTrue)' [debug] Searching for '(?PTrue)' in '%(was_live)s' [MetadataParser] Could not interpret '%(was_live)s' as '(?PTrue)' [Exec] Executing command: IF EXIST "~~" CD "~~" && IF EXIST !nfo ren !nfo !nfo.old & IF EXIST tech ren tech tech.old ========================================================= 2023-05-29 [19:14:51] ========== stage 2: after_filter ================================================================ x84f4u7 has no subtitles [info] Available formats for x84f4u7: ID EXT RESOLUTION │ FILESIZE TBR PROTO │ VCODEC VBR ACODEC ───────────────────────────────────────────────────────────────────────────────── hls-380-0 mp4 512x288 │ ~ 96.76MiB 461k m3u8 │ avc1.4d0016 461k mp4a.40.5 hls-380-1 mp4 512x288 │ ~ 96.76MiB 461k m3u8 │ avc1.4d0016 461k mp4a.40.5 http-380-0 mp4 512x288 │ ~ 96.76MiB 461k http │ avc1.4d0016 461k mp4a.40.5 http-380-1 mp4 512x288 │ ~ 96.76MiB 461k http │ avc1.4d0016 461k mp4a.40.5 hls-480-0 mp4 848x480 │ ~175.69MiB 836k m3u8 │ avc1.64001f 836k mp4a.40.2 hls-480-1 mp4 848x480 │ ~175.69MiB 836k m3u8 │ avc1.64001f 836k mp4a.40.2 http-480-0 mp4 848x480 │ ~175.69MiB 836k http │ avc1.64001f 836k mp4a.40.2 http-480-1 mp4 848x480 │ ~175.69MiB 836k http │ avc1.64001f 836k mp4a.40.2 hls-720-0 mp4 1280x720 │ ~451.53MiB 2149k m3u8 │ avc1.64001f 2149k mp4a.40.2 hls-720-1 mp4 1280x720 │ ~451.53MiB 2149k m3u8 │ avc1.64001f 2149k mp4a.40.2 http-720-0 mp4 1280x720 │ ~451.53MiB 2149k http │ avc1.64001f 2149k mp4a.40.2 http-720-1 mp4 1280x720 │ ~451.53MiB 2149k http │ avc1.64001f 2149k mp4a.40.2 hls-1080-0 mp4 1920x1080 │ ~ 1.28GiB 6222k m3u8 │ avc1.640028 6222k mp4a.40.2 hls-1080-1 mp4 1920x1080 │ ~ 1.28GiB 6222k m3u8 │ avc1.640028 6222k mp4a.40.2 http-1080-0 mp4 1920x1080 │ ~ 1.28GiB 6222k http │ avc1.640028 6222k mp4a.40.2 http-1080-1 mp4 1920x1080 │ ~ 1.28GiB 6222k http │ avc1.640028 6222k mp4a.40.2 Enter format selector: b [info] x84f4u7: Downloading 1 format(s): http-1080-1 [debug] Replacing all '_?-program_audio_0-[^+]+' in format_id with '' [MetadataParser] Did not find '_?-program_audio_0-[^+]+' in format_id [debug] Searching for '(?P)' in '' [MetadataParser] Parsed comments from '': '' [debug] Searching for '(?P)' in '' [MetadataParser] Parsed formats from '': '' [debug] Searching for '(?P)' in '' [MetadataParser] Parsed thumbnails from '': '' ========================================================= 2023-05-29 [19:14:51] ========== stage 3: video ======================================================================= [info] Writing '%(chapters|)#j' to: c:\hp\-\#YT\2021-09-24 737 Max: Boeings Todesmaschinen\!nfo\chapters.json [info] Writing '%(requested_formats)#j' to: c:\hp\-\#YT\2021-09-24 737 Max: Boeings Todesmaschinen\tech\AV info.json [info] Writing '%(playlist&--------------- PLAYLIST -------------------------------|)s%(playlist&\r\nplaylist_title: |)s%(playlist_title|)s%(playlist&\r\nplaylist: |)s%(playlist|)s%(playlist&\r\nplaylist_id: |)s%(playlist_id|)s%(playlist&\r\nplaylist uploader: |)s%(playlist_uploader|)s%(playlist&\r\nplaylist uploader id: |)s%(playlist_uploader_id|)s%(playlist&\r\nplaylist index: |)s%(playlist_index&{} / |)s%(playlist_count|)s%(playlist&\r\nplaylist autonumber: |)s%(playlist_autonumber&{} / |)s%(n_entries|)s%(playlist&\r\n---------------------------------------------------------\r\n|)sfilename: %(filename|)s\r\n---------------- TITLE ----------------------------------\r\ntitle: %(title|)s\r\nfulltitle: %(fulltitle|)s\r\nalt title: %(alt_title|)s\r\ncategories: %(categories|)l\r\ntags: %(tags|)l\r\n---------------- WWW ------------------------------------\r\nid: %(id|)s\r\ndisplay id: %(display_id|)s\r\noriginal url: %(original_url|)s\r\nwebpage url: %(webpage_url|)s\r\nprotocol: %(protocol|)s\r\nextractor: %(extractor|)s\r\nextractor key: %(extractor_key|)s\r\n---------------- TIME ----------------------------------\r\nlocation: %(location|)s\r\nlive status: %(live_status|)s (islive:%(islive&yes|no)s, waslive:%(waslive&yes|no)s)\r\nreleased: %(release_timestamp>%Y-%m-%d %H:%M,release_date>%Y-%m-%d,release_year|)s\r\nlast modified: %(modified_timestamp>%Y-%m-%d %H:%M,modified_date>%Y-%m-%d|)s\r\nuploaded: %(upload_timestamp>%Y-%m-%d %H:%M,upload_date>%Y-%m-%d|)s\r\nbecame available: %(timestamp>%Y-%m-%d %H:%M|)s\r\ndownload start: %(epoch>%Y-%m-%d %H:%M:%S)s UTC\r\n runtime: %(duration>%H:%M:%S,duration_string|)19s\r\nstart time: %(start_time>%H:%M:%S|)19s\r\n end time: %(end_time>%H:%M:%S|)19s\r\n---------------- A/V -----------------------------------\r\nformat id: %(format_id|)s\r\nformat: %(format|)s\r\nformat note: %(format_note|)s\r\nhas DRM: %(has_drm,_has_drm|-)s / %(requested_formats.0.has_drm|)s / %(requested_formats.1.has_drm|)s\r\naudio bitrate: %(abr|)4d KBit/s\r\nvideo bitrate: %(vbr|)4d KBit/s\r\na/v bitrate: %(tbr|)4d KBit/s\r\nfilesize: %(filesize&{:,} bytes|)s%(filesize_approx&~{:,} bytes|)s\r\next: %(ext|)s%(container& / {}|)s\r\n--------------- VIDEO ----------------------------------\r\nvcodec: %(vcodec|)s\r\nresolution: %(width|)s*%(height|)s\r\naspect ratio: %(aspect_ratio|)s\r\nstretched ratio: %(stretched_ratio|)s\r\nframe rate: %(fps|)s fps\r\ndynamic range: %(dynamic_range|)s\r\n--------------- AUDIO ----------------------------------\r\nacodec: %(acodec|)s\r\nasr: %(asr|)s Hz\r\naudio channels: %(audio_channels|)s\r\nlanguage: %(language&{} |)s%(requested_formats.0.language&{} |)s%(requested_formats.1.language&{} |)s%(requested_formats.2.language&{} |)s%(requested_formats.3.language&{} |)s%(requested_formats.4.language&{} |)s%(requested_formats.5.language&{} |)s\r\n-------------- web 2.0 ---------------------------------\r\naverage rating: %(average_rating|)s\r\nage limit: %(age_limit|)s\r\nviewcount: %(view_count&{} views|)16s\r\nconcurrent views: %(concurrent_view_count&{} views|)16s\r\nlikes: %(like_count&{} likes|)16s%(dislike_count& / {} dislikes|)s\r\nreposts: %(repost_count&{} reposts|)18s\r\ncomments: %(comment_count&{} comments|)19s\r\n-------------- source ----------------------------------\r\ncreator: %(creator|)s\r\nuploader: %(uploader|)s\r\nuploader id: %(uploader_id|)s\r\nchannel: %(channel|)s\r\nchannel id: %(channel_id|)s\r\nchannel followers: %(channel_follower_count&{} followers|)s\r\navailability: %(availability|)s\r\nlicense: %(license|)s\r\nURLs:\r\n%(urls,url|)s' to: c:\hp\-\#YT\2021-09-24 737 Max: Boeings Todesmaschinen\tech\factsheet.nfo [info] Writing video description to: c:\hp\-\#YT\2021-09-24 737 Max: Boeings Todesmaschinen\!nfo\synopsis.description [info] There are no subtitles for the requested languages [info] There are no video thumbnails to download [info] Writing video metadata as JSON to: c:\hp\-\#YT\2021-09-24 737 Max: Boeings Todesmaschinen\tech\metadata.info.json WARNING: There are no annotations to write. [info] Writing internet shortcut (.url) to: c:\hp\-\#YT\2021-09-24 737 Max: Boeings Todesmaschinen\tech\link.url [info] Writing internet shortcut (.desktop) to: c:\hp\-\#YT\2021-09-24 737 Max: Boeings Todesmaschinen\tech\link.desktop ========================================================= 2023-05-29 [19:14:51] ========== stage 4: before_dl =================================================================== [debug] Searching for '(?P.*\\\\)' in '%(filename)s' [MetadataParser] Parsed download_dir from '%(filename)s': 'c:\\hp\\-\\#YT\\2021-09-24 737 Max: Boeings Todesmaschinen\\' [debug] Searching for '(?P(?<=c:\\\\hp\\\\-\\\\#YT\\\\).*\\\\)' in '%(filename)s' [MetadataParser] Parsed download_dir_headless from '%(filename)s': '2021-09-24 737 Max: Boeings Todesmaschinen\\' [debug] Invoking http downloader on "https://proxy-032.ix7.dailymotion.com/sec(qXn33KyUlfq1gBmyc-Vtn7WOnmfpWc2h4zlCmbIRyVfWmyXVr1Yubw8pXa6aMGBGzf_WOyTMewCSuCNbSmm4Bg)/video/389/351/491153983_mp4_h264_aac_fhd.mp4" [debug] File locking is not supported. Proceeding without locking [download] Destination: c:\hp\-\#YT\2021-09-24 737 Max: Boeings Todesmaschinen\737 Max: Boeings Todesmaschinen [dailymotion.x84f4u7] {http-1080-1}.mp4 [download] 3.4% of 1.23GiB at 181.39KiB/s ETA 01:54:24[download] Got error: Downloaded 45554816 bytes, expected 1320524734 bytes. Retrying (1/inf)... Sleeping 1.00 seconds ... [download] Destination: c:\hp\-\#YT\2021-09-24 737 Max: Boeings Todesmaschinen\737 Max: Boeings Todesmaschinen [dailymotion.x84f4u7] {http-1080-1}.mp4 [download] 3.6% of 1.23GiB at 33.71KiB/s ETA 10:14:36[download] Got error: Downloaded 47466880 bytes, expected 1320524734 bytes. Retrying (2/inf)... Sleeping 2.00 seconds ... [download] Destination: c:\hp\-\#YT\2021-09-24 737 Max: Boeings Todesmaschinen\737 Max: Boeings Todesmaschinen [dailymotion.x84f4u7] {http-1080-1}.mp4 [download] 3.7% of 1.23GiB at 48.28KiB/s ETA 07:08:41[download] Got error: Downloaded 48927488 bytes, expected 1320524734 bytes. Retrying (3/inf)... Sleeping 4.00 seconds ... [download] Destination: c:\hp\-\#YT\2021-09-24 737 Max: Boeings Todesmaschinen\737 Max: Boeings Todesmaschinen [dailymotion.x84f4u7] {http-1080-1}.mp4 [download] 4.0% of 1.23GiB at 83.73KiB/s ETA 04:06:23[download] Got error: Downloaded 53007104 bytes, expected 1320524734 bytes. Retrying (4/inf)... Sleeping 8.00 seconds ... [download] Destination: c:\hp\-\#YT\2021-09-24 737 Max: Boeings Todesmaschinen\737 Max: Boeings Todesmaschinen [dailymotion.x84f4u7] {http-1080-1}.mp4 [download] 4.2% of 1.23GiB at 56.75KiB/s ETA 06:02:52[download] Got error: Downloaded 55312384 bytes, expected 1320524734 bytes. Retrying (5/inf)... Sleeping 16.00 seconds ... [download] Destination: c:\hp\-\#YT\2021-09-24 737 Max: Boeings Todesmaschinen\737 Max: Boeings Todesmaschinen [dailymotion.x84f4u7] {http-1080-1}.mp4 [download] 4.2% of 1.23GiB at 20.16KiB/s ETA 17:01:06[download] Got error: Downloaded 55695452 bytes, expected 1320524734 bytes. Retrying (6/inf)... Sleeping 32.00 seconds ... [download] Destination: c:\hp\-\#YT\2021-09-24 737 Max: Boeings Todesmaschinen\737 Max: Boeings Todesmaschinen [dailymotion.x84f4u7] {http-1080-1}.mp4 [download] 4.4% of 1.23GiB at 49.86KiB/s ETA 06:51:55[download] Got error: Downloaded 58753024 bytes, expected 1320524734 bytes. Retrying (7/inf)... Sleeping 64.00 seconds ... [download] Destination: c:\hp\-\#YT\2021-09-24 737 Max: Boeings Todesmaschinen\737 Max: Boeings Todesmaschinen [dailymotion.x84f4u7] {http-1080-1}.mp4 [download] 4.7% of 1.23GiB at 42.24KiB/s ETA 08:04:52[download] Got error: Downloaded 62200960 bytes, expected 1320524734 bytes. Retrying (8/inf)... Sleeping 128.00 seconds ... [download] Destination: c:\hp\-\#YT\2021-09-24 737 Max: Boeings Todesmaschinen\737 Max: Boeings Todesmaschinen [dailymotion.x84f4u7] {http-1080-1}.mp4 [download] 4.8% of 1.23GiB at 61.47KiB/s ETA 05:32:46[download] Got error: Downloaded 63733760 bytes, expected 1320524734 bytes. Retrying (9/inf)... Sleeping 256.00 seconds ... [download] Destination: c:\hp\-\#YT\2021-09-24 737 Max: Boeings Todesmaschinen\737 Max: Boeings Todesmaschinen [dailymotion.x84f4u7] {http-1080-1}.mp4 [download] 5.2% of 1.23GiB at 71.38KiB/s ETA 04:45:29[download] Got error: Downloaded 68585856 bytes, expected 1320524734 bytes. Retrying (10/inf)... Sleeping 512.00 seconds ... [download] Destination: c:\hp\-\#YT\2021-09-24 737 Max: Boeings Todesmaschinen\737 Max: Boeings Todesmaschinen [dailymotion.x84f4u7] {http-1080-1}.mp4 [download] 5.5% of 1.23GiB at 70.18KiB/s ETA 04:49:30[download] Got error: Downloaded 72155136 bytes, expected 1320524734 bytes. Retrying (11/inf)... Sleeping 1024.00 seconds ... [download] Destination: c:\hp\-\#YT\2021-09-24 737 Max: Boeings Todesmaschinen\737 Max: Boeings Todesmaschinen [dailymotion.x84f4u7] {http-1080-1}.mp4 [download] 5.6% of 1.23GiB at 44.26KiB/s ETA 07:38:27[download] Got error: Downloaded 73812352 bytes, expected 1320524734 bytes. Retrying (12/inf)... Sleeping 2048.00 seconds ... [download] Destination: c:\hp\-\#YT\2021-09-24 737 Max: Boeings Todesmaschinen\737 Max: Boeings Todesmaschinen [dailymotion.x84f4u7] {http-1080-1}.mp4 [download] 5.7% of 1.23GiB at 65.34KiB/s ETA 05:10:10[download] Got error: Downloaded 75338496 bytes, expected 1320524734 bytes. Retrying (13/inf)... Sleeping 4096.00 seconds ... ```
pukkandan commented 1 year ago

Retrying (13/inf)...

It is failing on the same block again and again, as seen from the increasing retry count.

Why do you think otherwise?

chrizilla commented 1 year ago

@pukkandan : I don't know how you can look at the log, particularly the increasing progress percentage and don't come to the conclusion that there is download progress ?

[download]   3.4%
[download]   3.6%
[download]   3.7%
[download]   4.0%
[download]   4.2%
[download]   4.2%
[download]   4.4%
[download]   4.7%
[download]   4.8%
[download]   5.2%
[download]   5.5%
[download]   5.6%
[download]   5.7%

My request is that

when yt-dlp wakes up from sleep and […] ... does not immediately fail again (i.e. successfully continues to download), the sleep time counter should be reset.

Grub4K commented 1 year ago

Basically you want any download progress to be treated as not a retry, even on reset?

chrizilla commented 1 year ago

@Grub4K : Could you please define "reset" in this context here ? (Pending understanding of what exactly a reset is, I would tentatively answer your question as: Download progress should reset the sleep counter to its initial value.)

chrizilla commented 1 year ago

pukkandan added invalid label

@pukkandan : You may not like the suggestion (and I can't fathom how one could possibly not like it), but it certainly is not "invalid", so I think there must be a misunderstanding somewhere.

My request for enhancement is this:

• error ► sleep ► wake up ► immediate fail ► increase exponential waiting time ► sleep • error ► sleep ► wake up ► download continues successfully (no immediate fail) ► reset sleep time counter ► later fails again ► sleep (with initial duration)