[Question] Sleep interval for subtitles on youtube.

[Ditiae]

Checklist

• [x] I'm asking a question
• [x] I've looked through the README and FAQ for similar questions
• [x] I've searched the bugtracker for similar questions including closed ones

Question

Is it by design that Youtubedlc applys the sleep interval to subtitles downloads? Youtubedl does not do this, it makes the downloads take so long to complete and I'm not sure if this is the intended behavior.

[blackjack4494]

Hm. Can you provide log? It shouldn't by default. You could try to set a max sleep interval (1 second, not sure what happens with 0). Last time I downloaded a video with like 20+ subtitles it was done almost instantly.

[Ditiae]

Took me forever to find a video with enough subtitles to properly show what I mean. Heres some logs: https://pastebin.com/raw/5eH42PV5

[blackjack4494]

Yes I tried it as well and see what you mean. Technically the docs says this will be applied for all downloads so subtitle download would included. Why this won't happen with youtube-dl I don't know. Would need to compare code. However I could think of passing an option that allows subtitles to be excluded by the sleep interval rule.

[Ditiae]

That would be cool. I have to download hundreds of thousands of videos and waiting for subtitles is really inconvenient and slows down the process for each video by up to 10x in some cases.

[blackjack4494]

@Wea1thRS I may have found the cause. Will do some testing and let you know. The code that changes the described behaviour is here. The idea was to use the new function dl which triggers fd.download and this checks for the sleep interval. While the old code didn't invoke dl to download subtitles but rather download it 'manually'.

There are two possible solutions to that.
Either revert changes and exclude subtitles completely from sleep interval
or
add another flag that tells the downloader to include subtitles in the sleep interval.
For the second solution I would set the behaviour to the one youtube-dl has right now so by default subtitles are excluded from sleep interval. However if you explicitly want subtitle downloads be affected by sleep interval you could add that flag.

Let me know what you think. Maybe some other people are reading this as well.

[Ditiae]

I'm not sure which solution would be better. If I'm understanding this correctly, using a singular dl method would be more proper, however, the waiting the sleep interval for each subtitle is a huge change for me. I've been downloading from a list of videos now for over a month now and still only about ~5% of the way through it and sleeping in-between each subtitle is a huge slow-down. I'm not sure how much spamming subtitle downloads affects the chance of inheriting some sort of rate limiting, though from my time using regular yt-dl I've never encountered any using my sleep settings. So I assume the limits relating to subtitles are fairly relaxed.

Either solution works for me, I know I will be using the flag to disclude subtitles from the rate-limit, however, someone else may have a use for it.

That got a little ramble-y, apologies.

[blackjack4494]

The idea would be to default not to include subtitles to be rate limited only if you explicitly assing the flag. So it behaves like youtube-dl but you can rate limit subtitle download with a flag as well.

[blackjack4494]

Default

PS D:\gitkraken\youtube-dl> python3 -m youtube_dlc "https://www.youtube.com/watch?v=gKyTZRDquB8" -v --all-subs --min-sleep-interval 10 --max-sleep-interval 20 [debug] System config: [] [debug] User config: [] [debug] Custom config: [] [debug] Command-line args: ['https://www.youtube.com/watch?v=gKyTZRDquB8', '-v', '--all-subs', '--min-sleep-interval', '10', '--max-sleep-interval', '20'] [debug] Loading archive file None [debug] Encodings: locale cp65001, fs utf-8, out utf-8, pref cp65001 [debug] youtube-dlc version 2020.09.23-1 [debug] Git HEAD: 88bdacf33 [debug] Python version 3.6.5 (CPython) - Windows-10-10.0.19041-SP0 [debug] exe versions: ffmpeg 4.2.1, ffprobe 4.2.1 [debug] Proxy map: {} [debug] Default format spec: bestvideo+bestaudio/best [info] Writing video subtitles to: ASMR Massive ear drum and brain throbbing fast ear massage (subtitles, latex gloves)-gKyTZRDquB8.zh-Hans.vtt [info] Writing video subtitles to: ASMR Massive ear drum and brain throbbing fast ear massage (subtitles, latex gloves)-gKyTZRDquB8.zh-Hant.vtt ...

With flag

PS D:\gitkraken\youtube-dl> python3 -m youtube_dlc "https://www.youtube.com/watch?v=gKyTZRDquB8" -v --all-subs --min-sleep-interval 10 --max-sleep-interval 20 --sleep-subtitles [debug] System config: [] [debug] User config: [] [debug] Custom config: [] [debug] Command-line args: ['https://www.youtube.com/watch?v=gKyTZRDquB8', '-v', '--all-subs', '--min-sleep-interval', '10', '--max-sleep-interval', '20', '--sleep-subtitles'] [debug] Loading archive file None [debug] Encodings: locale cp65001, fs utf-8, out utf-8, pref cp65001 [debug] youtube-dlc version 2020.09.23-1 [debug] Git HEAD: 88bdacf33 [debug] Python version 3.6.5 (CPython) - Windows-10-10.0.19041-SP0 [debug] exe versions: ffmpeg 4.2.1, ffprobe 4.2.1 [debug] Proxy map: {} [youtube] gKyTZRDquB8: Downloading webpage [debug] Default format spec: bestvideo+bestaudio/best [info] Writing video subtitles to: ASMR Massive ear drum and brain throbbing fast ear massage (subtitles, latex gloves)-gKyTZRDquB8.zh-Hans.vtt subtitles have sleep interval enabled [debug] Invoking downloader on 'https://www.youtube.com/api/timedtext?lang=zh-Hans&v=gKyTZRDquB8&fmt=vtt&name=' [download] Sleeping 14.50 seconds...

[blackjack4494]

Implemented in latest release (2020.09.30)