[Dailymotion] Youtube-dl only scrapes the last 1000 videos on a channel

KUR0N3K0H commented 3 years ago

Checklist

[x] I'm reporting a broken site support
[x] I've verified that I'm running youtube-dl version 2021.06.06
[x] I've checked that all provided URLs are alive and playable in a browser
[x] I've checked that all URLs and arguments with special characters are properly quoted or escaped
[x] I've searched the bugtracker for similar issues including closed ones

Verbose log

[debug] System config: [] [debug] User config: [] [debug] Custom config: [] [debug] Command-line args: [u'-v', u'https://www.dailymotion.com/tyler-d3'] [debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8 [debug] youtube-dl version 2021.06.06 [debug] Python version 2.7.18 (CPython) - Linux-5.4.0-77-generic-x86_64-with-LinuxMint-20-ulyana [debug] exe versions: ffmpeg 4.2.4, ffprobe 4.2.4, rtmpdump 2.4 [debug] Proxy map: {} [download] Downloading playlist: tyler-d3 [dailymotion:user] Downloading Access Token [dailymotion:user] tyler-d3: Downloading page 1 [dailymotion:user] tyler-d3: Downloading page 2 [dailymotion:user] tyler-d3: Downloading page 3 [dailymotion:user] tyler-d3: Downloading page 4 [dailymotion:user] tyler-d3: Downloading page 5 [dailymotion:user] tyler-d3: Downloading page 6 [dailymotion:user] tyler-d3: Downloading page 7 [dailymotion:user] tyler-d3: Downloading page 8 [dailymotion:user] tyler-d3: Downloading page 9 [dailymotion:user] tyler-d3: Downloading page 10 [dailymotion:user] tyler-d3: Downloading page 11 [dailymotion:user] playlist tyler-d3: Downloading 1000 videos [download] Downloading video 1 of 1000 .....

[download] Downloading video 1000 of 1000 [download] Finished downloading playlist: tyler-d3

Description

While trying to archive a user's Dailymotion channel I noticed that youtube-dl could only scrape their last 1000 videos. As shown in the log output once youtube-dl hits video number 1000 it thinks it has finished ripping every video from the channel. However the channel I'm trying to rip (https://www.dailymotion.com/tyler-d3/videos) has 1400 videos. I believe this is a similar issue to bug number #29311

handaza commented 3 years ago

I'd like to confirm this. This is exactly what's happening with me.

uecarlos commented 3 years ago

This is a limitation of the Dailymotion API. (https://developer.dailymotion.com/api/#response-lists)

ytdl-org / youtube-dl