TikTok only downloading 24 latest posts

bandicootdancing commented 5 years ago

Make sure you are using the latest version: run `youtube-dl --version` and ensure your version is 2019.01.30.1. If it's not, read this FAQ entry and update. Issues with outdated version will be rejected.

[X] I've verified and I assure that I'm running youtube-dl 2019.01.30.1

Before submitting an issue make sure you have:

[X] At least skimmed through the README, most notably the FAQ and BUGS sections
[X] Searched the bugtracker for similar issues including closed ones
[X] Checked that provided video/audio/playlist URLs (if any) are alive and playable in a browser

What is the purpose of your issue?

[X] Bug report (encountered problems with youtube-dl)
[ ] Site support request (request for adding support for a new site)
[ ] Feature request (request for a new functionality)
[ ] Question
[ ] Other

The following sections concertize particular purposed issues, you can erase any section (the contents between triple ---) not applicable to your issue

Description of your issue, suggested solution and other information

TikTok downloading appears to be limited to 24 videos. I've tried about 10 profiles and each one has been limited to 24 videos. I've confirmed in the app that there are more than 24 videos on the profiles I'm testing. Expected behavior is to grab the complete profile. I'm specifically using the short link format mentioned here #18108 in the site request.

ealgase commented 5 years ago

The TikTok profile view on the web only shows 24 videos. To fix this, the TikTok extractor would need to reverse engineer the way the TikTok app works (and require a log in).

bandicootdancing commented 5 years ago

Thanks for the clarification. Makes sense now that I see the pages for the share URLs. I wouldn't mind logging in to use this.

ItsABlackScreen commented 5 years ago

Tiktok now allows the viewing of all the videos of a user on the new url's. Hopefully support can be added to download all videos available by a user.

eg:- https://www.tiktok.com/share/user/6557888952919916545

 https://www.tiktok.com/share/user/9937375?refer=pc_reflow_user

Single video: http://v16-tiktokcdn-com.akamaized.net/283d8fedc4baa1d0d55658c3978112b4/5c853c15/video/n/v0102/c5c8b3569cf74475bab775e6bbaf337f/?rc=M25uZ28zdnlyazMzNDgzM0ApQHRud2xAbzdGNjo0OjM0MzM0NjM0MzQ1b0BoNnUpQGYzdSlmc2V6aGhkZjM0QG9rZmZiaGFyc18tLTUvNHNzNW8jbGhxbyMxMy0yLzAyLi4tLzEtMDYtOiNvIzphLXEjOmBoXitiK2p0OiM1Ll4%3D

P.s. I was a klutz and opened a new issue without realizing that this one was still open. Sorry!

jmraker commented 5 years ago

I noticed that there's a few file naming quirks with tiktok's newer user video profile pages. Not sure if it deserves a new issue as it's all filename related.

It correctly downloads a video but with the extension ".unknown_video". When it's renamed to a .mp4 file it plays.

When it downloads this url "https://www.tiktok.com/share/video/6675989091407891717?refer=pc_reflow_user" it adds "refer.pc_reflow_user" to the filename. The new url parameters are from clicking on a video at https://www.tiktok.com/share/user/6567659045795758085 in a web browser.

Another different and probably unnecessary file naming behavior from before is it adding " on TikTok" in the filename. It's adding that to every user video page I've tried today. For example 'Chris Alexander on TikTok-6673921230787071238refer=pc_reflow_user.unknown_video'

$ youtube-dl.exe -v "https://www.tiktok.com/share/video/6675989091407891717?refer=pc_reflow_user" [debug] System config: [] [debug] User config: [] [debug] Custom config: [] [debug] Command-line args: ['-v', 'https://www.tiktok.com/share/video/6675989091407891717?refer=pc_reflow_user'] [debug] Encodings: locale cp1252, fs mbcs, out cp1252, pref cp1252 [debug] youtube-dl version 2019.04.01 [debug] Python version 3.4.4 (CPython) - Windows-7-6.1.7601-SP1 [debug] exe versions: avconv v13_dev0-1440-g34c1133, avprobe v13_dev0-1440-g34c1133, ffmpeg 4.0.2, ffprobe 4.0.2 [debug] Proxy map: {} [generic] 6675989091407891717?refer=pc_reflow_user: Requesting header WARNING: Falling back on generic information extractor. [generic] 6675989091407891717?refer=pc_reflow_user: Downloading webpage [generic] 6675989091407891717?refer=pc_reflow_user: Extracting information [download] Downloading playlist: Tom on TikTok [generic] playlist Tom on TikTok: Collected 1 video ids (downloading 1 of them) [download] Downloading video 1 of 1 [debug] Default format spec: bestvideo+bestaudio/best [debug] Invoking downloader on 'http://v19.muscdn.com/ba54aff5597e5bbd69d1215f1d079884/5ca874d9/video/tos/maliva/tos-maliva-v-0068/4a248f13868e4fff8d5b905f42ff18cc/?rc=M2g8M3B1bXN4bDMzODczM0ApQHRwbndsQG9FNjQ7NTgzNDMzNjc1MzM0NW9AaDN1KUBmM3UpZnNlemhoZGY2NUBsYS9pbG5eLWZfLS0yMTZzczVvI1xsaHFvI0I0MC0vMTIuLjYwLjIxNi06I28jOmEtcSM6YGheK2IranQ6IzAuXg==' [download] Destination: Tom on TikTok-6675989091407891717refer=pc_reflow_user.unknown_video [download] 100% of 3.32MiB in 00:02 [download] Finished downloading playlist: Tom on TikTok

PSlava commented 5 years ago

When it downloads this url "https://www.tiktok.com/share/video/6675989091407891717?refer=pc_reflow_user" it adds "refer.pc_reflow_user" to the filename. The new url parameters are from clicking on a video at https://www.tiktok.com/share/user/6567659045795758085 in a web browser.

It happens because program downloads using the generic extractor. TikTok extractor doesn't support https://www.tiktok.com/share/video/6675989091407891717?refer=pc_reflow_user url. Regexp _VALID_URL should be updated to support this link. And for user too.

ytdl-org / youtube-dl