search

drawrowfly / tiktok-scraper

TikTok Scraper. Download video posts, collect user/trend/hashtag/music feed metadata, sign URL and etc.
4.42k stars 801 forks source link

Can't extract video metadata. #681

Open netherlink117 opened 3 years ago

netherlink117 commented 3 years ago

Describe the bug Some videos throws this kind of message to the console (CMD):

Can't extract video metadata. https://www.tiktok.com/@some_user/video/some_video_id

I think the message is thrown randomly for some reason (I ran the script several times, while the error was thrown for some videos the first time, it was thrown to another videos the next times).

NOTE: I also notice that using -s and --historypath flags is being ignored.

If fields below will be empty then issue will be Ignored and Closed

To Reproduce Steps to reproduce the behavior:

Screenshots If applicable, add screenshots to help explain your problem.

netherlink117 commented 3 years ago

Got a (somewhat) temporal fix for my issue using CLI in a batch file for CMD on Windows 10.

The script's name is ts-list.cmd and contains:

@echo off
setlocal enabledelayedexpansion
for /f "usebackq delims=" %%a in ("%~1") do (
  call :parse "%%a" 
)
goto :eof
:parse
set "url=%~1"
if not exist %cd%\%url:~-19,19%.mp4 (
  tiktok-scraper video -d -w --hd --session "sid_tt=<session>" "%url%"
)
goto :eof
endlocal

Just changed <session> to a correct value and placed the script in the user's folder (or somewhere accesible as the system variable PATH), and called the script with:

ts-list.cmd <file>

Where <file> is the path to a .txt file containing links to videos.

The tiktok-scraper stills throwing the mesage errors described in the issue, but since the script checks if the video was already downloaded and try to download only the ones that got the error on the past runs, it can download the full list of videos from the .txt file if it's ran more than one time.

aymenitc commented 2 years ago

I have the same issue. It was working fine before. I want NodeJS fix not CLI. any help plz?

BrunoQuaresma commented 2 years ago

I'm also having the same issue. Probably, some TikTok scrape protection.

vanities commented 2 years ago

adding this comment here, since @drawrowfly marked my issue as spam. This issue is 4 months old and not relevant to the new issues and isn't being discussed technically.

https://github.com/drawrowfly/tiktok-scraper/issues/724#issuecomment-1017907777

Slikon commented 2 years ago

I have some kind of similar issue. While trying to scrap metadata from a video https://vm.tiktok.com/ZMeb45Cv2/ it always throws error: Can't extract video metadata: https://vm.tiktok.com/ZMeb45Cv2/. But when I try to scrap the full link (that can be copied from browser, not the mobile app) it works fine: image

Could someone please advice on how to work with short TikTok URLs? Thanks!

P.S. Adding those headers mentioned in my code does not help to solve this issue.

@drawrowfly Could you please support with this problem?

drawrowfly commented 2 years ago

I will push soon updated version that will fix most of the existing problems

aymenitc commented 2 years ago

Thank you. Waiting for update

في السبت، ٢٢ يناير ٢٠٢٢ ٦:٥٣ م Andrew Nord @.***> كتب:

I will push soon updated version that will fix most of the existing problems

— Reply to this email directly, view it on GitHub https://github.com/drawrowfly/tiktok-scraper/issues/681#issuecomment-1019296626, or unsubscribe https://github.com/notifications/unsubscribe-auth/ASKHVRSNIYGEY62Y434OYELUXLHIBANCNFSM5EOKVNIA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you commented.Message ID: @.***>

aleksnavratil commented 2 years ago

Thanks for the fast response @drawrowfly :) I appreciate it and will stay tuned!

DaStormer commented 2 years ago

@drawrowfly hey, any update on this?

nguyen95 commented 2 years ago

Call video url in postman and return empty data.

leele2 commented 2 years ago

Tried both in CLI and docker and can't seem to get any videos to download due to failed metadata.

Is this scraper broken atm?

vanities commented 2 years ago

Having 100% failures as of today:

response.split("window['SIGI_STATE']=")[1]

is returning undefined

inside of getVideoMetadataFromHtml

and sometimes

const response = await this.request<string>(options);

is raising an error

Seems like SIGI are inside of scripts:

<script id="SIGI_STATE" type="application/json"> ....<script id="SIGI_RETRY" type="application/json">
vanities commented 2 years ago

PR for a fix is incoming

dannyb648 commented 2 years ago

I have the same issue, all of my video -d calls are failing, with both the mobile and full links.

brvnonascimento commented 2 years ago

@vanities Would you have the PR number/link in hand?

srinitude commented 2 years ago

Is there a fix for this yet?

nicolas-wl commented 2 years ago

When is this getting fixed?

srinitude commented 2 years ago

@drawrowfly Any updates on this?

shifoc commented 2 years ago

I have the same issue, is the npm repo getting updated?

JoahTheron commented 2 years ago

When is this getting fixed?

this is open source how about doing it yourself?

yongchin0821 commented 2 years ago

The problem has not been fixed yet

ThisMe124 commented 2 years ago

Can't Extract Video Metadata response happened to me too💀

JakeMaldonado commented 2 years ago

@ThisMe124 Check this out, fixed it for me

https://github.com/drawrowfly/tiktok-scraper/issues/794#issuecomment-1266028293

JakeMaldonado commented 1 year ago

Never mind, I can no longer access the https://www.tiktok.com/node/share/video/@franzundwach/7117621530086968582 endpoint. Is anyone able to? I found before that I only could If I logged in certain ways, but now I cant seem to hit it at all.