search

dfreelon / pyktok

A simple module to collect video, text, and metadata from Tiktok.
BSD 3-Clause "New" or "Revised" License
315 stars 44 forks source link

Multi-url download not working #57

Closed deboutlesdamnes closed 1 month ago

deboutlesdamnes commented 1 month ago

Hi, I'm trying to download a couple hundred videos using a list of URLs, and the script worked great for the first fifty or so but is now throwing a new error (file location information removed for privacy):

Traceback (most recent call last): pyk.save_tiktok_multi_urls(urls['URL'].tolist(), False, Path(file).stem + '_metadata.csv') save_tiktok(u,save_video,metadata_fn,browser_name) data_slot = tt_json["__DEFAULT_SCOPE__"]['webapp.video-detail']['itemInfo']['itemStruct'] KeyError: 'webapp.video-detail'

This is my program:

from pathlib import Path import pyktok as pyk import pandas as pd import numpy as np import browser_cookie3 import requests import os pyk.specify_browser('chrome') browser_cookie3.chrome()

for file in os.listdir('../'): if file.endswith('.xlsx'): urls = pd.read_excel(os.path.join("../" + file)) pyk.save_tiktok_multi_urls(urls['URL'].tolist(), False, Path(file).stem + '_metadata.csv')

dfreelon commented 1 month ago

TikTok has been known to block rapid requests. I suggest throttling your use of Pyktok, experimenting with increasing time intervals. You can also try inspecting the results of alt_get_tiktok_json to see if the result it returns contains an error or some indication of what's going on.

dfreelon commented 1 month ago

Closed due to lack of activity--feel free to reopen if you can provide more details.