dfreelon / pyktok

A simple module to collect video, text, and metadata from Tiktok.
BSD 3-Clause "New" or "Revised" License
316 stars 44 forks source link

ItemModule error while downloading #53

Closed LauraPalmer110 closed 2 months ago

LauraPalmer110 commented 2 months ago

Hey there! I tried to download videos fro an array of link, and this is the error I got:

KeyError Traceback (most recent call last) Cell In[67], line 1 ----> 1 pyk.save_tiktok_multi_urls(tiktok_videos, 2 False, 3 'tiktok_data.csv', 4 1)

File ~\anaconda3\lib\site-packages\pyktok\pyktok.py:334, in save_tiktok_multi_urls(video_urls, save_video, metadata_fn, sleep, browser_name) 332 tt_urls = video_urls 333 for u in tt_urls: --> 334 save_tiktok(u,save_video,metadata_fn,browser_name) 335 time.sleep(random.randint(1, sleep)) 336 print('Saved',len(tt_urls),'videos and/or lines of metadata')

File ~\anaconda3\lib\site-packages\pyktok\pyktok.py:225, in save_tiktok(video_url, save_video, metadata_fn, browser_name, return_fns) 222 tt_json = get_tiktok_json(video_url,browser_name) 224 if tt_json is not None: --> 225 video_id = list(tt_json['ItemModule'].keys())[0] 227 if save_video == True: 228 regex_url = re.findall(url_regex, video_url)[0]

KeyError: 'ItemModule'

For context, I had a column of tiktok links on a Pandas Dataframe, and i turned it into a list. Here's the procedure:

tiktok_videos = df["Link"].tolist() pyk.save_tiktok_multi_urls(tiktok_videos, False, 'tiktok_data.csv', 1)

LikeADemonicMind commented 2 months ago

Hi! I've got the same error... Have you found any solution?

dfreelon commented 2 months ago

It would be helpful to see some examples of the URLs you're feeding it... that goes for both of you, thx.

dfreelon commented 2 months ago

I just tested this and it works for me, so please also verify the version you're using.

LikeADemonicMind commented 2 months ago

For example: https://www.tiktok.com/@sweetbearoline

I've made some research and found that tt_script = soup.find('script', attrs={'id':"SIGI_STATE"}) (line 183 in pyktok.py) is always None

dfreelon commented 2 months ago

Line 183 should not be running. Please verify which version you're using.

dfreelon commented 2 months ago

Also, the function you mentioned requires multiple video URLs and unfortunately cannot download multiple videos from a given user page as you listed. This is a known limitation.

LikeADemonicMind commented 2 months ago

Well, I'm not using the same function as @LauraPalmer110 but this one : pyk.save_tiktok_multi_page('https://www.tiktok.com/@sweetbearoline',save_video=False,save_metadata=True, browser_name='Firefox') and I get the same error, this is why I answered here, sorry for not being clear.

I'm using pyktok 0.0.22

dfreelon commented 2 months ago

Then your issue is a duplicate of #41 for which there is currently no known solution, unfortunately.

LikeADemonicMind commented 2 months ago

Oh sorry ^^' thanks a lot for you answer tho! And thanks for the great job!

dfreelon commented 2 months ago

@LikeADemonicMind, you may be interested in my recent update re #41. @LauraPalmer110 I cannot reproduce your issue without seeing your input data, so you can feel free to post that at your leisure, but I'm closing this for now.