取list不行

[S1U]

[renyijiu]

可以按照错误提示自己搜索下，如果代码有问题，欢迎提交你的解决方案

[S1U]

不知道对这个空 list 从何入手，请教下是哪方面可能有问题？

[renyijiu]

最简单的你直接重试运行，出错的话你可以看输出日志的最后一行提示搜索一下，应该请求网页数据出现了问题

[eromoe]

Similar error here .

retry...
get empty list, {'cursor': 10, 'status_code': 0, 'has_more': 1, 'aweme_list': []}
retry...
get empty list, {'cursor': 10, 'status_code': 0, 'has_more': 1, 'aweme_list': []}
retry...
Traceback (most recent call last):
  File "douyin.py", line 574, in <module>
    CrawlerScheduler(content, favorite)
  File "douyin.py", line 370, in __init__
    self.scheduling(favorite)
  File "douyin.py", line 379, in scheduling
    self.download_user_videos(user_id, favorite)
  File "douyin.py", line 397, in download_user_videos
    self.push_download_job(uid, dytk, 0, favorite)
  File "douyin.py", line 408, in push_download_job
    list_json = get_list_by_uid(user_id, dytk, cursor, favorite)
  File "douyin.py", line 138, in get_list_by_uid
    r.html.render()
  File "/home/kasim/miniconda3/envs/douyin/lib/python3.6/site-packages/requests_html.py", line 598, in render
    content, result, page = self.session.loop.run_until_complete(self._async_render(url=self.url, script=script, sleep=sleep, wait=wait, content=self.html, reload=reload, scrolldown=scrolldown, timeout=timeout, keep_page=keep_page))
  File "/home/kasim/miniconda3/envs/douyin/lib/python3.6/asyncio/base_events.py", line 488, in run_until_complete
    return future.result()
  File "/home/kasim/miniconda3/envs/douyin/lib/python3.6/site-packages/requests_html.py", line 505, in _async_render
    page = await self.browser.newPage()
  File "/home/kasim/miniconda3/envs/douyin/lib/python3.6/site-packages/pyppeteer/browser.py", line 192, in newPage
    return await self._defaultContext.newPage()
  File "/home/kasim/miniconda3/envs/douyin/lib/python3.6/site-packages/pyppeteer/browser.py", line 329, in newPage
    return await self._browser._createPageInContext(self._id)
  File "/home/kasim/miniconda3/envs/douyin/lib/python3.6/site-packages/pyppeteer/browser.py", line 200, in _createPageInContext
    'Target.createTarget', options)).get('targetId')
pyppeteer.errors.NetworkError: Protocol error Target.createTarget: Target closed.

Sometimes got

Traceback (most recent call last):
  File "douyin.py", line 574, in <module>
    CrawlerScheduler(content, favorite)
  File "douyin.py", line 370, in __init__
    self.scheduling(favorite)
  File "douyin.py", line 379, in scheduling
    self.download_user_videos(user_id, favorite)
  File "douyin.py", line 396, in download_user_videos
    uid, dytk = get_user_info(user_id)
  File "douyin.py", line 78, in get_user_info
    uid = r.html.search('uid: "{uid}"')['uid']
TypeError: 'NoneType' object is not subscriptable

Sometimes

exit Thread!
exit Thread!
exit Thread!
exit Thread!
exit Thread!
exit Thread!
exit Thread!
exit Thread!
exit Thread!
exit Thread!
successful downloaded!

or

get empty list, {'status_code': 0, 'has_more': True, 'aweme_list': []}
retry...
get empty list, {'status_code': 0, 'has_more': True, 'aweme_list': []}
retry...
list data save success!
exit Thread!
exit Thread!
exit Thread!
exit Thread!
exit Thread!
exit Thread!
exit Thread!
exit Thread!
exit Thread!
exit Thread!
successful downloaded!

No video downloaded .

[eromoe]

Found solution for pyppeteer.errors.NetworkError: Protocol error Target.createTarget: Target closed.

pip install websockets==6

@renyijiu assign in requirements.txt would be good .

So, the left problem is endless empty list ... I have no idea for now .

[renyijiu]

@eromoeIf you test and find that the code is working properly, welcome to submit your code. A new PR is great

[eromoe]

Webpage is empty in chrome mobile mode , even empty in mobile browser . So there may be a special UA or something called signature you mention in other threads . That is really hard , unless somebody inside tiktok can leak detail to us .

[renyijiu]

@eromoe Sorry, i have not tried. you may use the project to replace the signature and try download the video, if it works perfectly， welcome to sumbit your code to optimize this project