dataabc / weiboSpider

新浪微博爬虫,用python爬取新浪微博数据
8.14k stars 1.95k forks source link

程序运行__main___出错 #582

Open NocTurn294 opened 1 month ago

NocTurn294 commented 1 month ago

为了更好的解决问题,请认真回答下面的问题。等到问题解决,请及时关闭本issue。

答:python3.12

答:是

答:是

答:否

答:

答:运行_main__.py 或者运行终端命令都会出现这个问题,


Traceback (most recent call last):
  File "D:\code\PythonCode\spider_2\weiboSpider\weibo_spider\parser\index_parser.py", line 37, in get_user
    self.user.weibo_num = string_to_int(user_info[0][3:-1])
                                        ~~~~~~~~~^^^
IndexError: list index out of range
None
****************************************************************************************************
'NoneType' object has no attribute 'nickname'
Traceback (most recent call last):
  File "D:\code\PythonCode\spider_2\weiboSpider\weibo_spider\spider.py", line 228, in _get_filepath
    dir_name = self.user.nickname
               ^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'nickname'
expected str, bytes or os.PathLike object, not NoneType
Traceback (most recent call last):
  File "D:\code\PythonCode\spider_2\weiboSpider\weibo_spider\writer\csv_writer.py", line 25, in __init__
    with open(self.file_path, 'a', encoding='utf-8-sig',
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: expected str, bytes or os.PathLike object, not NoneType
****************************************************************************************************
Progress:   0%|          | 0/1 [00:02<?, ?it/s]
list index out of range
Traceback (most recent call last):
  File "D:\code\PythonCode\spider_2\weiboSpider\weibo_spider\parser\page_parser.py", line 66, in get_one_page
    is_exist = info[0].xpath("div/span[@class='ctt']")
               ~~~~^^^
IndexError: list index out of range
cannot unpack non-iterable NoneType object
Traceback (most recent call last):
  File "D:\code\PythonCode\spider_2\weiboSpider\weibo_spider\spider.py", line 180, in get_weibo_info
    weibos, self.weibo_id_list, to_continue = PageParser(
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: cannot unpack non-iterable NoneType object```
dataabc commented 1 month ago

可能是爬取速度太快,被暂时限制了。一般情况下,一段时间后限制会自动解除。可通过降低爬取速度避免被限制。

MarthaWWW commented 1 month ago

出现了一样的问题

QinXie123 commented 1 month ago

出现了一样的问题,降低爬取速度,隔了几天均无法进行爬取

MarthaWWW commented 1 month ago

更换在不同浏览器的cookie就行,edge的cookie不行就换chrome,亲测有用

xiaoyequ04 commented 1 month ago

出现了一样的问题