dataabc / weibo-crawler

新浪微博爬虫,用python爬取新浪微博数据,并下载微博图片和微博视频
3.38k stars 754 forks source link

批量下载时,中间有提示错误,但是下载还是继续,其他微博的下载也正常 #417

Open wangshiyijiu opened 7 months ago

wangshiyijiu commented 7 months ago

string indices must be integers, not 'str' Traceback (most recent call last): File "Y:\weibo-crawler\weibo.py", line 883, in get_one_weibo weibo = self.parse_weibo(weibo_info) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "Y:\weibo-crawler\weibo.py", line 782, in parse_weibo weibo["pics"] = self.get_pics(weibo_info) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "Y:\weibo-crawler\weibo.py", line 448, in get_pics pic_list = [pic["large"]["url"] for pic in pic_info]


TypeError: string indices must be integers, not 'str'
dataabc commented 7 months ago

可能是个别微博出错,别的微博正常应该就没问题了。

wangshiyijiu commented 7 months ago

新出现了一个错误,我看不太懂,麻烦问下是啥原因 --- Logging error --- Traceback (most recent call last): File "C:\Program Files\Python312\Lib\logging__init.py", line 1164, in emit self.flush() File "C:\Program Files\Python312\Lib\logging__init__.py", line 1144, in flush self.stream.flush() OSError: [Errno 22] Invalid argument Call stack: File "Y:\weibo-crawler\weibo.py", line 2080, in main() File "Y:\weibo-crawler\weibo.py", line 2070, in main wb.start() # 爬取微博信息 File "Y:\weibo-crawler\weibo.py", line 2029, in start self.get_pages() File "Y:\weibo-crawler\weibo.py", line 1950, in get_pages is_end = self.get_one_page(page) File "Y:\weibo-crawler\weibo.py", line 1222, in get_one_page logger.info( Message: '已获取用户 戚薇工作室 的微博,内容为 #我是杜拉拉#大伟哥你说你那么爱我们拉拉,就不能分哪怕一点点给剧照师嘛~整理照片时发现镜头都要被瞪了稀碎啦\n。@王耀庆工作室 ' Arguments: () 已获取用户 戚薇工作室 的微博,内容为 三岁生日快乐哦~ --- Logging error --- Traceback (most recent call last): File "C:\Program Files\Python312\Lib\logging\init.py", line 1164, in emit self.flush() File "C:\Program Files\Python312\Lib\logging\init__.py", line 1144, in flush self.stream.flush() OSError: [Errno 22] Invalid argument Call stack: File "Y:\weibo-crawler\weibo.py", line 2080, in main() File "Y:\weibo-crawler\weibo.py", line 2070, in main wb.start() # 爬取微博信息 File "Y:\weibo-crawler\weibo.py", line 2029, in start self.get_pages() File "Y:\weibo-crawler\weibo.py", line 1950, in get_pages is_end = self.get_one_page(page) File "Y:\weibo-crawler\weibo.py", line 1222, in get_one_page logger.info( Message: '已获取用户 戚薇工作室 的微博,内容为 三岁生日快乐哦~' Arguments: ()

dataabc commented 7 months ago

看起来是写日志时出错,可能是message里有不能处理的乱码猜测,如果不影响csv文件等,就没问题。