提示TypeError: expected string or bytes-like object

[lisicheng1997]

C:\Users\lisicheng\weibo-search\weibo>scrapy crawl search -s JOBDIR=crawls/search 2022-09-11 18:16:22 [scrapy.core.scraper] ERROR: Spider error processing <GET https://s.weibo.com/weibo?q=%E8%BF%AA%E4%B8%BD%E7%83%AD%E5%B7%B4&scope=ori&suball=1&timescope=custom:2020-03-01-0:2020-03-02-0> (referer: None) Traceback (most recent call last): File "c:\users\lisicheng\miniconda3\lib\site-packages\scrapy\utils\defer.py", line 132, in iter_errback yield next(it) File "c:\users\lisicheng\miniconda3\lib\site-packages\scrapy\utils\python.py", line 354, in next return next(self.data) File "c:\users\lisicheng\miniconda3\lib\site-packages\scrapy\utils\python.py", line 354, in next return next(self.data) File "c:\users\lisicheng\miniconda3\lib\site-packages\scrapy\core\spidermw.py", line 66, in _evaluate_iterable for r in iterable: File "c:\users\lisicheng\miniconda3\lib\site-packages\scrapy\spidermiddlewares\offsite.py", line 29, in process_spider_output for x in result: File "c:\users\lisicheng\miniconda3\lib\site-packages\scrapy\core\spidermw.py", line 66, in _evaluate_iterable for r in iterable: File "c:\users\lisicheng\miniconda3\lib\site-packages\scrapy\spidermiddlewares\referer.py", line 342, in return (_set_referer(r) for r in result or ()) File "c:\users\lisicheng\miniconda3\lib\site-packages\scrapy\core\spidermw.py", line 66, in _evaluate_iterable for r in iterable: File "c:\users\lisicheng\miniconda3\lib\site-packages\scrapy\spidermiddlewares\urllength.py", line 40, in return (r for r in result or () if _filter(r)) File "c:\users\lisicheng\miniconda3\lib\site-packages\scrapy\core\spidermw.py", line 66, in _evaluate_iterable for r in iterable: File "c:\users\lisicheng\miniconda3\lib\site-packages\scrapy\spidermiddlewares\depth.py", line 58, in return (r for r in result or () if _filter(r)) File "c:\users\lisicheng\miniconda3\lib\site-packages\scrapy\core\spidermw.py", line 66, in _evaluate_iterable for r in iterable: File "C:\Users\lisicheng\weibo-search\weibo\spiders\search.py", line 107, in parse for weibo in self.parse_weibo(response): File "C:\Users\lisicheng\weibo-search\weibo\spiders\search.py", line 425, in parse_weibo attitudes_count = re.findall(r'\d+.*', attitudes_count) File "c:\users\lisicheng\miniconda3\lib\re.py", line 241, in findall return _compile(pattern, flags).findall(string) TypeError: expected string or bytes-like object

C:\Users\lisicheng\weibo-search\weibo>

重装也未能解决问题，在旧电脑里旧版本跑只能获取微量数据就结束。我自己看微博搜索界面存在38页结果，非常好奇是哪里出现了问题，麻烦各位老师了

[dataabc]

感谢反馈,是最新版本吗？

[lisicheng1997]

是的，前天git clone的版本

发自我的iPhone

在 2022年9月12日，21:50，Chen Lei @.***> 写道：

 感谢反馈,是最新版本吗？

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.

[yitiaojie]

感谢反馈,是最新版本吗？

我也出现这样的问题了

[lisicheng1997]

收到，辛苦大佬了。我试一下

------------------ 原始邮件 ------------------ 发件人: @.***>; 发送时间: 2022年9月13日(星期二) 凌晨0:48 收件人: "dataabc/weibo-search"; 抄送: "Author"; 主题: Re: [dataabc/weibo-search] 提示TypeError: expected string or bytes-like object (Issue #268)

参考另一个回答 把attitudes_count 改成 str(attitudes_count) 试试

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>

[lisicheng1997]

修改后就可以用了，微量数据的问题也没有再出现。当前页面搜索为空 不影响整体数据爬取

[lisicheng1997]

替换掉 search.py中两行这个参数就可以了 ，不过会有部分当前页面搜索结果为空，但是应该没什么影响

发自我的iPhone

在 2022年9月13日，00:48，little rubbish @.***> 写道：

 参考另一个回答 把attitudes_count 改成 str(attitudes_count) 试试

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.