search

dataabc / weibo-search

获取微博搜索结果信息，搜索即可以是微博关键词搜索，也可以是微博话题搜索
1.72k stars 375 forks source link

OSError: [WinError 87] 参数错误。 #50

Open callmeYe opened 3 years ago

callmeYe commented 3 years ago 
{'created_at': '2020-06-12 13:52',
 'id': '4514990885256149',
 'location': '',
 'text': '#严浩翔时代少年团主rap担#yhx#严浩翔镜头雷达#�第一rapper��严浩翔��飒人气场��严浩翔��美颜如画��严浩翔��唱跳俱佳��严浩翔?2020
-12-09 14:01:11 [scrapy.core.scraper] ERROR: Spider error processing <GET https://s.weibo.com/weibo?q=%E5%8F%B0%E9%A3%8E&region=custom:51:1000&scop
e=ori&suball=1&timescope=custom:2020-08-19-6:2020-08-19-7&page=1> (referer: https://s.weibo.com/weibo?q=%E5%8F%B0%E9%A3%8E&scope=ori&suball=1&times
cope=custom:2020-08-19-6:2020-08-19-7&page=1)
Traceback (most recent call last):
  File "d:\programdata\anaconda3\lib\site-packages\scrapy\utils\defer.py", line 120, in iter_errback
    yield next(it)
  File "d:\programdata\anaconda3\lib\site-packages\scrapy\utils\python.py", line 353, in __next__
    return next(self.data)
  File "d:\programdata\anaconda3\lib\site-packages\scrapy\utils\python.py", line 353, in __next__
    return next(self.data)
  File "d:\programdata\anaconda3\lib\site-packages\scrapy\core\spidermw.py", line 62, in _evaluate_iterable
    for r in iterable:
  File "d:\programdata\anaconda3\lib\site-packages\scrapy\spidermiddlewares\offsite.py", line 29, in process_spider_output
    for x in result:
  File "d:\programdata\anaconda3\lib\site-packages\scrapy\core\spidermw.py", line 62, in _evaluate_iterable
    for r in iterable:
  File "d:\programdata\anaconda3\lib\site-packages\scrapy\spidermiddlewares\referer.py", line 340, in <genexpr>
    return (_set_referer(r) for r in result or ())
  File "d:\programdata\anaconda3\lib\site-packages\scrapy\core\spidermw.py", line 62, in _evaluate_iterable
    for r in iterable:
  File "d:\programdata\anaconda3\lib\site-packages\scrapy\spidermiddlewares\urllength.py", line 37, in <genexpr>
    return (r for r in result or () if _filter(r))
  File "d:\programdata\anaconda3\lib\site-packages\scrapy\core\spidermw.py", line 62, in _evaluate_iterable
    for r in iterable:
  File "d:\programdata\anaconda3\lib\site-packages\scrapy\spidermiddlewares\depth.py", line 58, in <genexpr>
    return (r for r in result or () if _filter(r))
  File "d:\programdata\anaconda3\lib\site-packages\scrapy\core\spidermw.py", line 62, in _evaluate_iterable
    for r in iterable:
  File "E:\weibo-search\weibo\spiders\search.py", line 237, in parse_by_hour_province
    for weibo in self.parse_weibo(response):
  File "E:\weibo-search\weibo\spiders\search.py", line 517, in parse_weibo
    print(weibo)
OSError: [WinError 87] 参数错误。
{'created_at': '2020-06-12 13:52',
 'id': '4514990885256149',
 'location': '',
 'text': '#严浩翔时代少年团主rap担#yhx#严浩翔镜头雷达#�第一rapper��严浩翔��飒人气场��严浩翔��美颜如画��严浩翔��唱跳俱佳��严浩翔?2020
-12-09 14:01:17 [scrapy.core.scraper] ERROR: Spider error processing <GET https://s.weibo.com/weibo?q=%E5%8F%B0%E9%A3%8E&region=custom:44:53&scope=
ori&suball=1&timescope=custom:2020-08-19-5:2020-08-19-6&page=1> (referer: https://s.weibo.com/weibo?q=%E5%8F%B0%E9%A3%8E&region=custom:44:1000&scop
e=ori&suball=1&timescope=custom:2020-08-19-5:2020-08-19-6&page=1)
Traceback (most recent call last):
  File "d:\programdata\anaconda3\lib\site-packages\scrapy\utils\defer.py", line 120, in iter_errback
    yield next(it)
  File "d:\programdata\anaconda3\lib\site-packages\scrapy\utils\python.py", line 353, in __next__
    return next(self.data)
  File "d:\programdata\anaconda3\lib\site-packages\scrapy\utils\python.py", line 353, in __next__
    return next(self.data)
  File "d:\programdata\anaconda3\lib\site-packages\scrapy\core\spidermw.py", line 62, in _evaluate_iterable
    for r in iterable:
  File "d:\programdata\anaconda3\lib\site-packages\scrapy\spidermiddlewares\offsite.py", line 29, in process_spider_output
    for x in result:
  File "d:\programdata\anaconda3\lib\site-packages\scrapy\core\spidermw.py", line 62, in _evaluate_iterable
    for r in iterable:
  File "d:\programdata\anaconda3\lib\site-packages\scrapy\spidermiddlewares\referer.py", line 340, in <genexpr>
    return (_set_referer(r) for r in result or ())
  File "d:\programdata\anaconda3\lib\site-packages\scrapy\core\spidermw.py", line 62, in _evaluate_iterable
    for r in iterable:
  File "d:\programdata\anaconda3\lib\site-packages\scrapy\spidermiddlewares\urllength.py", line 37, in <genexpr>
    return (r for r in result or () if _filter(r))
  File "d:\programdata\anaconda3\lib\site-packages\scrapy\core\spidermw.py", line 62, in _evaluate_iterable
    for r in iterable:
  File "d:\programdata\anaconda3\lib\site-packages\scrapy\spidermiddlewares\depth.py", line 58, in <genexpr>
    return (r for r in result or () if _filter(r))
  File "d:\programdata\anaconda3\lib\site-packages\scrapy\core\spidermw.py", line 62, in _evaluate_iterable
    for r in iterable:
  File "E:\weibo-search\weibo\spiders\search.py", line 274, in parse_page
    for weibo in self.parse_weibo(response):
  File "E:\weibo-search\weibo\spiders\search.py", line 517, in parse_weibo
    print(weibo)
OSError: [WinError 87] 参数错误。
{'created_at': '2020-06-12 13:52',
 'id': '4514990885256149',
 'location': '',
 'text': '#严浩翔时代少年团主rap担#yhx#严浩翔镜头雷达#�第一rapper��严浩翔��飒人气场��严浩翔��美颜如画��严浩翔��唱跳俱佳��严浩翔?2020
-12-09 14:01:28 [scrapy.core.scraper] ERROR: Spider error processing <GET https://s.weibo.com/weibo?q=%E5%8F%B0%E9%A3%8E&region=custom:44:52&scope=
ori&suball=1&timescope=custom:2020-08-19-5:2020-08-19-6&page=1> (referer: https://s.weibo.com/weibo?q=%E5%8F%B0%E9%A3%8E&region=custom:44:1000&scop
e=ori&suball=1&timescope=custom:2020-08-19-5:2020-08-19-6&page=1)
Traceback (most recent call last):
  File "d:\programdata\anaconda3\lib\site-packages\scrapy\utils\defer.py", line 120, in iter_errback
    yield next(it)
  File "d:\programdata\anaconda3\lib\site-packages\scrapy\utils\python.py", line 353, in __next__
    return next(self.data)
  File "d:\programdata\anaconda3\lib\site-packages\scrapy\utils\python.py", line 353, in __next__
    return next(self.data)
  File "d:\programdata\anaconda3\lib\site-packages\scrapy\core\spidermw.py", line 62, in _evaluate_iterable
    for r in iterable:
  File "d:\programdata\anaconda3\lib\site-packages\scrapy\spidermiddlewares\offsite.py", line 29, in process_spider_output
    for x in result:
  File "d:\programdata\anaconda3\lib\site-packages\scrapy\core\spidermw.py", line 62, in _evaluate_iterable
    for r in iterable:
  File "d:\programdata\anaconda3\lib\site-packages\scrapy\spidermiddlewares\referer.py", line 340, in <genexpr>
    return (_set_referer(r) for r in result or ())
  File "d:\programdata\anaconda3\lib\site-packages\scrapy\core\spidermw.py", line 62, in _evaluate_iterable
    for r in iterable:
  File "d:\programdata\anaconda3\lib\site-packages\scrapy\spidermiddlewares\urllength.py", line 37, in <genexpr>
    return (r for r in result or () if _filter(r))
  File "d:\programdata\anaconda3\lib\site-packages\scrapy\core\spidermw.py", line 62, in _evaluate_iterable
    for r in iterable:
  File "d:\programdata\anaconda3\lib\site-packages\scrapy\spidermiddlewares\depth.py", line 58, in <genexpr>
    return (r for r in result or () if _filter(r))
  File "d:\programdata\anaconda3\lib\site-packages\scrapy\core\spidermw.py", line 62, in _evaluate_iterable
    for r in iterable:
  File "E:\weibo-search\weibo\spiders\search.py", line 274, in parse_page
    for weibo in self.parse_weibo(response):
  File "E:\weibo-search\weibo\spiders\search.py", line 517, in parse_weibo
    print(weibo)
OSError: [WinError 87] 参数错误。
{'created_at': '2020-06-12 13:52',
 'id': '4514990885256149',
 'location': '',
 'text': '#严浩翔时代少年团主rap担#yhx#严浩翔镜头雷达#�第一rapper��严浩翔��飒人气场��严浩翔��美颜如画��严浩翔��唱跳俱佳��严浩翔?2020
-12-09 14:01:39 [scrapy.core.scraper] ERROR: Spider error processing <GET https://s.weibo.com/weibo?q=%E5%8F%B0%E9%A3%8E&region=custom:44:51&scope=
ori&suball=1&timescope=custom:2020-08-19-5:2020-08-19-6&page=1> (referer: https://s.weibo.com/weibo?q=%E5%8F%B0%E9%A3%8E&region=custom:44:1000&scop
e=ori&suball=1&timescope=custom:2020-08-19-5:2020-08-19-6&page=1)
Traceback (most recent call last):
  File "d:\programdata\anaconda3\lib\site-packages\scrapy\utils\defer.py", line 120, in iter_errback
    yield next(it)
  File "d:\programdata\anaconda3\lib\site-packages\scrapy\utils\python.py", line 353, in __next__
    return next(self.data)
  File "d:\programdata\anaconda3\lib\site-packages\scrapy\utils\python.py", line 353, in __next__
    return next(self.data)
  File "d:\programdata\anaconda3\lib\site-packages\scrapy\core\spidermw.py", line 62, in _evaluate_iterable
    for r in iterable:
  File "d:\programdata\anaconda3\lib\site-packages\scrapy\spidermiddlewares\offsite.py", line 29, in process_spider_output
    for x in result:
  File "d:\programdata\anaconda3\lib\site-packages\scrapy\core\spidermw.py", line 62, in _evaluate_iterable
    for r in iterable:
  File "d:\programdata\anaconda3\lib\site-packages\scrapy\spidermiddlewares\referer.py", line 340, in <genexpr>
    return (_set_referer(r) for r in result or ())
  File "d:\programdata\anaconda3\lib\site-packages\scrapy\core\spidermw.py", line 62, in _evaluate_iterable
    for r in iterable:
  File "d:\programdata\anaconda3\lib\site-packages\scrapy\spidermiddlewares\urllength.py", line 37, in <genexpr>
    return (r for r in result or () if _filter(r))
  File "d:\programdata\anaconda3\lib\site-packages\scrapy\core\spidermw.py", line 62, in _evaluate_iterable
    for r in iterable:
  File "d:\programdata\anaconda3\lib\site-packages\scrapy\spidermiddlewares\depth.py", line 58, in <genexpr>
    return (r for r in result or () if _filter(r))
  File "d:\programdata\anaconda3\lib\site-packages\scrapy\core\spidermw.py", line 62, in _evaluate_iterable
    for r in iterable:
  File "E:\weibo-search\weibo\spiders\search.py", line 274, in parse_page
    for weibo in self.parse_weibo(response):
  File "E:\weibo-search\weibo\spiders\search.py", line 517, in parse_weibo
    print(weibo)
OSError: [WinError 87] 参数错误。

我将search.py里的50改成了45，然后自己注释掉了一些爬取项，程序会成功运行一段时间，直到碰到这条微博然后就卡住了一直报错，不知道这条微博有什么特殊的地方。

dataabc commented 3 years ago

感谢反馈。

因为微博中有特殊符号，print无法打印，注释掉所有print就行了。