Open zhanglu19970326 opened 1 year ago
换个cmd工具看看。
您好!换了cmder运行,运行结果还是和之前一样(如下),是要再尝试别的运行工具吗?
D:\weibo\weibo-search-master
λ scrapy crawl search -s JOBDIR=crawls/search h
Traceback (most recent call last):
File "D:\Programs\Python\Python38\lib\runpy.py", line 193, in _run_module_as_main
return _run_code(code, main_globals, None,
File "D:\Programs\Python\Python38\lib\runpy.py", line 86, in _run_code
exec(code, run_globals)
File "D:\Programs\Python\Python38\Scripts\scrapy.exe__main.py", line 7, in
File "D:\Programs\Python\Python38\lib\site-packages\scrapy\cmdline.py", line 160, in execute
cmd.crawler_process = CrawlerProcess(settings)
File "D:\Programs\Python\Python38\lib\site-packages\scrapy\crawler.py", line 332, in init
super().init(settings)
File "D:\Programs\Python\Python38\lib\site-packages\scrapy\crawler.py", line 202, in init
self.spider_loader = self._get_spider_loader(settings)
File "D:\Programs\Python\Python38\lib\site-packages\scrapy\crawler.py", line 196, in _get_spider_loader
return loader_cls.from_settings(settings.frozencopy())
File "D:\Programs\Python\Python38\lib\site-packages\scrapy\spiderloader.py", line 72, in from_settings
return cls(settings)
File "D:\Programs\Python\Python38\lib\site-packages\scrapy\spiderloader.py", line 27, in init
self._load_all_spiders()
File "D:\Programs\Python\Python38\lib\site-packages\scrapy\spiderloader.py", line 56, in _load_all_spiders
for module in walk_modules(name):
File "D:\Programs\Python\Python38\lib\site-packages\scrapy\utils\misc.py", line 106, in walk_modules
submod = import_module(fullpath)
File "D:\Programs\Python\Python38\lib\importlib\
return _bootstrap._gcd_import(name[level:], package, level)
File "
File "
File "
File "
File "
File "
File "
File "
File "D:\weibo\weibo-search-master\weibo\spiders\search.py", line 157
else:
^
SyntaxError: invalid syntax
修改过search.py吗?
没有修改,我看现在的代码已经把search.py中的parse_by_hour修改成 这个了: def parse_by_hour(self, response): """以小时为单位筛选""" keyword = response.meta.get('keyword') is_empty = response.xpath( '//div[@class="card card-no-result s-pt20b40"]') if is_empty: print('当前页面搜索结果为空') else:
for weibo in self.parse_weibo(response):
self.check_environment()
yield weibo
next_url = response.xpath(
'//a[@class="next"]/@href').extract_first()
if next_url:
next_url = self.base_url + next_url
yield scrapy.Request(url=next_url,
callback=self.parse_page,
meta={'keyword': keyword})
我看到search.py157行的报错是这样的(如下),我看不太懂 Statement expected,found Py:COLON Statement expected,found Py:ELSE_KEYWORD Unexpected indent Expected type 'str',got 'object'instead start_date =datetime.strptime(start_date_str,%Y-%m-%d-%H) start date str:object date +'-0'
猜测没有对齐,您增删下附近的空格试试。
好的,我自己再摸索摸索,谢谢你~
我直接重新下载了代码,很顺利爬出来了,谢谢你!
D:\weibo\weibo-search-master>scrapy crawl search Traceback (most recent call last): File "D:\Programs\Python\Python38\lib\runpy.py", line 193, in _run_module_as_main return _run_code(code, main_globals, None, File "D:\Programs\Python\Python38\lib\runpy.py", line 86, in _run_code exec(code, run_globals) File "D:\Programs\Python\Python38\Scripts\scrapy.exe__main.py", line 7, in
File "D:\Programs\Python\Python38\lib\site-packages\scrapy\cmdline.py", line 160, in execute
cmd.crawler_process = CrawlerProcess(settings)
File "D:\Programs\Python\Python38\lib\site-packages\scrapy\crawler.py", line 332, in init
super().init(settings)
File "D:\Programs\Python\Python38\lib\site-packages\scrapy\crawler.py", line 202, in init
self.spider_loader = self._get_spider_loader(settings)
File "D:\Programs\Python\Python38\lib\site-packages\scrapy\crawler.py", line 196, in _get_spider_loader
return loader_cls.from_settings(settings.frozencopy())
File "D:\Programs\Python\Python38\lib\site-packages\scrapy\spiderloader.py", line 72, in from_settings
return cls(settings)
File "D:\Programs\Python\Python38\lib\site-packages\scrapy\spiderloader.py", line 27, in init
self._load_all_spiders()
File "D:\Programs\Python\Python38\lib\site-packages\scrapy\spiderloader.py", line 56, in _load_all_spiders
for module in walk_modules(name):
File "D:\Programs\Python\Python38\lib\site-packages\scrapy\utils\misc.py", line 106, in walk_modules
submod = import_module(fullpath)
File "D:\Programs\Python\Python38\lib\importlib\ init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1014, in _gcd_import
File "", line 991, in _find_and_load
File "", line 975, in _find_and_load_unlocked
File "", line 671, in _load_unlocked
File "", line 779, in exec_module
File "", line 916, in get_code
File "", line 846, in source_to_code
File "", line 219, in _call_with_frames_removed
File "D:\weibo\weibo-search-master\weibo\spiders\search.py", line 157
else:
^
SyntaxError: invalid syntax