-
我在Ubuntu 12.04中按照说明配置了单mongodb的环境,但是在运行时报错如下:
/woaidu_crawler/woaidu_crawler/spiders/woaidu_detail_spider.py:12: ScrapyDeprecationWarning: woaidu_crawler.spiders.woaidu_detail_spider.WoaiduSpider inh…
-
$ scrapy crawl woaidu
Traceback (most recent call last):
File "/usr/local/bin/scrapy", line 4, in
execute()
File "/usr/local/lib/python2.7/dist-packages/scrapy/cmdline.py", line 143, in exec…
-
使用Github抓取博客链接、使用mongodb存储数据,在抓取阶段出现问题
`https://blog.akimio.top/links/`是用的是`butterfly`魔改主题(solitude)[https://github.com/everfu/hexo-theme-solitude],之前是可以正常抓取的,**一开始我怀疑是主题的问题,找了一个原版butterfly主题的友链,还是出现…
-
Similar to #30 , but i use latest version of scrapy-fake-useragent 1.4.4
here is my `setting.py` :
```
DOWNLOADER_MIDDLEWARES = {
'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware'…
-
Like in Scrapy https://github.com/scrapy/scrapy/blob/c316ca45a5b1b19622c96049c9378d8c45adba60/scrapy/crawler.py#L255
We'd need to set up a communication method between the threads and the main thre…
-
Salve, purtroppo oggi volevo grattare di nuovo i dati ma al momento lo script sembra non estrarre piu nulla..
Vi mostro qui sotto cosa scrive:
Hi, unfortunately today I wanted to scratch the data a…
-
D:\weibo\weibo-search-master>scrapy crawl search
Traceback (most recent call last):
File "D:\Programs\Python\Python38\lib\runpy.py", line 193, in _run_module_as_main
return _run_code(code, ma…
-
starbook:scrapy star$ pwd
/Users/star/project/docker/AntSpider/scrapy
starbook:scrapy star$ scrapy list
Traceback (most recent call last):
File "/Users/star/Library/Python/3.7/bin/scrapy", lin…
-
D:\weibo\weibo-search-master>scrapy crawl search
Traceback (most recent call last):
File "D:\Programs\Python\Python38\lib\runpy.py", line 193, in _run_module_as_main
return _run_code(code, ma…
-
Chrome driver