-
Hi,
I’m trying to integrate Elasticsearch with Scrapy. I’ve followed the steps from https://github.com/knockrentals/scrapy-elasticsearch,
but it’s not loading the pipeline. Im using Scrapy 1.3.3 w…
-
Is the new portia scrape next page data?
I build spider in portia with nextpage recorded option which is run in docker, but when i deployed it in scrapyd there is no data scraped i got output like
20…
-
I wanted to deploy my Portia Project on Scrapyd. However, I received the following error:
```
C:\Users\eric\Desktop\project>scrapyd-deploy
Packing version 1480440269
Traceback (most recent call …
-
I am using scrapy to scrape multiple sites and Scrapyd to run spiders.
I had written 7 spiders and each spider processes at least 50 start URLs. I have around 7000 URL's. 1000 URL's for each spider…
-
按照配置已配置好,目前在spiders选项页面无相关显示~不清楚是什么原因,请指教谢谢拉
-
Scrapyd was under an unspecific release policy over the last few years.
Bugfixes and new features were being merged to master directly
followed by occasional version bumps without backports.
I wa…
-
## useage
``` python
import commands
commands.getstatusoutput('scrapyd-deploy vmaxcrawler -p crawlers_husoe')
```
## result
```
Deploying to project "crawlers_husoe" in http://192.168.125.1:6800/add…
-
When trying to deploy to all targets using -a, scrapyd-deploy returns that it has no option -a, this option is also no listed in help
-
- 使用scrapyd进行打包部署
- 学习下RESTful的概念
-
I am going to revert both commits from #10 because of
https://github.com/scrapy/scrapyd-client/pull/8#issuecomment-135969830 and https://github.com/mariotux/scrapyd-client/commit/c8ea75596c8e62f3da84…