-
I'm trying to run malspider on a remote server. I'm accessing the dashboard just fine using "serveripaddress":8080. However, all the links from the dashboard that are supposed to lead to the spider da…
-
I have a project created by portia. I want to config database for this project because portia isn't support store data to database so I installed portia2code to convert portia project to scrapy. But w…
-
I'm using `RabbitMQ` with `Scrapy`and when the spider finished, the Job on `Scrapyd` stay on Running status.
Can someone help me to fix that ?
```
2016-10-06 14:05:27 [scrapy] INFO: Scrapy 1.2.0 star…
-
I followed the document at: http://doc.scrapy.org/en/latest/topics/ubuntu.html
But when I tried to run `sudo apt-get update && sudo apt-get install scrapy scrapyd`. I got these errors:
```
Reading pa…
-
I am having a use case scenario where we are running new job for each task on same domain
I am handling this using `scrapyd` and `schedule.json` API
like
```
curl http://localhost:6800/schedule.json…
-
seems this line is actually fail to dectect cffi
https://github.com/heroku/heroku-buildpack-python/blob/master/bin/steps/cryptography#L23
and please add scrapy into the list after fixed this problem.…
-
I really like your book, but I have a question: Are there any possibilities to self-host scraping hub? A lot of people have their own infrastructure and so it would be nice to use it. Could you recomm…
-
Inside webservice.py -> lListJobs function:
In the newest code in the repository:
```
running = [{"id": s.job, "spider": s.spider,
"start_time": s.start_time.isoformat(' ')} for s in spid…
-
I was naively getting an argument from the command line and it was ok, but when I moved the project to scrapyd I noticed the arguments were being urlencoded when sent by python-requests. I needed to u…
-
The pypi scrapyd version does not contain component scrapyd.webservice.DaemonStatus.