-
Hi djm:
Since scrapyd server may be down , so while you using this scrapyd api , you may be block by the http error or other error, need to add the timeout param to handle this situation.
-
Hello, when i run `pip install mysqlclient` i get:
> building '_mysql' extension
> creating build/temp.linux-x86_64-3.6
> x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -…
-
Current Spider Scheduling allows only 1 spider running at a time and to be run after a fixed interval on the assumption that spider has finished operation before next schedule is called. This is ineff…
-
IMHO in scrapyd API, there should be some option to know the state of job like (pending, running and finished) on the basis of job id. Something like
result=scrapyd.state("job_id")
Thanks
-
Hi,
I'm trying Hyphe (I did the advanced install on Ubuntu 14.04). I can see a web page on localhost/hyphe ( with this message : YOU ARE DISCONNECTED FROM HYPHE :-( ).
When I try `bin/hyphe start`…
-
Dear all and dear @oncletom ,
I'am a Linux user, but in a flavor not well supported (yet) by hyphe (Fedora 22). So I want take the docker solution... my procedure is :
```
sudo systemctl start dock…
ElCep updated
6 years ago
-
I have read all issues there, but I do not find the answer to solve my problem.
I create a scrapyd server on Ubuntu 14.04 LTS version, use nginx with basic authentication
and managed by supervisor.…
-
Bravo! for the packaging work done through docker
I just succeeded installing it on my OSX 10.8.5 (very last master version from github this day)
the frontend seems to work, but when i create a new p…
-
The documentation is a little simple and does not explain how to integrate with Kafka and Hbase for a fully distributed architecture.
Could you, please provide an example in the examples folder of a w…
-
Hi, We are deploying the scrapy on docker (scrapyd (1.1.0), scrapyd-client (1.0.1))
we restart(cronjob) the job by send message to 'http://0.0.0.0:6800/cancel.json' and got
Traceback (most recent c…