-
It looks like it is not possible to configure Hyphe to use a proxy identified by an IP address.
**Context**
- Dockerized Hyphe running on a VM
- Proxy installed and running on the VM, bound to 12…
-
I use scrapyd on windows, I successfully installed it, but when I try to use the API I get an error that is related to the environment variables:
it happens in addversion call:
```
2014-11-12 17:52:…
-
after I upload an egg file to scrapyd server and request for scrapyd's listspiders API, it returns me the names of spiders in the egg file. But how can I get other attrs of the spiders? like `allowed_…
-
**Summary**
When a rather "data-heavy" keyword argument is passed into `POST schedule.json` action, it yields the following error:
```shell
... b'-a', b'_job=e40210a6c52911ea9de2d8f2ca734204'] in e…
-
Hello, after reading the documentation I couldn't find if it is possible to run a function/script as soon as a job is finished.
Let's say my data flow requires to:
1. Launch a spider and scrape…
-
```
2020-04-27 17:50:34 [scrapy.utils.signal] ERROR: Error caught on signal handler:
Traceback (most recent call last):
File "/home/ocdskfs/scrapyd/.ve/lib/python3.6/site-packages/twisted/intern…
-
Recent runs of the `uk_contracts_finder` spider (collections `1883` and `1889`) returned lots of HTTP 429 errors, presumably due to rate limiting.
Can the spider be updated to account for this?
…
-
Hi everyone! After I succeed in installing Docker in my Windows computer, I try to follow this instruction:
2. Download Hyphe
Collect Hyphe's source code from this git repository (recommended way…
-
I'm trying to run my spider via scrapyd_api like follow.
` scrapyd = ScrapydAPI('http://localhost:6800')
spider_pid = scrapyd.schedule(
'default',
…
-
* [x] I think we should add inline docs to settings.py to document the variables, which is a much more reasonable place for a user to configure their settings. Having an `env.sh` file that they need t…