-
when the job exited, it's still in the running list which is returned from listjobs.json.
However, it shows the process has exited when i request the cancel.json.
PS: I used webdriver in this scra…
-
There are 2 binary eggs in master, containing pyc files.
Besides making reviewing harder (need to decompile)
I'd expect them to bug us when testing in different pythons
(the old egg is for py2.6).
Ma…
-
Hi,
I'm honestly unsure if this is expected behavior or not but it's caused me problems with storage: I literally ran out of inodes when `/root/.cache/Python-Eggs` had over five million files.
I…
mas-4 updated
3 months ago
-
This is a feature request.
I wish scrapyd would not run the same Spider at the same time. Maybe this could be a configuration option? I can see 3 behaviours:
- continue scheduling, as it does today
-…
-
The PostgreSQL database is quite large now, because `keep_all_data` was checked, and there's a lot still in Kingfisher Process and Pelican (run `\l+` in `psql`).
There's also a lot of data in Kingf…
-
As described [here](http://www.spect.cl/blog/2015/11/security-audit-scrapyd/) (point 1,2,3 and 5) there are some security issues related to **project name**. I've thought in a fix sanitizing the proje…
-
How to implement the work of scrapyd with the virtual environments of my projects?
For example, I have **scrapyd, scrapykeeper, scrapyd-client** installed in one virtual environment. Another virtual…
-
Allowing to configure scrapyd instance level confirmation values (bind_address, http_port, username, password) via env vars instead of via config file or [command line](https://github.com/scrapy/scrap…
-
The following error occurs when you request http://localhost:6800/schedule.json with Axios:
Error Message
```sh
Expected one of [b'HEAD, b'object, b'POST']
```
The cause is an error caused by…
-
Hi,
I was wondering if it possible to add custom endpoints scrapyd? So to extend the current endpoints by allowing people to write one themselves?
Or perhaps if anyone knows if any workaround t…