-
There is a good deal of confusion among users when they encounter the following errors due to scrapyd not being installed. See [1](https://github.com/scrapinghub/portia/issues/786),[2](https://www.bo…
-
This is the Epic User Story to create a crawler system for iFiltr, it should be broken down to several other user stories and additional tasks for each user story.
1) Introduction
iFiltr will index …
-
When i set a schedule on a spider, i set "max_instances" to 1 and "coalesce" as "True", but it seems not work. After a moment, the spider has more than one instance are running.
-
微博内容精选
-
1. Started Portia with Docker: `docker run -v ~/portia_projects:/app/data/projects:rw -p 9001:9001 scrapinghub/portia`
2. Configured some test-spider
3. Tried to actually run the spider (from the UI…
-
i use scrapyd to manage the job but when the job is finished ,it also exist in the joblist and the process of python is persistent there ,no i have a question, when i renew a ne w job it will existed …
-
**Describe the bug**
Ive set the option DATABASE_URL to support MySQL in a correct format and restart scrapydweb,but no DBS in [DB_APSCHEDULER, DB_TIMERTASKS, DB_METADATA, DB_JOBS] had been created a…
-
[ScrapydWeb](https://github.com/my8100/scrapydweb) is a full-featured web UI for Scrapyd cluster management, with Scrapy log analysis & visualization supported. Other features include Basic auth, Auto…
-
The Jobs table in the web interface is really bare. The Scrapy stats collector contains a lot of valuable data, which should be included in this table.
I see a few ways of accessing this data:
1. Par…
-
(From discussion from https://github.com/open-contracting/kingfisher-process/pull/152#issuecomment-491908018 )