-
1. 直接在web管理界面添加scrapyd主机，并且支持有https连接的scrapyd主机
2. 添加中文语言支持
3. 能够配置各种请求的超时时间与重试次数
-
In Project Manage page, only can add one project. If can add more than one project?
And, i know, scrapyd can add several scrapy project.
-
linux：HTTPConnectionPool(host='192.168.0.24', port=6801): Max retries exceeded with url: /listprojects.json (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 111] Connectio…
-
使用系统: centos7
问题复现
1.使用auto_manage_spiders.py提示正常上传，但系统上无显示
(base) [root@localhost scrapyd_web_manager]# python auto_manage_spiders.py -dp
deploy True
POST Fetch: http://192.168.1.94:5000/1/d…
-
I ran your code according to the instructions in the readme. I can view the responses in the logs directory in scrapy_app but when I open up my sqlite3 prompt, there are no tables or databases that ha…
-
I used jmeter locally to conduct stress testing on Python, targeting both cases of having and not having tracing. The stress testing results are as follows：
test performance tool：jmeter
concurre…
-
## Summary
With the `JOBDIR` setting the documentation states each spider should utilize its own directory, while there is nothing currently in place to automatically handle this as there is fo…
-
Utlimate goal is to be able to store, version and deploy rulesets between development, test and production environments. This story should implement the basics for this goal -- as soon as we can seria…
-
Hello,
Thank your for your fantastic project. We are facing a really hard to solve bug while running scapy inside celery task. Sometimes we get this error:
```
Unhandled Error
Traceback (most re…
-
Currently, Docker / Kubernetes logs are used for logging. This is sometimes good enough, but in many situations not. These logs are often truncated at night (and potentially more often when grown to a…