-
Scheduled jobs are not run in FIFO+priority order.
Instead, there are multiple queues
that are also arranged in a queue-like fashion
but not round-robin or anything,
just an "arbitrary but constan…
-
使用系统: centos7
问题复现
1.使用auto_manage_spiders.py提示正常上传,但系统上无显示
(base) [root@localhost scrapyd_web_manager]# python auto_manage_spiders.py -dp
deploy True
POST Fetch: http://192.168.1.94:5000/1/d…
-
# Scrapy 搞起篇(2014年8月30日-8月31日)
```
开始于:Beijing,Raining,2014-8-30 22:45:00
结束于:2014-8-31 01:46:00
```
---
## 0. 环境
> python 2.7.8
>
> windows 8.1 x64
**如果接下来用pip安装或者直接安装出现:**
```
UnicodeEncodeErr…
-
今天在项目中放了一个js文件,导致部署失败。。。。
-
Thank you so much for awesome package - i really appreciate it.
Would it possible to disable log's or printing debugging mode. After 10 hours scraping - my log file for that task was more than 2Gb.…
GGPay updated
6 years ago
-
![image](https://user-images.githubusercontent.com/6241705/45284476-4582b480-b513-11e8-879c-127a9497260b.png)
![image](https://user-images.githubusercontent.com/6241705/45284741-d35e9f80-b513-11e8-…
-
I have 3 projects and I have created:
50 jobs for `project_a` like this:
`curl http://localhost:6800/schedule.json -d project=project_a -d spider=spider_name -d priority=0`
Then 50 jobs for `pr…
-
感谢作者,这是我找到的最好的爬虫集群操作平台。提几个需求:
1,如何支持基于scrapy-redis的分布式爬虫的配置、启动?
其他两个小需求:
1,给每个node加描述,方便自己看。
2,通过手机短信发送报警信息。
-
Hi, first, thanks for building SpiderKeeper - it's really easy to use.
We have some scrapers that utilize SK and we want to automate our deployments with a continuous deployment script. The only ma…
-
ERROR - 2022-11-03 22:09:06,050 - process: 14087 - utils.py - gerapy.server.core.utils - 564 - utils - invalid literal for int() with base 10: 'dev-supply_chain_fumei_data_gather-cement'
Traceback (m…