-
I ran your code according to the instructions in the readme. I can view the responses in the logs directory in scrapy_app but when I open up my sqlite3 prompt, there are no tables or databases that ha…
-
Hello
When deploy, it shows "Cannot set property 'NOW_DEPLOYED' of undefined".
How to set?
-
英文不好，就用中文来描述，请见谅。
连续两天Scrapydweb莫名就挂掉了，通过系统使用情况发现，是因为内存突然在某个时刻耗尽。
进而又发现，这个时间点有从网络读取了大量内容。
目前我在用单台服务器安装Scrapydweb管理其他几个爬虫服务器，并没有做其他事。所以这个网络读取肯定是来自和几个爬虫服务器的交互。进而发现，在同一个时刻有一台爬虫服务器，流出很大数据包。
简单来讲…
-
python 3.7.1
scrapydweb 1.4.0
Public network access items
公网访问scrapydweb正常，但是items访问显示链接为局域网ip，无法直接通过公网进行访问，这个是无法支持还是可通过配置scrapyd来实现
The public network accesses the scrapedweb normally, but …
-
Stopping jobs mostly works, but it has a number of cases to test.
1. Just created, but not running yet -> remove container without stopping it (not tested)
2. Running -> send signal (tested in PR …
-
I am currently in a situation where I have to create an ssh tunnel to deploy eggs via the scrapyd client, which causes me to deploy through localhost and in turn leads to this error:
> Deploy faile…
-
2023-04-08T15:21:06+0000 [twisted.web.server.Request#critical]
Traceback (most recent call last):
File "/usr/local/lib/python3.11/site-packages/twisted/web/server.py", line 292, i…
-
When i cliked "run spider button" to run project:
requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=6800): Max retries exceeded with url: /schedule.json (Caused by NewCon…
-
-
Scheduled jobs are not run in FIFO+priority order.
Instead, there are multiple queues
that are also arranged in a queue-like fashion
but not round-robin or anything,
just an "arbitrary but constan…