-
**Describe the bug**
When using DATABASE_URL from heroku postgres options in scrapydweb, the app crashes because it mistakes the username at the start of the url as the database name when it should b…
-
英文不好，就用中文来描述，请见谅。
连续两天Scrapydweb莫名就挂掉了，通过系统使用情况发现，是因为内存突然在某个时刻耗尽。
进而又发现，这个时间点有从网络读取了大量内容。
目前我在用单台服务器安装Scrapydweb管理其他几个爬虫服务器，并没有做其他事。所以这个网络读取肯定是来自和几个爬虫服务器的交互。进而发现，在同一个时刻有一台爬虫服务器，流出很大数据包。
简单来讲…
-
When I started scrapydweb, I encountered an error:
Unsupported Media Type
Did not attempt to load JSON data because the request Content-Type was not 'application/json'.
-
I'm currently developing locally on windows 10 and have the `SCRAPY_PROJECTS_DIR` setting set to `SCRAPY_PROJECTS_DIR = 'C:/Users/mhill/PycharmProjects/dScrapy/d_webscraping'`
In that directory, I …
-
submit cron job successfully
but not run in time, seems a bug, please test and check
Dockerfile:
FROM python:3.6-slim
ENV DEBIAN_FRONTEND noninteractive
RUN apt update && apt install git…
-
When having auth enabled, my timer tasks stop working.
The response visible in result is:
 are running on a remote host (on same VPN) and the ScrapyDWeb is running on a seperate node then the links to Logs and Items become broken by design.
…
-
Scrapyd has a basic web interface. It would be useful to have something here too, for easy access.
Granted, you can already inspect everything with `docker` or `kubectl`, but a basic web interface st…
-
Hello,
We are using timer tasks from the same spider with different arguments. And when opening the jobs it's not possible to see which arguments was used to run it. Should be nice and I think an i…