-
-
Hi,
I was wondering if it possible to add custom endpoints scrapyd? So to extend the current endpoints by allowing people to write one themselves?
Or perhaps if anyone knows if any workaround t…
-
I am maintaining a different table with all the finished tasks in my Django project, so I would like to delete the tables in the scrapyd finished jobs list. Then I can move a task to my table and dele…
-
When using the cancel REST API method, the crawler process is terminated without calling the registered shutdown handler (spider_closed), at least on Windows. This is my code:
class SpiderCtlExten…
-
Figure out how to obtain scrapyd-k8s' deployment name + current namespace from within scrapyd-k8s.
This will probably [need the Downward API](https://stackoverflow.com/questions/42274229/kubernetes-de…
-
## the scrapy understand
Scrapy是一个应用程序框架，用于对网站进行爬行和提取结构化数据，这些结构化数据可用于各种有用的应用程序，如数据挖掘、信息处理或历史存档。
#### 创建项目
cmd运行`scrapy startproject tutorial`,新建一个项目
创建一个tutorial目录：
tutorial/
scrapy.cfg 部署配…
-
First - thanks for publishing this buildpack! I've been able to get it to work, which is incredibly useful.
One thing I've noticed is that whenever I `heroku run bash -a myappname` - for example, t…
-
Add integration tests with Github Actions.
This would setup the cluster, test all endpoints (incl. running the example spider #1 or a test spider returning static data, checking results).
This would …
-
微博内容精选
-
https://docs.scrapy.org/en/latest/topics/items.html
Once we deploy to Python >= 3.7.