-
The only parts of Scrapy that we take advantage of are the scheduler and the downloader. Its management of crawlers and spiders doesn't add anything to our usecase, and the abstractions provided add c…
-
**请描述该需求尝试解决的问题**
Hello,
I'd like to suggest to improve git sync functionality in order to make it possible for scenarios where there are dozens (or even hundreds) of spiders. Currently the function…
-
Imported from JIRA [DS-3673] created by bram
We have been working together with COUNTER to move the management of their botlist to Github. The result is available at: https://github.com/atmire/COUNTER…
-
I believe that being able to register docker instances without exported ports (suitably tagged) should be one of the things that registrator supports.
Use cases for such dockers would be portless do…
-
I have a brief question: I have been looking for some type of reliable method which would allow specific scrapy spiders to be assigned to a "realtime" style method of execution. More specifically: I o…
-
```
Python 3.9.13
Daphne 4.0.0
Django 4.1.2
Channels 4.0.0
Scrapy 2.7.0
scrapy-playwright 0.0.22
```
My settings:
```python
DOWNLOAD_HANDLERS = {
"http": "scrapy_playwright.handler.Sc…
-
[crash-2024-08-25_23.41.58-server.txt](https://github.com/user-attachments/files/16744118/crash-2024-08-25_23.41.58-server.txt)
-
Currently don't see any SSL support for crawling sites with SSL enabled?
-
Hi @Insutanto
Great! Your work is really impressive. However, I would like to add some suggestions.
First of all, I wanna open the front console of RabbitMQ(http://127.0.0.1:15672), but it didn'…
-
[ansible-role-ara_api](https://github.com/ansible-community/ara-collection/tree/master/roles/ara_api) currently supports a few different deployment options:
- with or without gunicorn to launch the d…