-
First of all: thanks for your great project.
In "both" mode, I see 78 wemportal entities and I am bit lost between them. Some of them are in german, some in english due to issue #22. Some of them hav…
-
## Comportamento Esperado
Espera-se que, ao final da execução de um coletor dinâmico, não tenhamos a exceção: `RuntimeError: This event loop is already running
`
## Comportamento Atual
O coletor é…
-
Hello
using any example given in the example.py
I am getting 407 error on a batch of valid proxies.
i ran an example with just scrapy and one with just requests, both work with the same proxy t…
-
I'm trying to use scrapy-playwright with firefox and proxies and it's not easy.
In Playwright-Python and Node as well just passing proxy config to server is not enough because authorization header…
-
-
您的异步操作非常及时！不过我最近加了一个心跳机制，就是因为我爬虫的过程有时需要很长时间，而中间前后端没有通信，我发现每过15分钟通信就会自动断开。
然而加的心跳机制在爬虫运行过程中时触发不了的，所以考虑用您的异步操作，我的路径只有一条，我将它设为异步路径，然后在爬虫的函数前加入await，还是没有达到想要的效果，爬虫过程中并不会触发心跳。
代码大致如下，请指教：
```
@route(…
-
I have the following spider:
```python
# Python
from typing import Any, Dict, Iterator, List
from urllib.parse import urlparse
# 3rd Party
from scrapy.spiders import CrawlSpider, Rule
from …
dsism updated
2 years ago
-
Sometimes we get
`'OK' not in '/home/runner/work/scrapy/scrapy/.tox/asyncio/lib/python3.10/site-packages/coverage/inorout.py:472: CoverageWarning: --include is ignored because --source is set (incl…
-
I have a threading Web spider that is triggering a race condition in anyio/_core/_eventloop.py's get_asynclib() function. This race can be triggered if you have a lot of threads trying to initialize t…
-
Using the default config from the readme Im getting an asyncio error.
Python 3.9.10
Scrapy 2.5.1
scrapy-requests 0.2.0
```
Traceback (most recent call last):
File "/Users/jquick/.virtual…