-
Not working
scrapy 2.7 have to_native_str: https://github.com/scrapy/scrapy/blob/2.7/scrapy/utils/python.py
but deprecated('to_unicode')
I want to fix this error
```shell
(project_name) ➜ …
-
Is there a way to take a screenshot for a `process_spider_exception` error?
I can't figure out how to access the page object in that middleware.
-
### Description
If a middleware raises an exception, running `scrapy crawl` or `scrapy check` raises the exception to the shell but returns with exit code 0, instead of the expected 1.
### Steps…
-
# Overview
From the following PRs:
- https://github.com/zytedata/zyte-spider-templates/pull/41, https://github.com/zytedata/zyte-spider-templates/pull/50
- https://github.com/zytedata/zyte-spider…
-
I think what you are doing in the project is that for every request for product review you are creating a scrapy job. This is compute heavy job and will not handle more than 10 requests at a time on …
-
# Description
If i insert start url to redis before run scrapy, is successful.
But if i run scrapy first and insert url, listen url will get fail info:
```
2023-08-13 17:11:59 [scrapy.utils.…
-
依赖已经全部安装了 但不知道为何还是会报错
2024-07-08 15:46:54 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023
Traceback (most recent call last):
File "D:\WeiboSpider\weibospider\run_spide…
-
Решил разобраться с scrapy но почему то не смог даже запустить ваш проект.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Progr…
-
Запустил пример из README, но данных не получил... Может изменилась разметка сайта?
```
scrapy crawl wb -o artifacts/wb.json -a category_url="https://www.wildberries.ru/catalog/zhenshchinam/odez…
-
This happens on Raspberry Pi OS with Python 3.7.3, after successful installation and installing some dependencies not listed as prerequisites. It happens both when issuing a complete scrapy command, o…