-
"C:\Program Files\Python310\python.exe" E:/www/yd/python/Real-time-visualization-of-national-news-main/xinhua/test.py
Traceback (most recent call last):
File "E:\www\yd\python\Real-time-visualizat…
-
Is this the intended behaviour of `LinkExtractor`? I seem to not be able to extract relative URLs when using it. Alternatively, if I use a selector for `a` elements, I can capture everything.
For r…
-
There is a UI in https://github.com/TeamHG-Memex/arachnado (demo: https://www.youtube.com/watch?v=JPyvmW-eOLs); what about adding something similar to Scrapy itself, maybe as an extension in a separat…
kmike updated
8 years ago
-
Hi,
I run aduana with the version 0.2.1 in PyPI and everything was fine. But just after cloning the master branch I started to get the following error:
```
2015-11-12 17:30:51 [scrapy] INFO : Spider…
-
getting error:
Error(kindlegen):E30005: Could not find file *.epub
-
Hi,
I do not understand how to set `meta` parameters in a frontier Request generated from a seeder.
It seems that there are two kinds of meta parameters: frontier ones and scrapy ones. I would like to…
-
I currently use single thread scraper to crawl google.com. But I have tons of search terms. scrapy-selenium only open 1 browser, so I could only search one term at one time. Should I use remote browse…
-
I profiled a simple Scrapy spider which just downloads pages and follows links extracted using LinkExtractor; it turns out one of the main bottlenecks is urlparse module and our related functions like…
kmike updated
8 months ago
-
Hello,
I am running the dockerized portiacrawl and keep running into this UnicodeDecodeError. I saw this issue reported previously as well, with a fix expected when portia was ported over to python…
-
hi kmike, i use scrapy-splash and meet a issue, when i first run 'scrapy crawl toutiao', it's run right, bug when i run it's second, it occur a issue.
i find the issue because headers i add, when i…