Closed IaroslavR closed 4 years ago
Your example is using Scrapy-Splash without configuring it first. Please, check the README.
Adding this to the spider makes it work:
class TestSpider(scrapy.Spider):
# …
custom_settings = {
'SPLASH_URL': 'http://localhost:8050',
'DOWNLOADER_MIDDLEWARES': {
'scrapy_splash.SplashCookiesMiddleware': 723,
'scrapy_splash.SplashMiddleware': 725,
'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware': 810,
},
'SPIDER_MIDDLEWARES': {
'scrapy_splash.SplashDeduplicateArgsMiddleware': 100,
},
'DUPEFILTER_CLASS': 'scrapy_splash.SplashAwareDupeFilter',
}
# …
Python 3.6.2, Scrapy 1.4.0, scrapy_splash 0.7.2, splash from
docker pull scrapinghub/splash
Spider:
with scrapy.Request all ok, but in splash.png I see garbage instead of the Google page screenshot.