scrapinghub / portia

Visual scraping for Scrapy
BSD 3-Clause "New" or "Revised" License
9.22k stars 1.41k forks source link

SPLASH_URL ignored by spider? #894

Open ronniehicks opened 4 years ago

ronniehicks commented 4 years ago

The documentation is pretty sparse on where exactly you're supposed to put the SPLASH_URL property in your environment - I'd prefer to have it global than needing to modify each spider/settings.py file if that's the preferred method.

However, when I put my setting into my <project>\spiders\settings.py file (right after the import os statement, it doesn't seem to pick up the setting and process things through Splash? Am I doing something wrong? By the way, I'm running both splash and portia in docker containers side-by-side.