search

scrapy-plugins / scrapy-zyte-smartproxy

Zyte Smart Proxy Manager (formerly Crawlera) middleware for Scrapy
BSD 3-Clause "New" or "Revised" License
356 stars 88 forks source link

Custom geo location #38

Closed vimagick closed 7 years ago

vimagick commented 7 years ago

Some websites return different contents based on geo location. Is it possible to use country codes as parameter in headers? It tells crawlera which proxies to choose from shared pool.

X-Crawlera-Country: US,UK,CA
eLRuLL commented 7 years ago

Hi @vimagick you'll have to ask this on crawlera support.

As I understand you can ask them to give you several crawlera users separated by region but only with non-basic plans.

If crawlera adds this as configurable settings, we'll include it ASAP on this middleware.