scrapy-plugins / scrapy-splash

Scrapy+Splash for JavaScript integration
BSD 3-Clause "New" or "Revised" License
3.15k stars 451 forks source link

Format of request.cookies #56

Closed lopuhin closed 8 years ago

lopuhin commented 8 years ago

Say we want to add some cookies we got elsewhere to request: we set request.cookies. But the format is HAR, which is not a native python format. I think it would be convenient to allow setting cookies in the format of a list of http.cookiejar.Cookie.__dict__ or http.cookiejar.Cookie to avoid workarounds like this https://github.com/TeamHG-Memex/undercrawler/commit/27d87f22263f6b21a4f44f1feba8e4d1ac8a7ac3. Do you think it's worth adding, and if yes, what format should scrapy-splash support for request.cookies?

kmike commented 8 years ago

'cookies' is a standard scrapy.Request argument (see http://doc.scrapy.org/en/latest/topics/request-response.html):

request_with_cookies = Request(url="http://www.example.com",
                               cookies=[{'name': 'currency',
                                        'value': 'USD',
                                        'domain': 'example.com',
                                        'path': '/currency'}])

The format is not HAR, but it is very close; all documented attributes match HAR attributes, so I just made it HAR in scrapy-splash.

lopuhin commented 8 years ago

Thanks for the clarification! I'll send just the documented values then, that's perfect.