calgo-lab / green-db

The monorepo that powers the GreenDB.

https://calgo-lab.github.io/green-db/

22 stars 2 forks source link

set Accept-Language to all #54

Closed en-GB closed 2 years ago

en-GB commented 2 years ago

Accept-Language was set to german. this may cause non german sites to redirect to a german version. setting Accept-Language to * should prevent this.

51

BigDatalex commented 2 years ago

When using * we have to make sure that the german otto and zalando are still working, because I think they rely on the de setting. In case of asos this won't be a problem, because I used spider-specific settings via custom_settings see:

https://github.com/calgo-lab/green-db/blob/e3c89d6e2453472759edc371bbbc359fa2503025/scraping/scraping/spiders/asos.py#L37-L40

So maybe you can use the same approach in your spider for specifying a different Accept-Language header?

se-jaeger commented 2 years ago

Since this is actually documented here: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Accept-Language It should work. However, we should test it ;) Does some one of you have capacity to test it? I'm currently testing zalando_fr

se-jaeger commented 2 years ago

Since this is actually documented here: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Accept-Language It should work. However, we should test it ;) Does some one of you have capacity to test it? I'm currently testing zalando_fr

Just started the test.

se-jaeger commented 2 years ago

Hi, it works! - Since this is also integrate in #58 I close this. Thanks for discussing this!