Redirects not handled in the media pipeline

TeamHG-Memex / scrapy-cdr

Item definition and utils for storing items in CDR format for scrapy

MIT License

7 stars 6 forks source link

Redirects not handled in the media pipeline #12

Closed lopuhin closed 7 years ago

lopuhin commented 7 years ago

2017-06-14 16:52:07 [scrapy.pipelines.files] WARNING: File (code: 302): Error downloading file from <GET http://i82.fastpic.ru/big/2016/0911/ab/<snip>.jpg> referred in <None>

This should have been fixed in https://github.com/scrapy/scrapy/issues/2004, but I'm using scrapy 1.4 and still get this issue - perhaps it requires some changes in scrapy-cdr as well, or it's something else

lopuhin commented 7 years ago

Ah, one needs to allow them! MEDIA_ALLOW_REDIRECTS = True