scrapy gunzip inclusion?

scrapy / w3lib

Python library of web-related functions

BSD 3-Clause "New" or "Revised" License

392 stars 104 forks source link

scrapy gunzip inclusion? #43

Open nyov opened 8 years ago

nyov commented 8 years ago

Would scrapy's gunzip and is_gzipped functions (scrapy/utils/gz.py) fit in w3lib? If is_gzipped would take headers instead of response as argument, there is no other dependency.

Gallaecio commented 5 years ago

I’ve just proposed to deprecate is_gzipped in Scrapy: https://github.com/scrapy/scrapy/pull/3718

Maybe we could implement Scrapy’s gzip_magic_number as is_gzipped here?

As for gunzip, it may make sense to move it to w3lib. I think it would become the easiest answer for https://stackoverflow.com/q/1732709/939364

@redapple, @dangra, @kmike Thoughts?