Closed yzqzss closed 1 year ago
@milahu try this. :)
It really doesn't seem like a good idea to me to just refuse to download compressed images entirely. If the user doesn't ask for bypassing, it should at most warn.
oops, sorry, in too much of a hurry here.
If the user doesn't ask for bypassing, it should at most warn.
Yeap, if dumpgenerator detects the cf-polished
HTTP header, it will raise AssertionError
to warn the user to use --bypass-cdn-image-compression
. (https://github.com/mediawiki-client-tools/mediawiki-scraper/pull/163/commits/cc62f57cb660d541440be1ded03e1138582cac57#diff-8cb0e49bf7f47158ff3db3113db064e57d908a7351079af7606e441f85f874c3R67)
It really doesn't seem like a good idea to me to just refuse to download compressed images entirely.
So, let's finish the TODO, add a --nocheck-image-size
option?
Add
--bypass-cdn-image-compression
option to bypass CDN image compression. (CloudFlare Polish, etc.)By adding random params to image URLs.
fix: #155 reported by @milahu