Closed metalwarrior665 closed 2 years ago
cc @szmarczak
require('axios')('https://www.tripplite.com/products/display-mount-accessories-dual-bar-mounts~56-203', { decompress: true, headers: { 'accept-encoding': 'deflate' } }).then(x => x.headers).then(console.log)
I tried axios and the same happens. Not sure how browsers decode this.
Ok got it, it's missing zlib headers. inflateRaw
works like a charm, but this (99%) won't be supported by Got. I guess we could retry with decompress: false
?
Can we somehow integrate this retry into got-scraping
? Those decompression errors are not frequent, but common. E.g. https://github.com/apify/apify-js/issues/373 https://github.com/apify/apify-js/issues/266
Should be addressed in latest got-scraping via https://github.com/apify/got-scraping/pull/64
Describe the bug This website throws
RequestError: unknown compression method
every time. Not sure if this should be fixed in libraries or it is just an edge case. It works with Puppeteer.To Reproduce
Expected behavior A clear and concise description of what you expected to happen.
System information: {"apifyVersion":"2.1.0","apifyClientVersion":"2.0.2","osType":"Linux","nodeVersion":"v16.13.0"}
Additional context Sample run - https://console.apify.com/view/runs/le9zcQFmeteY00dnR