node need to add 'gzip:true' when there has 'content-encoding: gzip' in response header

[XxLittleCxX]

When the response data has gzip, it should add 'gzip: true' to the options. (Poor English, sorry)

[Baelx]

@XxLittleCxX I'm not one of the maintainers but response headers seem to be outside the scope of this project. All this project does is convert curl to a node-compatible output - using the request library. If you're having trouble handling a gzipped response, you should refer to the request documentation, which is a separate and unaffiliated project. From the request docs:

To accept gzip-compressed responses, set the gzip option to true. Note that the body data passed through request is automatically decompressed while the response object is unmodified and will contain compressed data if the server sent a compressed response.

  const request = require('request')
  request(
    { method: 'GET'
    , uri: 'http://www.google.com'
    , gzip: true
    }
  , function (error, response, body) {
      // body is the decompressed response body
      console.log('server encoded the data as: ' + (response.headers['content-encoding'] || 'identity'))
      console.log('the decoded data is: ' + body)
    }
  )

[b10c77]

When the response data has gzip, it should add 'gzip: true' to the options. (Poor English, sorry)

Agree, I've converted a cURL to node.js that needs to have "gzip: true" on the request options otherwise the response will be all garbled.

[jgroom33]

What would the curl command be in this example?

[jgroom33]

This might be relevant: https://www.garron.me/en/bits/check-gzip-encoding-nginx-apache-with-curl-headers.html Is the goal to set this gzip Boolean when accept encoding=gzip?

[verhovsky]

Looking at the code

https://github.com/request/request/blob/3c0cddc7c8eb60b470e9519da85896ed7ee0081e/request.js#L1014

, it seems that request used to never decode compressed data, you would just get it as-is. Then they added it but put it behind a boolean option, so gzip: true does two things, it adds an accept-encoding header to the request if it doesn't have one already and it enables decoding content when it's gzip or deflate.

I think the least we need to do here is to add gzip: true when the curl command has an accept-encoding header that contains "gzip" or "deflate". We could also add it regardless of what accept-encoding is set to, as long as its set, maybe there are other values that are like "gzip" and cause the server to send gzip'd data that I don't know about. We could also detect when accept-encoding is set to exactly "gzip, deflate" and completely remove that header and have gzip: true set it for us but I'm not going to do that.