villadora / express-http-proxy

Proxy middleware for express/connect
Other
1.22k stars 235 forks source link

option "intercept" throw error when use a truncated gzip request. #177

Open JanzenZhangChen opened 7 years ago

JanzenZhangChen commented 7 years ago

When using "intercept" option, the http request is using gzip,but the data is truncated.So the proxy will throw an error of "Error: unexpected end of file at Zlib._handle.onerror (zlib.js:370:17) at Gunzip.Zlib._processChunk (zlib.js:538:30) at zlibBufferSync (zlib.js:239:17)" we can use a official method to solve this problem. https://nodejs.org/api/zlib.html#zlib_compressing_http_requests_and_responses (search truncated)

so I add some code in your lib.

function zipOrUnzip(method) { return function(rspData, res) { return (isResGzipped(res)) ? zlib[method](rspData, { finishFlush: zlib.Z_SYNC_FLUSH //here }) : rspData; }; }

then the intercept option would be like: intercept: function(rsp, data, req, res, callback) { var resdata = data.toString(); //data would be Buffer callback(null, resdata); },

monkpow commented 7 years ago

@JanzenZhangChen Thank you for this report and suggestion. Do you have a test case that's failing that I can use to verify your fix? Thanks!

JanzenZhangChen commented 7 years ago

sorry, i cant provide our cgi to you,but I found one in cn.bing.com(china)

http://cn.bing.com/HPImageArchive.aspx?format=js&idx=0&n=1&nc=1490973090201&pid=hp&video=1

if you use fiddler to catch this request, in Raw section of response body, it said: FIDDLER: RawDisplay truncated at 128 characters. Right-click to disable truncation.

phenomnomnominal commented 7 years ago

The fix suggested by @JanzenZhangChen fixed some issues I was getting too. Do you need a PR?

monkpow commented 7 years ago

I found an separate bug in the parser for zip that I think may have been causing this. The patch is in the 1.0.* line.

If you're still experiencing the issue, I'd be very happy to get a PR if you would like to accelerate resolution.

JanzenZhangChen commented 7 years ago

I created a PR to the project. I couldnt pass the "on the nose" mocha test. And I have no idea how to fix this problem.

jpetitcolas commented 7 years ago

Note we have the issue when receiving a 204 (No-content) response on a DELETE request. We circumvent the issue using a code like:

if (srcReq.method === 'DELETE') {
    delete proxyReq.headers['accept-encoding'];
}
scr4bble commented 6 years ago

Hi guys, any update on this ? I am experiencing the same issue.

zlib.js:421
      throw error;
      ^
Error: unexpected end of file
    at Gunzip.zlibOnError (zlib.js:153:15)
    at Gunzip._processChunk (zlib.js:411:30)
    at zlibBufferSync (zlib.js:144:38)
    at Object.gunzipSync (zlib.js:590:14)
    at /usr/src/app/node_modules/express-http-proxy/index.js:323:46
    at IncomingMessage.<anonymous> (/usr/src/app/node_modules/express-http-proxy/index.js:100:23)
    at emitNone (events.js:111:20)
    at IncomingMessage.emit (events.js:208:7)
    at endReadableNT (_stream_readable.js:1064:12)
    at _combinedTickCallback (internal/process/next_tick.js:138:11)