search

alibaba / anyproxy

A fully configurable http/https proxy in NodeJS
http://anyproxy.io
Apache License 2.0
7.76k stars 1.21k forks source link

当 response 的 Content-Encoding 是 gzip，同时 body 为空时程序崩溃 #305

Open hokor opened 6 years ago

hokor commented 6 years ago

Plese fill the template when you reporting a new issue, thanks!

Which platform are you running AnyProxy

Mac

The version of the AnyProxy

4.0.2

Your expected behavior of AnyProxy

无论什么请求，程序都正常运行

The actual behavior of AnyProxy

程序崩溃。

The log of the error

test request：

curl -x 127.0.0.1:8001 'https://beacon.tingyun.com/xhr1?pvid=e90422ff-80d8-470e-8de0-407d3f6bdbec&ref=https%3A%2F%2Ftest.example.com%2F&referrer=&key=vK4oZhra2XA&v=1.7.3&av=1.7.3&did=b51f4119-bd75-4fac-8050-e512ac7eddab&sid=f095450d-a0a9-4075-9256-55120e8be309&__r=1514212434717' -H 'Pragma: no-cache' -H 'Origin: https://test.example.com' -H 'Accept-Encoding: gzip, deflate, br' -H 'Accept-Language: zh-CN,zh;q=0.9,en-US;q=0.8,en;q=0.7' -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.84 Safari/537.36' -H 'Content-Type: text/plain;charset=UTF-8' -H 'Accept: */*' -H 'Cache-Control: max-age=0' -H 'Referer: https://www.growingio.com/' -H 'Cookie: qtrans_cookie_test=qTranslate+Cookie+Test; Hm_lvt_93daecc2db29efa5d83566cf580e84af=1514212350; NTKF_T2D_CLIENTID=guest5DACA43B-1338-4E4B-FC1C-8E16DC2EE088; nTalk_CACHE_DATA={uid:kf_9212_ISME9754_guest5DACA43B-1338-4E,tid:1514212351021605}; Hm_lpvt_93daecc2db29efa5d83566cf580e84af=1514212352' -H 'Connection: keep-alive' --data-binary '{"xhr":[{"id":0,"req":"POST https://test.example.com","start":1514212433017,"du":32,"cb":0,"status":200,"err":0,"rec":0,"send":386},{"id":1,"req":"POST https://test.example.com/","start":1514212434573,"du":33,"cb":0,"status":200,"err":0,"rec":0,"send":12044}]}' --compressed

Log：

[AnyProxy Log][2017-12-25 22:43:59]: received https CONNECT request beacon.tingyun.com [AnyProxy Log][2017-12-25 22:43:59]: will forward to local https server [AnyProxy Log][2017-12-25 22:43:59]: [internal https]proxy server for beacon.tingyun.com established [AnyProxy Log][2017-12-25 22:43:59]: received request to: POST beacon.tingyun.com/xhr1?pvid=e90422ff-80d8-470e-8de0-407d3f6bdbec&ref=https%3A%2F%2Ftest.example.com%2F&referrer=&key=vK4oZhra2XA&v=1.7.3&av=1.7.3&did=b51f4119-bd75-4fac-8050-e512ac7eddab&sid=f095450d-a0a9-4075-9256-55120e8be309&__r=1514212434717 [AnyProxy ERROR][2017-12-25 22:43:59]: got an uncaught exception, is there anything goes wrong in your rule file ? TypeError: Cannot read property 'length' of null at Gunzip.zlibBufferOnEnd (zlib.js:100:17) at Gunzip.emit (events.js:164:20) at endReadableNT (_stream_readable.js:1054:12) at _combinedTickCallback (internal/process/next_tick.js:138:11) at process._tickCallback (internal/process/next_tick.js:180:9) [AnyProxy Log][2017-12-25 22:43:59]: destorying https connection : 127.0.0.1:63791 [AnyProxy Log][2017-12-25 22:43:59]: endding https cltSocket : 127.0.0.1:63791 [AnyProxy Log][2017-12-25 22:43:59]: clearing cache file... [AnyProxy Log][2017-12-25 22:43:59]: closing webserver...

codingfishman commented 6 years ago

Hi @hokor 我们在4.0.2里面对这个问题进行了过修复，本地我通过4.0.2也无法重现这个报错了。请帮忙确认所运行的版本是否为4.0.2, 可以通过 anyproxy --version 来获得版本号。

gucong3000 commented 6 years ago 
[AnyProxy Log][2018-01-10 14:25:30]: received request to: POST api.advisormedia.cz/v2/partner-domain
E:\work\sms-bomber\node_modules\brotli\build\encode.js:3
1<process.argv.length?process.argv[1].replace(/\\/g,"/"):"unknown-program");b.arguments=process.argv.slice(2);"undefined"!==typeof module&&(module.exports=b);process.on("uncaughtException",function(a){if(!(a instanceof y))throw a;});b.inspect=function(){return"[Emscripten Module object]"}}else if(x)b.print||(b.print=print),"undefined"!=typeof printErr&&(b.printErr=printErr),b.read="undefined"!=typeof read?read:function(){throw"no read() available (jsc?)";},b.readBinary=function(a){if("function"===

             ^

TypeError: Cannot read property 'length' of null
    at Gunzip.zlibBufferOnEnd (zlib.js:100:17)
    at Gunzip.emit (events.js:164:20)
    at endReadableNT (_stream_readable.js:1054:12)
    at _combinedTickCallback (internal/process/next_tick.js:138:11)
    at process._tickCallback (internal/process/next_tick.js:180:9)
$ anyproxy --version
4.0.4

4.0.4下依然能够复现此问题，复现方法为本地安装anyproxy后，Windows10下Chrome浏览器中使用代理访问github首页 不添加根证书时报错时会报错。

codingfishman commented 6 years ago

@gucong3000 AnyProxy在做unzip前会先判断body是否有值，我在osx下也确实没重现。 想确认一下，这个brotli的报错，在被AnyProxy用来做unzip的时候报错对吧？

gucong3000 commented 6 years ago

AnyProxy在做unzip前会先判断body是否有值

要不您给个此逻辑的大概位置，我在Windows下调试看看

想确认一下，这个brotli的报错，在被AnyProxy用来做unzip的时候报错对吧？

不，我并不确定是啥时候报的错，我没有进行过调试。

gucong3000 commented 6 years ago

进一步测试后，我发现了复现此问题的详细步骤

  1. 删除证书
  2. 命令行运行anyproxy --intercept
  3. Chrome浏览器通过代理打开https://github.com，显示证书错误，多点几次刷新
  4. 几秒后，命令行窗口中，anyproxy已经报错退出
E:\work\anyproxy-rule-mirror>anyproxy --intercept
[AnyProxy Log][2018-01-15 11:00:10]: Http proxy started on port 8001
[AnyProxy Log][2018-01-15 11:00:10]: web interface started on port 8002
[AnyProxy Log][2018-01-15 11:00:30]: received https CONNECT request github.com
[AnyProxy Log][2018-01-15 11:00:30]: received https CONNECT request assets-cdn.github.com
[AnyProxy Log][2018-01-15 11:00:30]: received https CONNECT request assets-cdn.github.com
[AnyProxy Log][2018-01-15 11:00:30]: received https CONNECT request avatars0.githubusercontent.com
[AnyProxy Log][2018-01-15 11:00:30]: received https CONNECT request avatars0.githubusercontent.com
[AnyProxy Log][2018-01-15 11:00:30]: received https CONNECT request avatars1.githubusercontent.com
[AnyProxy Log][2018-01-15 11:00:30]: received https CONNECT request avatars1.githubusercontent.com
[AnyProxy Log][2018-01-15 11:00:30]: received https CONNECT request avatars1.githubusercontent.com
[AnyProxy Log][2018-01-15 11:00:30]: received https CONNECT request avatars1.githubusercontent.com
[AnyProxy Log][2018-01-15 11:00:30]: received https CONNECT request avatars2.githubusercontent.com
[AnyProxy Log][2018-01-15 11:00:30]: received https CONNECT request avatars2.githubusercontent.com
[AnyProxy Log][2018-01-15 11:00:30]: received https CONNECT request avatars2.githubusercontent.com
[AnyProxy Log][2018-01-15 11:00:30]: received https CONNECT request avatars3.githubusercontent.com
[AnyProxy Log][2018-01-15 11:00:30]: received https CONNECT request avatars3.githubusercontent.com
[AnyProxy Log][2018-01-15 11:00:30]: received https CONNECT request avatars3.githubusercontent.com
[AnyProxy Log][2018-01-15 11:00:30]: received https CONNECT request avatars3.githubusercontent.com
[AnyProxy Log][2018-01-15 11:00:30]: received https CONNECT request avatars3.githubusercontent.com
[AnyProxy Log][2018-01-15 11:00:30]: received https CONNECT request avatars3.githubusercontent.com
[AnyProxy Log][2018-01-15 11:00:30]: will forward to local https server
[AnyProxy Log][2018-01-15 11:00:30]: will forward to local https server
[AnyProxy Log][2018-01-15 11:00:30]: will forward to local https server
[AnyProxy Log][2018-01-15 11:00:30]: will forward to local https server
[AnyProxy Log][2018-01-15 11:00:30]: will forward to local https server
[AnyProxy Log][2018-01-15 11:00:30]: will forward to local https server
[AnyProxy Log][2018-01-15 11:00:30]: will forward to local https server
[AnyProxy Log][2018-01-15 11:00:30]: will forward to local https server
[AnyProxy Log][2018-01-15 11:00:30]: will forward to local https server
[AnyProxy Log][2018-01-15 11:00:30]: will forward to local https server
[AnyProxy Log][2018-01-15 11:00:30]: will forward to local https server
[AnyProxy Log][2018-01-15 11:00:30]: will forward to local https server
[AnyProxy Log][2018-01-15 11:00:30]: will forward to local https server
[AnyProxy Log][2018-01-15 11:00:30]: will forward to local https server
[AnyProxy Log][2018-01-15 11:00:30]: will forward to local https server
[AnyProxy Log][2018-01-15 11:00:30]: will forward to local https server
[AnyProxy Log][2018-01-15 11:00:30]: will forward to local https server
[AnyProxy Log][2018-01-15 11:00:30]: will forward to local https server
[AnyProxy Log][2018-01-15 11:00:30]: [internal https]proxy server for assets-cdn.github.com established
[AnyProxy Log][2018-01-15 11:00:30]: [internal https]proxy server for assets-cdn.github.com established
[AnyProxy Log][2018-01-15 11:00:30]: [internal https]proxy server for avatars0.githubusercontent.com established
[AnyProxy Log][2018-01-15 11:00:30]: [internal https]proxy server for avatars0.githubusercontent.com established
[AnyProxy Log][2018-01-15 11:00:30]: [internal https]proxy server for avatars1.githubusercontent.com established
[AnyProxy Log][2018-01-15 11:00:30]: [internal https]proxy server for avatars1.githubusercontent.com established
[AnyProxy Log][2018-01-15 11:00:30]: [internal https]proxy server for avatars1.githubusercontent.com established
[AnyProxy Log][2018-01-15 11:00:30]: [internal https]proxy server for avatars3.githubusercontent.com established
[AnyProxy Log][2018-01-15 11:00:30]: [internal https]proxy server for avatars3.githubusercontent.com established
[AnyProxy Log][2018-01-15 11:00:30]: [internal https]proxy server for avatars3.githubusercontent.com established
[AnyProxy Log][2018-01-15 11:00:30]: [internal https]proxy server for avatars3.githubusercontent.com established
[AnyProxy Log][2018-01-15 11:00:30]: [internal https]proxy server for github.com established
[AnyProxy Log][2018-01-15 11:00:30]: [internal https]proxy server for avatars1.githubusercontent.com established
[AnyProxy Log][2018-01-15 11:00:30]: [internal https]proxy server for avatars2.githubusercontent.com established
[AnyProxy Log][2018-01-15 11:00:30]: [internal https]proxy server for avatars2.githubusercontent.com established
[AnyProxy Log][2018-01-15 11:00:30]: [internal https]proxy server for avatars2.githubusercontent.com established
[AnyProxy Log][2018-01-15 11:00:30]: [internal https]proxy server for avatars3.githubusercontent.com established
[AnyProxy Log][2018-01-15 11:00:30]: [internal https]proxy server for avatars3.githubusercontent.com established
[AnyProxy Log][2018-01-15 11:00:39]: received https CONNECT request github.com
[AnyProxy Log][2018-01-15 11:00:39]: will forward to local https server
[AnyProxy Log][2018-01-15 11:00:39]: [internal https]proxy server for github.com established
[AnyProxy Log][2018-01-15 11:00:39]: received https CONNECT request github.com
[AnyProxy Log][2018-01-15 11:00:39]: will forward to local https server
[AnyProxy Log][2018-01-15 11:00:39]: [internal https]proxy server for github.com established
[AnyProxy Log][2018-01-15 11:00:39]: received https CONNECT request assets-cdn.github.com
[AnyProxy Log][2018-01-15 11:00:39]: received https CONNECT request assets-cdn.github.com
[AnyProxy Log][2018-01-15 11:00:39]: received https CONNECT request avatars0.githubusercontent.com
[AnyProxy Log][2018-01-15 11:00:39]: received https CONNECT request avatars0.githubusercontent.com
[AnyProxy Log][2018-01-15 11:00:39]: received https CONNECT request avatars1.githubusercontent.com
[AnyProxy Log][2018-01-15 11:00:39]: received https CONNECT request avatars1.githubusercontent.com
[AnyProxy Log][2018-01-15 11:00:39]: received https CONNECT request avatars1.githubusercontent.com
[AnyProxy Log][2018-01-15 11:00:39]: received https CONNECT request avatars2.githubusercontent.com
[AnyProxy Log][2018-01-15 11:00:39]: received https CONNECT request avatars2.githubusercontent.com
[AnyProxy Log][2018-01-15 11:00:39]: received https CONNECT request avatars3.githubusercontent.com
[AnyProxy Log][2018-01-15 11:00:39]: received https CONNECT request avatars3.githubusercontent.com
[AnyProxy Log][2018-01-15 11:00:39]: received https CONNECT request avatars3.githubusercontent.com
[AnyProxy Log][2018-01-15 11:00:39]: received https CONNECT request avatars3.githubusercontent.com
[AnyProxy Log][2018-01-15 11:00:39]: will forward to local https server
[AnyProxy Log][2018-01-15 11:00:39]: will forward to local https server
[AnyProxy Log][2018-01-15 11:00:39]: will forward to local https server
[AnyProxy Log][2018-01-15 11:00:39]: will forward to local https server
[AnyProxy Log][2018-01-15 11:00:39]: will forward to local https server
[AnyProxy Log][2018-01-15 11:00:39]: will forward to local https server
[AnyProxy Log][2018-01-15 11:00:39]: will forward to local https server
[AnyProxy Log][2018-01-15 11:00:39]: received https CONNECT request github.com
[AnyProxy Log][2018-01-15 11:00:39]: will forward to local https server
[AnyProxy Log][2018-01-15 11:00:39]: will forward to local https server
[AnyProxy Log][2018-01-15 11:00:39]: will forward to local https server
[AnyProxy Log][2018-01-15 11:00:39]: will forward to local https server
[AnyProxy Log][2018-01-15 11:00:39]: will forward to local https server
[AnyProxy Log][2018-01-15 11:00:39]: will forward to local https server
[AnyProxy Log][2018-01-15 11:00:39]: will forward to local https server
[AnyProxy Log][2018-01-15 11:00:39]: received request to: POST api.advisormedia.cz/v2/partner-domain
[AnyProxy Log][2018-01-15 11:00:39]: [internal https]proxy server for avatars1.githubusercontent.com established
[AnyProxy Log][2018-01-15 11:00:39]: [internal https]proxy server for avatars1.githubusercontent.com established
[AnyProxy Log][2018-01-15 11:00:39]: [internal https]proxy server for avatars1.githubusercontent.com established
[AnyProxy Log][2018-01-15 11:00:39]: [internal https]proxy server for avatars2.githubusercontent.com established
[AnyProxy Log][2018-01-15 11:00:39]: [internal https]proxy server for avatars3.githubusercontent.com established
[AnyProxy Log][2018-01-15 11:00:39]: [internal https]proxy server for avatars3.githubusercontent.com established
[AnyProxy Log][2018-01-15 11:00:39]: [internal https]proxy server for avatars3.githubusercontent.com established
[AnyProxy Log][2018-01-15 11:00:39]: [internal https]proxy server for assets-cdn.github.com established
[AnyProxy Log][2018-01-15 11:00:39]: [internal https]proxy server for assets-cdn.github.com established
[AnyProxy Log][2018-01-15 11:00:39]: [internal https]proxy server for avatars0.githubusercontent.com established
[AnyProxy Log][2018-01-15 11:00:39]: [internal https]proxy server for avatars0.githubusercontent.com established
[AnyProxy Log][2018-01-15 11:00:39]: [internal https]proxy server for avatars2.githubusercontent.com established
[AnyProxy Log][2018-01-15 11:00:39]: [internal https]proxy server for avatars3.githubusercontent.com established
[AnyProxy Log][2018-01-15 11:00:39]: [internal https]proxy server for github.com established
[AnyProxy ERROR][2018-01-15 11:00:40]: got an uncaught exception, is there anything goes wrong in your rule file ?
TypeError: Cannot read property 'length' of null
    at Gunzip.zlibBufferOnEnd (zlib.js:100:17)
    at Gunzip.emit (events.js:164:20)
    at endReadableNT (_stream_readable.js:1054:12)
    at _combinedTickCallback (internal/process/next_tick.js:138:11)
    at process._tickCallback (internal/process/next_tick.js:180:9)
[AnyProxy Log][2018-01-15 11:00:40]: destorying https connection : 127.0.0.1:55083
[AnyProxy Log][2018-01-15 11:00:40]: endding https cltSocket : 127.0.0.1:55083
[AnyProxy Log][2018-01-15 11:00:40]: clearing cache file...
[AnyProxy Log][2018-01-15 11:00:40]: closing webserver...
codingfishman commented 6 years ago

更新： 在Mac上关闭证书信任，并多次刷新github主页未能重现错误。

@gucong3000 可否提供你的nodejs版本？也许这个node版本可以重现。

我怀疑出错在nodejs的这一段, 按理说end 事件不会在data事件前抛出，最好能直接debug。

hokor commented 6 years ago

4.0.6 版本还是复现这个问题。

anyproxy -V
 4.0.6

anyproxy -i
[AnyProxy Log][2018-03-08 11:54:42]: Http proxy started on port 8001
[AnyProxy Log][2018-03-08 11:54:42]: web interface started on port 8002
[AnyProxy Log][2018-03-08 11:54:45]: received https CONNECT request beacon.tingyun.com
[AnyProxy Log][2018-03-08 11:54:45]: will forward to local https server
[AnyProxy Log][2018-03-08 11:54:45]: [internal https]proxy server for beacon.tingyun.com established
[AnyProxy Log][2018-03-08 11:54:45]: received request to: POST beacon.tingyun.com/xhr1?pvid=e90422ff-80d8-470e-8de0-407d3f6bdbec&ref=https%3A%2F%2Ftest.example.com%2F&referrer=&key=vK4oZhra2XA&v=1.7.3&av=1.7.3&did=b51f4119-bd75-4fac-8050-e512ac7eddab&sid=f095450d-a0a9-4075-9256-55120e8be309&__r=1514212434717
[AnyProxy ERROR][2018-03-08 11:54:45]: got an uncaught exception, is there anything goes wrong in your rule file ?
TypeError: Cannot read property 'length' of null
    at Gunzip.zlibBufferOnEnd (zlib.js:100:17)
    at Gunzip.emit (events.js:164:20)
    at endReadableNT (_stream_readable.js:1054:12)
    at _combinedTickCallback (internal/process/next_tick.js:138:11)
    at process._tickCallback (internal/process/next_tick.js:180:9)
[AnyProxy Log][2018-03-08 11:54:45]: destorying https connection : 127.0.0.1:56691
[AnyProxy Log][2018-03-08 11:54:45]: endding https cltSocket : 127.0.0.1:56691
[AnyProxy Log][2018-03-08 11:54:45]: clearing cache file...
[AnyProxy Log][2018-03-08 11:54:45]: closing webserver...
codingfishman commented 6 years ago

@hokor 可否提供node的版本？

CirnoV commented 6 years ago

This issue has been resolved in v9.3.0

https://github.com/nodejs/node/pull/17042