HttpProxyIntercept beforeRequest modify request body failed when origin body length bigger than modified body length

[T0DDY]

new HttpProxyIntercept() { // handle bodys @Override public void beforeRequest(Channel clientChannel, HttpContent httpContent, HttpProxyInterceptPipeline pipeline) throws Exception { String body = httpContent.content().toString(Charset.forName("UTF-8")); String uri = pipeline.getHttpRequest().uri(); HttpHeaders headers = pipeline.getHttpRequest().headers(); System.out.println(body.length()); // out: 10;假设这里是10； body="1234567890"+"ab"; // this change the body length 12;这里body length 变成了12. httpContent.content().clear(); httpContent.content().writeBytes(body.getBytes(Charset.forName("UTF-8"))); // issue here；这里就有问题了。 super.beforeRequest(clientChannel, httpContent, pipeline); }

猜测httpContent.content()里面用的是fixed buffer length，所以变长的byte 无法全部塞进去。 I'm wondering that the httpContent.content() leverage a fixed buffer length, so all the longer bytes could not be filled in.

[T0DDY]

Please @monkeyWie take a look. 麻烦 @monkeyWie 看下。还是我用法不对。

[T0DDY]

上面代码是POC，在具体使用的时候http header 里面的 content-length 重新设置过了。

[monkeyWie]

@T0DDY You should be use FullRequestIntercept or FullResponseIntercept.

[T0DDY]

OK, thanks a lot. 多谢多谢。@monkeyWie

[T0DDY]

Version: 1.2.2 After change to : new FullRequestIntercept() { @Override public boolean match(HttpRequest httpRequest, HttpProxyInterceptPipeline httpProxyInterceptPipeline) { return true; }

                @Override
                public void handleRequest(FullHttpRequest httpRequest, HttpProxyInterceptPipeline pipeline){
                    ByteBuf httpContent = httpRequest.content();
                    System.out.println(httpRequest.toString());

                    String body = httpContent.toString(Charset.forName("UTF-8"));
                    String uri = pipeline.getHttpRequest().uri();
                    HttpHeaders headers = pipeline.getHttpRequest().headers();

                    if (uri.contains("testfull")) {
                        System.out.println("Before body:"+body);
                        body = body+"append";
                        System.out.println("After body: " + body);
                        httpContent.clear();
                        httpContent.writeBytes(body.getBytes());
                    }
                }

Some exceptions throws: java.util.NoSuchElementException: decompress at io.netty.channel.DefaultChannelPipeline.getContextOrDie(DefaultChannelPipeline.java:1073) at io.netty.channel.DefaultChannelPipeline.remove(DefaultChannelPipeline.java:423) at com.github.monkeywie.proxyee.intercept.common.FullRequestIntercept.afterResponse(FullRequestIntercept.java:54) at com.github.monkeywie.proxyee.intercept.HttpProxyInterceptPipeline.afterResponse(HttpProxyInterceptPipeline.java:87) at com.github.monkeywie.proxyee.handler.HttpProxyClientHandle.channelRead(HttpProxyClientHandle.java:30) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:251) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650) at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576) at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)

Is that because I have used the method of "GET" and the content length is zero, so there is no decompress to attach? 我换了FullRequestIntercept 就抛了上面的exception，是因为GET 方法要特殊处理，因为content - length 为零吗？ @monkeyWie thx ahead

[monkeyWie]

@T0DDY 我这边测试get请求正常：

curl -k -x 127.0.0.1:9999 https://www.baidu.com

能提供下复现步骤吗

[T0DDY]

request: HttpObjectAggregator$AggregatedFullHttpRequest(decodeResult: success, version: HTTP/1.1, content: CompositeByteBuf(ridx: 0, widx: 0, cap: 0, components=0)) GET /testfull?test=test HTTP/1.1 Host: www.baidu.com User-Agent: curl/7.64.1 Accept: / content-length: 0 Before body: After body: append

Command: curl -k -x "http://127.0.0.1:9999" "https://www.baidu.com/testfull?test=test" @monkeyWie

[monkeyWie]

@T0DDY 没有抛异常呀

[T0DDY]

.httpProxyExceptionHandle(new HttpProxyExceptionHandle() { @Override public void beforeCatch(Channel clientChannel, Throwable cause) { cause.printStackTrace(); System.out.println("Before catch exception"); }

        @Override
        public void afterCatch(Channel clientChannel, Channel proxyChannel, Throwable cause){
            cause.printStackTrace();
            System.out.println("After catch exception");
        }
    })

The stack trace printed in this section. And my code looks like:

Anything wired?

[T0DDY]

.httpProxyExceptionHandle(new HttpProxyExceptionHandle() { @override public void beforeCatch(Channel clientChannel, Throwable cause) { cause.printStackTrace(); System.out.println("Before catch exception"); }

        @Override
        public void afterCatch(Channel clientChannel, Channel proxyChannel, Throwable cause){
            cause.printStackTrace();
          System.out.println("After catch exception");
        }
    })

The stack trace printed in this section. And my code looks like:

Anything wired?

It may help that: I have implemented a local http server for testing purpose. The response from this http server contains no headers like compress (deflate / gzip / or any other fields.)

[monkeyWie]

This is a bug that integration FullRequestIntercept and FullResponseIntercept, it will be fixed in v1.2.3.

[T0DDY]

This is a bug that integration FullRequestIntercept and FullResponseIntercept, it will be fixed in v1.2.3.

Is there any work around for now?

[monkeyWie]

I have published the jar to maven center, you can upgrade proxyee version late.

[T0DDY]

Nice of U, close this issue.