monkeyWie / proxyee

HTTP proxy server,support HTTPS&websocket.MITM impl,intercept and tamper HTTPS traffic.
MIT License
1.52k stars 572 forks source link

[question] How to correlate request and response (with body) #245

Closed grzegorzSowa3 closed 1 year ago

grzegorzSowa3 commented 1 year ago

Hi, Great work you have done here, I appreciate it very much :) I have a question, regarding my use-case. I am trying to get request headers&body and response headers&body in one place, is there any way to do this? I managed to get whole request in beforeRequest and whole response in handleResponse, but I am having trouble correlating them with one another. Tried storing unique value in ThreadLocal<> but it doesn't seem to work, either it is different thread or beforeRequest is not called for every request/response exchange. Do you have any idea how I could achieve that? :)

monkeyWie commented 1 year ago

you can use pipeline on handleResponse, like this:

public void handleResponse(HttpRequest httpRequest, FullHttpResponse httpResponse, HttpProxyInterceptPipeline pipeline) {
    pipeline.getHttpRequest();
    pipeline.getHttpResponse();
}
grzegorzSowa3 commented 1 year ago

But then, how can I get request body? I have FullHttpResponse that contains response body, but HttpRequest doesn't :(

monkeyWie commented 1 year ago

You can refer to this code: https://github.com/monkeyWie/proxyee/blob/master/src/test/java/com/github/monkeywie/proxyee/InterceptFullHttpProxyServer.java

grzegorzSowa3 commented 1 year ago

Tried it, but when I try to get httpRequest body, calling httpRequest.content().toString(Charset.defaultCharset()) in FullHttpResponseInterceptor I get:

io.netty.util.IllegalReferenceCountException: refCnt: 0
    at io.netty.buffer.AbstractByteBuf.ensureAccessible(AbstractByteBuf.java:1454) ~[netty-buffer-4.1.85.Final.jar:4.1.85.Final]
    at io.netty.buffer.AbstractByteBuf.checkIndex(AbstractByteBuf.java:1383) ~[netty-buffer-4.1.85.Final.jar:4.1.85.Final]
    at io.netty.buffer.AbstractByteBuf.checkDstIndex(AbstractByteBuf.java:1409) ~[netty-buffer-4.1.85.Final.jar:4.1.85.Final]
    at io.netty.buffer.CompositeByteBuf.getBytes(CompositeByteBuf.java:1062) ~[netty-buffer-4.1.85.Final.jar:4.1.85.Final]
    at io.netty.buffer.CompositeByteBuf.getBytes(CompositeByteBuf.java:49) ~[netty-buffer-4.1.85.Final.jar:4.1.85.Final]
    at io.netty.buffer.ByteBufUtil.decodeString(ByteBufUtil.java:1270) ~[netty-buffer-4.1.85.Final.jar:4.1.85.Final]
    at io.netty.buffer.AbstractByteBuf.toString(AbstractByteBuf.java:1246) ~[netty-buffer-4.1.85.Final.jar:4.1.85.Final]
    at io.netty.buffer.AbstractByteBuf.toString(AbstractByteBuf.java:1241) ~[netty-buffer-4.1.85.Final.jar:4.1.85.Final]
monkeyWie commented 1 year ago

the code already updated: https://github.com/monkeyWie/proxyee/blob/master/src/test/java/com/github/monkeywie/proxyee/InterceptFullHttpProxyServer.java

grzegorzSowa3 commented 1 year ago

Thanks for the answer! You forgot to pass request headers but the code is generally good, it is working as intended. Cheers!