search

andybalholm / redwood

Web content filter that runs as an HTTP proxy
BSD 2-Clause "Simplified" License
221 stars 37 forks source link

A weird issue with the site: https://myip.net.il/ #78

Open elico opened 3 years ago

elico commented 3 years ago

I own the site: https://myip.net.il/

It's a simple httpd (apache) server and it seems to work fine on http via redwood but not on intercepted traffic. I am using the latest version of Redwood built today from sources. I have tested the issue and for now what I got to the point is that I have from the browser a har file: myip.net.il_Archive [21-07-27 20-56-16].har

From the server side I see a full response being sent to the client but for some reason RedWood think that the response is malformed. To see the issue just try to access the site URL at: https://myip.net.il/

behind Redwood with ssl-bump interception.

andybalholm commented 3 years ago

For me, the main part of the page works fine, but the image fails. However, if I fetch the image in a new tab, it works. It is an issue with reusing the connection. Probably if I would add malformed HTTP response "" to the list of errors that trigger a "redial" that would take care of it.

andybalholm commented 3 years ago

But you should try to figure out why your server has the issues with reusing the connection.

elico commented 3 years ago

It the most simple apache server... I will try to verify the issue next week. Thanks.

בתאריך יום ד׳, 28 ביולי 2021, 0:58, מאת Andy Balholm ‏< @.***>:

But you should try to figure out why your server has the issues with reusing the connection.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/andybalholm/redwood/issues/78#issuecomment-887861551, or unsubscribe https://github.com/notifications/unsubscribe-auth/AANUHMECWKJ6OFWHSJJEOOTTZ4TYNANCNFSM5BC3J7GA .

elico commented 3 years ago

@andybalholm I have verified what was the issue and I believe this should be fixed some how. The basic issue was that the Content-Length header was off by 1 byte '\n'. Due to this Redwood was reading only the Content-Length and was leaving 1 byte at the buff(er. Then when trying to read the next response it was actually reading the last byte from the last response and it was considered the "status line" for the new request..

I don't know if it's a golang or Redwood specific issue.

elico commented 3 years ago

@andybalholm The main question is: Is this issue specific to Redwood or is it a GoLang http library bug?

I have also tested this issue with couple other proxy services, both commercial and open source. All of the commercial products handled this issue pretty easy. Squid and couple proxies(both reverse and forward) handled this the same way without any other issue. Any browser accessing directly the service receives the html and the image in the same connection without any errors at all.

andybalholm commented 3 years ago

There is no bug anywhere but your server. Apparently that bug is common enough that most browsers and proxies have added workarounds, although I don't think we've run into it before.

We could add a workaround at https://github.com/andybalholm/redwood/blob/2efa4ab7d658fb3b767bd955b8f3d1b0172b2e21/transport.go#L172, checking for this error and calling ReadResponse again.

elico commented 2 years ago

@andybalholm Did we got into a conclusion what to do with this error?

andybalholm commented 2 years ago

I think fixing your server to send the correct Content-Length was the right thing to do.