justcoding121 / titanium-web-proxy

A cross-platform asynchronous HTTP(S) proxy server in C#.
MIT License
1.92k stars 598 forks source link

System.Net.WebException: The underlying connection was closed: The connection was closed unexpectedly #857

Open PCAssistSoftware opened 3 years ago

PCAssistSoftware commented 3 years ago

Hi

Hoping someone can help me.

I have an app which uses Postmark (https://postmarkapp.com/) to send email via an API, it was giving me error "System.Net.WebException: The remote server returned an error: (403) Forbidden" when running my app as admin, so I wanted to use Titanium to diagnose further.

Unfortunately when I have Titanium running (am using WPF example app for now) I get a different error “System.Net.WebException: The underlying connection was closed: The connection was closed unexpectedly” which I think Titanium is causing so it never gets to the above error I am trying to find cause of.

As soon as I close the Titanium app I see my 403 error again, so how can I run Titanium without getting “System.Net.WebException: The underlying connection was closed: The connection was closed unexpectedly”

If I use an app such as Fiddler or Wireshark then I don’t have same issue and can then see cause, but I prefer to support and use this project, so would like to know why it happens and how to get round it

Hope this make sense

Thanks

PCAssistSoftware commented 3 years ago

@honfika - any thoughts?

PCAssistSoftware commented 3 years ago

Anyone??

PCAssistSoftware commented 3 years ago

Every issue on here seems to get a response, but months later not one reply to my issue - can someone please help/advise? thank you

PCAssistSoftware commented 2 years ago

Could someone please at least try and assist?

honfika commented 2 years ago

Could you please specify, how to repoduse this problem with a public site? Otherwise we can't do too much.

Thank you for that you want to use this project even when you have other aleternatives. We also want to fix this issuse, but without reproducing it, it is not possible :(

PCAssistSoftware commented 2 years ago

Thank you for replying.

It turned out in the end that the initial error with my app was caused because there were old proxy settings in the registry of the machines on a domain I was using, they were showing in registry but not in Control Panel > Internet Options > Connections tab > LAN settings.

Once these settings were deleted my problem was resolved.

As I said it was odd because both Fiddler and Wireshark allowed me to see the cause of the blockage (e.g rogue proxy settings) but Titanium didn't, it seemed to mask the real error (System.Net.WebException: The remote server returned an error: (403) Forbidden) which contained the cause with another error instead (System.Net.WebException: The underlying connection was closed: The connection was closed unexpectedly)

You should be able to replicate the issue by adding below registry key, and then trying to access another website or service on web e.g. sending email or similar. You can even setup free account with postmark and I can supply a bit of code to send an email to replicate it potentially.

HKEY_CURRENT_USER\SOFTWARE\Microsoft\Windows\CurrentVersion\Internet Settings

Set

AutoConfigURL

to

http://webdefence.global.blackspider.com:8082/proxy.pac

Hope this helps, if not please come back to me and I will try and create a sample app to demonstrate it.

PCAssistSoftware commented 2 years ago

@honfika

I have created a sample app to fully demonstrate the problem for you, which I hope will help.

Attached is a small project which sends an email using the service from Postmark (problem occurs with other web services this is just easiest one for me to demonstrate with)

N.B. don't change email addresses or API GUID in app or it won't work - it is tied to a trial account I have setup just for demonstration purposes for this issue, so will only work with GUID and email address already set.

If you run the app normally (from bin folder rather than in VS) and click the Send Email button it will send a test email and confirm success in the text box.

If you now use the set.reg file included (in debug/bin folder) to set a proxy value in the registry (as above) and then run the app (from bin folder again) and click Send Email you now get a "System.Net.WebException: The remote server returned an error: (403) Forbidden" error.

So I was getting this problem with an app on multiple client computers (on same site) and needed to see what was blocking it, so immediately thought of using Titanium BUT if you have Titanium running (I was using the sample app - Titanium.Web.Proxy.Examples.Wpf) and you try click Send Email then you now get a different error "System.Net.WebException: The underlying connection was closed" so are then unable to see the cause of the original error as this new error masks the problem.

I then tried using both Fiddler and Wireshark instead and they worked perfectly fine and I could see that the cause of the error was traffic being blocked by "http://webdefence.global.blackspider.com:8082/proxy.pac" which then led me to find that URL in registry which led me to realize it was corrupt / old proxy settings and thus solved my problem.

It would be nice if I could have used Titanium to do this then and hopefully in future, hence the issue I logged above last April.

N.B. Run remove.reg (in debug/bin folder) to remove the proxy setting from your registry if not internet won't work.

Any questions at all please get back to me.

titanium_issue_example.zip

honfika commented 2 years ago

Thanks.. I think I found the problem.

When the upstream proxy fails, the response (unauthenticated) is simply ignored. (When it is not 200)

https://github.com/justcoding121/titanium-web-proxy/blob/9d2b5340bf834773679f381fc1273531bdd1c027/src/Titanium.Web.Proxy/Network/TcpConnection/TcpConnectionFactory.cs#L545-L551

I'll try to fix it, but don't know yet how... since now the response from the proxy connection should be passed to the client.

PCAssistSoftware commented 2 years ago

Okay thanks, any testing you need me to do this end just let me know :)

PCAssistSoftware commented 1 year ago

any progress on this?