justcoding121 / titanium-web-proxy

A cross-platform asynchronous HTTP(S) proxy server in C#.
MIT License
1.93k stars 618 forks source link

my code suddenly works only once for the same proxy port until restart #799

Open juraj111 opened 4 years ago

juraj111 commented 4 years ago

Hi guys,

I have been using Titanium Web Proxy in a project for over a year and it worked super sweet. However, in the past few weeks, the same code as before, has been causing some unexpected errors.

Symptoms: When start my program for the first time after the computer is restarted (say proxy using ExplicitEndPoint on port 8000), it works fine. When I stop and start the same program again, the proxy starts, but blocks all internet communication, and no data is captured from the proxy. I get no errors or exceptions, the program finishes "as usual", apart from that I can visually see, that the website, which Titanium was trying to capture data from, was never loaded even after 5 minutes, so no data was captured. The whole internet communication stops (including load of the desired website) on my computer, and it resumes just after the proxyServer.Stop() method was called in the code. Proxy did not capture any data. The program itself did not freeze in between, and my waiting loop, where I wait for about 3-5 minutes, worked and escaped well. Now when I go down to my source code, StartProxy method, change the proxy port to 8001, recompile, run the new .exe file... now everything works fine. Again, when I stop and rerun the same program (with the same port 8001), the same problem appears. I tried to change to code to use port 8002, and the same behaviour appears> it runs fine for the first time, and everything halts the second time I run the program - and resumes just after proxyServer.Stop() was called in my code. When I look at Windows netstat, I can see many entries with ports 8000, 8001, 8002 with state CLOSE_WAIT and FIN_WAIT_2. If I change the code to run on port 8003, the same problem will repeat, and now netstat will show a lot of entries on ports 8000, 8001, 8002, 8003. Before calling proxyServer.Stop(), i use the same code as displayed in the examples. I was trying to figure out what could have changed since May and June, when it ran perfectly on my development machine. I recently upgraded my .NET framework from 4.6 to 4.8. Now I uninstalled .NEt Framework 4.8 and moved back to the same setup I had months ago with .NET 4.6 and Targeting 4.5.2, with reloading NuGet libraries for 4.5.2. Same problem. I tried to experiment with many versions of .NET framework - same problem. Apart from using Titanium Web Proxy to grab HTTP Headers in communication between my own computer and one website (in connection with .NETs built-in WebBrowser component), I do not work with network communication or ports in my code, so its not really possible that other parts of my code would open and "forget to close" some ports. My need is to run the same program multiple times a day on a production environment on a server (automated), and I can't really set it use a new port on each run, leaving used ports in a zombie-state... The same problem, which I ran into on my development PC, seems to be on the server as well.

I am using Windows 10, .Net framework 4.5.2 and have also used 4.8. C#. Using the latest version of Titanium-Web-Proxy from NuGet.

What could be causing the issue and how to fix it? I have tried over 20 different approaches in the code, and the result is always the same.

Thank you very much for any help. Juraj

jxchenhui commented 4 years ago

I had solved my problem,var debug,find that the rootCert.pfx is the problem,program create it when run the app the first time,but when run it secondly,it try to load it,but it fail,so my method is changing the rootCert.pfx format using openssl,just below,and it works:(find method from other website): I had the same problem on Windows 8 and Server 2012/2012 R2 with two new certificates I recently received. On Windows 10, the problem no longer occurs (but that does not help me, as the code manipulating the certificate is used on a server). While the solution of Joe Strommen in principle works, the different private key model would require massive change to the code using the certificates. I find that a better solution is to convert the private key from CNG to RSA, as explained by Remy Blok here. Remy uses OpenSSL and two older tools to accomplish the private key conversion, we wanted to automate it and developed an OpenSSL-only solution. Given MYCERT.pfx with private key password MYPWD in CNG format, these are the steps to get a new CONVERTED.pfx with private key in RSA format and same password: Extract public keys, full certificate chain: OpenSSL pkcs12 -in "MYCERT.pfx" -nokeys -out "MYCERT.cer" -passin "pass:MYPWD" Extract private key: OpenSSL pkcs12 -in "MYCERT.pfx" -nocerts –out “MYCERT.pem" -passin "pass:MYPWD" -passout "pass:MYPWD" Convert private key to RSA format: OpenSSL rsa -inform PEM -in "MYCERT.pem" -out "MYCERT.rsa" -passin "pass:MYPWD" -passout "pass:MYPWD" Merge public keys with RSA private key to new PFX: OpenSSL pkcs12 -export -in "MYCERT.cer" -inkey "MYCERT.rsa" -out "CONVERTED.pfx" -passin "pass:MYPWD" -passout "pass:MYPWD" If you load the converted pfx or import it in the Windows certificate store instead of the CNG format pfx, the problem goes away and the C# code does not need to change. One additional gotcha that I encountered when automating this: we use long generated passwords for the private key and the password may contain ". For the OpenSSL command line, " characters inside the password must be escaped as "".

juraj111 commented 4 years ago

Thanks for your comment. It seems that my problem is a bit different, because: 1) I am using Windows 10 2) Even if I delete the rootCert.pfx after the first run, the problem does not disappear...

juraj111 commented 4 years ago

I have tried toset proxyServer.ReuseSocket to false

Now on the second run (while completely turning off and on the program in between) get an Exception.... which really sounds like proxyServer does not free up ports when stopping...

System.Exception: Endpoint Titanium.Web.Proxy.Models.ExplicitProxyEndPoint failed to start. Check inner exception and exception data for details. ---> System.Net.Sockets.SocketException: Only one usage of each socket address (protocol/network address/port) is normally permitted at System.Net.Sockets.Socket.DoBind(EndPoint endPointSnapshot, SocketAddress socketAddress) at System.Net.Sockets.Socket.Bind(EndPoint localEP) at System.Net.Sockets.TcpListener.Start(Int32 backlog) at Titanium.Web.Proxy.ProxyServer.listen(ProxyEndPoint endPoint) --- End of inner exception stack trace --- at Titanium.Web.Proxy.ProxyServer.listen(ProxyEndPoint endPoint) at Titanium.Web.Proxy.ProxyServer.Start() at Titanium.Web.Proxy.Examples.Basic.ProxyController.StartProxy()

bbhxwl commented 4 years ago

I can do it on PC, not on IOS? Never before. Not now

        proxyServer.CertificateManager.RootCertificate = new X509Certificate2(basePath1 + @"rootCert.pfx", string.Empty, X509KeyStorageFlags.Exportable);
         File.WriteAllBytes("ca.cer", proxyServer.CertificateManager.RootCertificate.Export(System.Security.Cryptography.X509Certificates.X509ContentType.Cert));
HughJeffner commented 3 years ago

I had a similar problem and I figured out it was because I spawning a child process that runs after my app is closed. When my app closed, the port was still in use but the owning process was 'detached'. Closing either the client app or the spawned process released the port.

I'm not entirely sure why but spawning the process before starting the proxy prevented that behavior, but the opposite caused the issue above.