search

braintree / braintree_dotnet

Braintree .NET library
https://developer.paypal.com/braintree/docs/start/overview
MIT License
136 stars 73 forks source link

Getting more and more socket/unable to connect to remote server errors lately #135

Closed nicholashead closed 1 year ago

nicholashead commented 1 year ago

General information

We're seeing something randomly lately - .NET Framework giving us socket "Unable to connect to remote server" errors, and we've had our systems running pretty stable with Braintree for a while (years). We are running NuGet package 5.9.0, and we're hosted on GCP.

System.Net.Sockets.SocketException: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond 76.223.13.31:443
  ?, in void Socket.DoConnect(EndPoint endPointSnapshot, SocketAddress socketAddress)
  ?, in WebExceptionStatus ServicePoint.ConnectSocketInternal(bool connectFailure, Socket s4, Socket s6, ref Socket socket, ref IPAddress address, ConnectSocketState state, IAsyncResult asyncResult, out Exception exception)
System.Net.WebException: Unable to connect to the remote server
  ?, in Stream HttpWebRequest.GetRequestStream(out TransportContext context) x 2
  ?, in XmlNode BraintreeService.GetXmlResponse(string URL, string method, Request requestBody, FileStream file)
  ?, in XmlNode BraintreeService.Post(string URL, Request requestBody)

We're going to try and upgrade to latest (5.17.0) - however I didn't see anything in the diff that may resolve it for us? Braintree support wanted us to upgrade to latest before helping further.

Are we supposed to register BraintreeGateway as singleton in our IOC container? We're not on .NET Core - still on .NET Framework for our monolith, unfortunately. So looks like we're not using HttpClient anyway - but WebRequest?

Years ago we configured ServicePointManager.DefaultConnectionLimit to something higher than defaults - 72 to be exact. I wonder if we are hitting some of those limits (WebRequest is affected by SPM I think) - and just need to increase the connection limit.. but would that lead to socket connection errrors?

I'm running out of ideas.

Any advice/guidance you can provide would be appreciated. Thanks so much!

hollabaq86 commented 1 year ago

I will definitely do my best to help! This issue has come up before but that was for .NET Core integrations. There's no changes between v5.9 and 5.17 that specifically touch the part of our SDK that's creating requests. I'm going to talk through what we're setting in the SDKs and hopefully that can help us narrow down possible causes.

For .NET Framework integrations, we're using WebRequest.Create() to create HttpWebRequest class objects, per Microsoft's docs:

Do not use the HttpWebRequest constructor. Use the WebRequest.Create method to initialize new HttpWebRequest objects.

We do set some configurations for TLS 1.2 and to override the default for keep-alive on requests here: https://github.com/braintree/braintree_dotnet/blob/3a03cb66aad9a002be24e7417e99a05d19e6a976/src/Braintree/HttpService.cs#L114-L127 I do see from Microsoft's docs that HttpWebRequest has a ServicePoint property to it definitely uses that configuration in some way.

Do you have any custom settings for requests that may be overriding the library? Always worth double-checking this, too.

Other questions that are jumping out at me: Are you currently creating one BraintreeGateway object and then using that object when making Braintree calls? We try to stay agnostic so that folks can implement Braintree however they want (re: IoC Container) but that could be a potential area eliminating this issue, or atleast some confusion when investigating.

One thing I did notice when investigating this is that Microsoft recommends folks use HttpClient for new development and not HttpWebRequest, so maybe we should update the library to use a single HttpClient instance for .NET Framework integrations, too...

nicholashead commented 1 year ago

@hollabaq86 thanks for the reply! We are not setting any custom settings for requests that I'm aware of. We are currently always creating a new BraintreeGateway - that is - we don't treat it as singleton/static in our DI. I believe that's okay/correct. But it would be easy enough for us to switch to singleton if that could be causing any problems.

I think switching the library to HttpClient would be a great direction to take, for sure. Especially since MS suggests not using it anymore.

It's just weird - we haven't touched any of our Braintree-related code in a very long time, and starting around 4/25 or so, we're getting this error more often. We didn't even deploy any changes around that time either. My theories so far:

I'll keep trying to dig in our codebase/implementation and see if I can find something. Just was hoping maybe there was something obvious we were missing. I truly appreciate the help/reply.

nicholashead commented 1 year ago

This seems to have resolved itself on its own. We didn't make any changes on our end, so it must've been something outside our control - perhaps GCP (our host) or something on BT's end? I'm not sure. But closing, and I truly appreciate the help and guidance offered @hollabaq86