Description
Customer reported a strange issue with their app when recycle app pool. They do this on daily bases to clear built up memory and also to keep the performance of the app up. Once the server has been restarted IIS is unstable.
Customer also reported that the only solution for this had been to replace the newrelic.config file in every website – no changes, just put the exact same file back. Once this is done the Application Pools stabilise and CPU usage drops. They also saw memory issues with the app pools dropping memory until this config is replaced.
Agent is writing thousands of log entries with the below error.
2022-07-25 16:31:16,464 NewRelic ERROR: [pid: 106828, tid: 20] Unable to connect to the New Relic service at collector.newrelic.com:443 (Proxy: newrelic-proxy.advhc.net:3128) : System.Web.HttpException (0x80004005): An error occurred while attempting to impersonate. Execution of this request cannot continue.
at System.Web.ImpersonationContext.ImpersonateToken(HandleRef token)
at System.Web.ApplicationImpersonationContext..ctor()
at System.Web.Util.FileUtil.PhysicalPathStatus(String physicalPath, Boolean directoryExistsOnError, Boolean fileExistsOnError, Boolean& exists, Boolean& isDirectory)
at System.Web.CachedPathData.GetConfigPathData(String configPath)
at System.Web.CachedPathData.GetConfigPathData(String configPath)
at System.Web.CachedPathData.GetApplicationPathData()
at System.Web.Configuration.HttpConfigurationSystem.GetApplicationSection(String sectionName)
at System.Web.Configuration.HttpConfigurationSystem.GetSection(String sectionName)
at System.Web.Configuration.HttpConfigurationSystem.System.Configuration.Internal.IInternalConfigSystem.GetSection(String configKey)
at System.Configuration.ConfigurationManager.GetSection(String sectionName)
at System.Configuration.ConfigurationManager.get_AppSettings()
at System.Net.ConnectStream.GetResponseDrainTimeout()
at System.Net.ConnectStream.DrainSocket()
at System.Net.ConnectStream.CloseInternal(Boolean internalCall, Boolean aborting)
at System.Net.ConnectStream.System.Net.ICloseEx.CloseEx(CloseExState closeState)
at System.Net.ConnectStream.Dispose(Boolean disposing)
at System.IO.Stream.Close()
at System.IO.Compression.DeflateStream.Dispose(Boolean disposing)
at System.IO.Stream.Close()
at System.IO.Compression.GZipStream.Dispose(Boolean disposing)
at System.IO.Stream.Close()
at System.IO.StreamReader.Dispose(Boolean disposing)
at System.IO.TextReader.Dispose()
at NewRelic.Agent.Core.DataTransport.HttpCollectorWire.GetResponse(WebRequest request)
at NewRelic.Agent.Core.DataTransport.HttpCollectorWire.SendData(String method, ConnectionInfo connectionInfo, String serializedData)
at NewRelic.Agent.Core.DataTransport.ConnectionHandler.SendDataOverWire[T](ICollectorWire wire, String method, Object[] data)
at NewRelic.Agent.Core.DataTransport.ConnectionHandler.SendNonDataRequest[T](String method, Object[] data)
at NewRelic.Agent.Core.DataTransport.ConnectionHandler.SendPreconnectRequest()
at NewRelic.Agent.Core.DataTransport.ConnectionHandler.Connect()
The error above shows that when the agent made a request to the collector using WebRequest, the WebRequest under the hood uses the ConfigurationManager to read some arbitrary settings from some physical places and this failed with the impersonation error.
With this observation, I believe the symptom has something to do with the customer's hosting environment and the way their apps are being hosted. Upon the investigation, the followings are what stands out to me about the customer setup:
They use proxy.
They use Web farm to host their applications and each of the applications is placed on a central network file share drive.
Expected Behavior
No errors
Steps to Reproduce
It MAY be reproducible if we can host a .NET Framework app on a server that has the following conditions:
The server has a proxy.
The server uses Web farm to host applications and each of the applications is placed on a central network file share drive.
Additional context
The agent currently uses WebRequest to send data to the collector. WebRequest is deprecated so we might consider migrate to use HttpClient to handle data send. This might or might not move the needle but it is still worth to try.
For Maintainers Only or Hero Triaging this bug
*T-Shirt size: L, Unknown
What happened?
Description Customer reported a strange issue with their app when recycle app pool. They do this on daily bases to clear built up memory and also to keep the performance of the app up. Once the server has been restarted IIS is unstable.
Customer also reported that the only solution for this had been to replace the newrelic.config file in every website – no changes, just put the exact same file back. Once this is done the Application Pools stabilise and CPU usage drops. They also saw memory issues with the app pools dropping memory until this config is replaced.
Agent is writing thousands of log entries with the below error.
The error above shows that when the agent made a request to the collector using WebRequest, the WebRequest under the hood uses the
ConfigurationManager
to read some arbitrary settings from some physical places and this failed with the impersonation error.With this observation, I believe the symptom has something to do with the customer's hosting environment and the way their apps are being hosted. Upon the investigation, the followings are what stands out to me about the customer setup:
Expected Behavior No errors
Steps to Reproduce It MAY be reproducible if we can host a .NET Framework app on a server that has the following conditions:
Additional context The agent currently uses WebRequest to send data to the collector. WebRequest is deprecated so we might consider migrate to use HttpClient to handle data send. This might or might not move the needle but it is still worth to try.
For Maintainers Only or Hero Triaging this bug *T-Shirt size: L, Unknown