search

aws / aws-xray-dotnet-agent

The official AWS X-Ray Auto Instrumentation Agent for .Net.
Apache License 2.0
23 stars 15 forks source link

Could not load type 'Microsoft.AspNetCore.HttpOverrides.Internal.IPEndPointParser' #21

Open Pharrox opened 3 years ago

Pharrox commented 3 years ago

Hello,

We are trying to implement auto-instrumentation into our apps running .NET Core 3.1, with the possibility of upgrading to .NET 5.0 in the future. Currently we are running into an issue where any time a service is hosted behind a load balancer (ALB) in our case, the traces get created but there are no HTTP details in the traces.

Logs show the error:

 AspNetCoreDiagnosticListener 707020|2020-12-21T20:53:59.520Z|ERROR|Invalid diagnostic source key (Microsoft.AspNetCore.Hosting.HttpRequestIn.Start) --> System.TypeLoadException: Could not load type 'Microsoft.AspNetCore.HttpOverrides.Internal.IPEndPointParser' from assembly 'Microsoft.AspNetCore.HttpOverrides, Version=3.1.8.0, Culture=neutral, PublicKeyToken=adb9793829ddae60'.

After some digging I was able to determine that this only happens when the app is hit through the load balancer. Specifically it occurs whenever an X-Forwarded-For header is present in any version of .NET Core newer than 2.2. The exception is being thrown at the line below when trying to parse the header:

https://github.com/aws/aws-xray-dotnet-agent/blob/421faf38c7598180911c99fb5fb9fb9441b7691c/src/sdk/Utils/AspNetCoreRequestUtil.cs#L199

Related: https://github.com/dotnet/runtime/issues/26916 https://github.com/dotnet/corefx/pull/33119 https://github.com/dotnet/aspnetcore/issues/6014

lupengamzn commented 3 years ago

Hey @Pharrox ,

Thanks for pointing this out

The IPEndPointParser.TryParse will be called when there is an X-Forwarded-For header due to the existence of load balancer. And a straightforward way to fix this is to create a similar method in AspNetCoreRequestUtil.cs class to replace IPEndPointParser.TryParse() API.

You may follow the logic of how .NET parse it originally to fix it real quick and we'll be fixing this issue on our end too.

Pharrox commented 3 years ago

Thanks for the response. I was able to work around this, at least for the time being, by replicating the logic used by the X-Ray SDK for handing X-Forwarded-For headers. While I understand the reason for the change, we've been using manual instrumentation with the SDK on some of our simpler apps for a while and found the simplified behavior it uses to be sufficient for our needs.

lupengamzn commented 3 years ago

Thanks for the workaround and feedback and glad you found it's useful!

Also the reason why we applied the parser is in some case when using diagnostic listener the IP addresses in X-Forwarded-For header contains port number which we would not like to record. And replicating the logic in .NET SDK definitely works too as long as it's acceptable in your case to have port number in the client_ip trace area.