[bug] OpenTelemetry.Instrumentation.AspNet makes the other middleware failed due to "System.NotSupportedException: Collection is read-only."

[rahjesh-vunet]

Component

OpenTelemetry.Instrumentation.AspNet

Package Version

Package Name	Version
OpenTelemetry .NET Instrumentation	1.7.0

Runtime Version

net47

Description

In my .NET Framework (4.7) application, I have a OWIN based middleware which set a custom header to the HTTP Request. `public class AuthMiddleware : OwinMiddleware { public AuthMiddleware(OwinMiddleware next) : base(next) { }

    public override async Task Invoke(IOwinContext context)
    {
        context.Request.Headers["Custom-Header"] = "MyHeaderValue";

        await Next.Invoke(context);

    }
}

}`

When I instrument this application using opentelemetry .net instrumentation power shell script, the application got exception. ( Please see the attached screenshot). If I just disable the Asp Net instrumentation (OTEL_DOTNET_AUTO_TRACES_ASPNET_INSTRUMENTATION_ENABLED = false), the application works fine and there was no exception. I tried with Opentelemetry .NET instrumentation versions 1.2.0, 1.4.0 and 1.7.0. All behave same way. Is the Opentelemetry Asp Net Instrumentation locks the request headers of some think like this which affects the other middlewares. My middleware was not able to set a custom header to the http request headers.

But the instrumentation works and I could see the traces in the OTel Collector.

Steps to Reproduce

1. Create a .NET Framework 4.7 Web application. Create a OWIN based middleware to set a custom request header. Create a web api for testing the middleware. You can get the project code in this github repo. https://github.com/rahjesh-vunet/Opentelemetry-AspNet-Instrumentation-Issue
2. Deploy the application in IIS and access the API http://localhost/api/test. You can see the api gives the output. "Hello, world111111111111111!"
3. Instrument this application running in IIS using the powershell script from https://github.com/open-telemetry/opentelemetry-dotnet-instrumentation/releases/tag/v1.7.0
4. Now access the API and you will see error.
5. Add the variable "OTEL_DOTNET_AUTO_TRACES_ASPNET_INSTRUMENTATION_ENABLED" to false in the "Application Settings" of your application in IIS. Restart IIS and check the API and it will work again.

Expected Result

The API works even after Opentelemetry .NET instrumentation

Actual Result

Additional Context

No response

[vishweshbankwar]

@rahjesh-vunet - looks like you are using https://github.com/open-telemetry/opentelemetry-dotnet-instrumentation?

[rahjesh-vunet]

@rahjesh-vunet - looks like you are using https://github.com/open-telemetry/opentelemetry-dotnet-instrumentation? @vishweshbankwar Yes. I was using https://github.com/open-telemetry/opentelemetry-dotnet-instrumentation. This is using the AspNet instrumentation from contrib. Isn't it?

[rahjesh-vunet]

@vishweshbankwar

I had a comment from Mateusz Łach in the slack channel.

[vishweshbankwar]

@lachmatt - Should this be moved to https://github.com/open-telemetry/opentelemetry-dotnet-instrumentation?

[lachmatt]

@vishweshbankwar yes

[Kielek]

@rahjesh-vunet, could you please provide Minimal, Reproducible Example? It will significantly reduce time to reproduce the issue. Then we can try to help/mitigate the problem.

[rahjesh-vunet]

@rahjesh-vunet, could you please provide Minimal, Reproducible Example? It will significantly reduce time to reproduce the issue. Then we can try to help/mitigate the problem.

@Kielek I have provided the following in the issue.

1. Steps to reproduce section which has the github link of the sample code and the complete steps to reproduce the error.
2. The expected result and actual result are also available

If you follow this, you can reproduce the issue.

Is that make sense or am I missing something?