How to specify Proxy Settings

[epignosisx]

We were trying out Application Insights at work and we are not getting any telemetry in the Azure portal other than Page Views. We are pretty sure it is related to our Corporate proxy (it has caused us high levels of pain in the past).

How can I specify the proxy settings? The default proxy will not cut it for us. We have to specify url, port, username and password for our proxy.

For our .NET Core we have to set it to the HttpClientHandler like this:

var httpClientHandler = new HttpClientHandler {
    Proxy = new MyProxy(), //our implementation of IWebProxy
};

I couldn't find the code in this repo in charge of sending the telemetry but if you are using HttpClient then it would be something similar.

Note that we are planning to use Application Insights in house, not in Azure, so we have the proxy restriction in all our environments.

[dnduffy]

Transmission is handled by the base SDK. Which specific class depends on whether you are using the InMemoryChannel or the ServerTelemetryChannel.

[epignosisx]

Thanks for pointing me in the right direction. I'm using the ServerTelemetryChannel. Looks like the issue is in this line.

Ideally, I would like to be able to provide Application Insights with an HttpClient to use or at least be able to participate in the creation of the HttpClient so I can set the Proxy of the HttpClientHandler.

Should I move this issue to the ApplicationInsights-dotnet repo?

[dnduffy]

@epignosisx Yes, please feel free to move it there, and if you want to submit a pull request that would be awesome!

[crippe]

@dnduffy | @epignosisx

What is the status of this issue? Is there a new code that can handle proxy configurations? We are also governed by a proxy, though we use .NET 4.5 and then it seems like CreateRequest (https://github.com/Microsoft/ApplicationInsights-dotnet/blob/develop/src/Core/Managed/Shared/Channel/Transmission.cs#L374) is used today.

If no one had the opportunity to do something, can/should I make the changes and add a pull request?

In order to simplify, there should be something like:

if (this.UseProxy)
{
    request.Proxy = new WebProxy (this.ProxyAddress, this.ProxyBypassOnLocal);
}

[dnduffy]

@crippe Are you able to get the channel from dependency injection and modify it? If not then please feel free to submit a pull request in that repo and we'll plan to take it in the next version.

[caretro]

Hi @dnduffy, any update on this issue? Is not it still possible to handle proxy configuration? We are governed by a proxy too. We are using the ServerTelemetryChannel.

[crippe]

Shortly after my comment, it was decided centrally to use technology other than Application Insights. I'm sorry for the decision, but I could not do anything about it. Therefore, I did not make a request.

[caretro]

Hi, any update on this issue. Seems that the milestone is moved forward continuously...

[tremblaysimon]

Since HttpClient uses system proxy, I think it's possible to use a tool like cntlm to store the proxy credentials and configure a system proxy without a username/password.

[wernerb90]

Any update on this issue, or an alternative example to use AI with a proxy on-prem?

[cijothomas]

@wernerbarnard If you are configuring a system wide proxy, then SDK would just pick it up. If you'd like to configure proxy settings to applicationinsights channel - that is not supported today. I'll tag this as an enhancement.

[davebally]

I too am hitting this issue using AI on .Net Core behind proxy. Im not convinced that HttpClient in .NET Core does use the system proxy either so need to be able to set the proxy explicitly on AI to resolve.

[cijothomas]

@davebally Please run the following in a .NET Core Console app, and check if it respects system proxy. If it does, then it'll print the AppId for the given ikey.

class Program
    {
        static void Main(string[] args)
        {
            HttpClient client = new HttpClient();
            var res = client.GetStringAsync("https://dc.services.visualstudio.com/api/profiles/your_instrumentationkey/appId").Result;
            Console.WriteLine(res);
        }       
    }

[davebally]

@cijothomas Thanks, i retract that statement. Looks like all is well, i can see the call coming through in fiddler without an explicit proxy being set.

[nulltoken]

@cijothomas Unfortunately, this doesn't seem to work in all cases.

Running this code in our corporate environment raises a 407 error.

In our case, to make this pass, we have to define a HttpHandler, value the Proxy and UseProxy properties, and build the HttpClient from it. From a dev environment standpoint, that relies on running a local cntlm proxy, and configuring the Proxy property so that it points to it.

Teaching AppInsights to accept a HttpHandler would be very helpful for us.

[cijothomas]

@nulltoken Yes we are tracking this as an enhancement - to modify to accept httpclient settings.

[davebally]

@cijothomas Can you confirm that the HttpClient that is returned by default from IHttpClientFactory should be using the proxy ? When i set the proxy with the below code i see the call come through in fiddler, if i dont , i dont

Thanks.

in startup ....

 services.AddHttpClient("external", c => { c.DefaultRequestHeaders.CacheControl = GetCacheControlHeader(); })
                .ConfigureHttpMessageHandlerBuilder(c => c.PrimaryHandler = GetHttpClientHandler(true));

  private static HttpClientHandler GetHttpClientHandler(bool isExternal = false)
        {
            var handler = new HttpClientHandler
            {
                AutomaticDecompression = System.Net.DecompressionMethods.GZip,
                SslProtocols = SslProtocols.Tls12,
                ClientCertificateOptions = ClientCertificateOption.Manual
            };

            if (isExternal)
            {

                var proxy = GetConfigProperty("Config", "Proxy", "Uri");
                if (!string.IsNullOrEmpty(proxy.Value))
                {
                    handler.Proxy = new WebProxy()
                    {
                        Address = new Uri(proxy.Value)
                    };
                }
            }
            else { 

                handler.ServerCertificateCustomValidationCallback = (message, cert, chain, errors) => true;
            }
            return handler;
        }

[cijothomas]

@davebally Sorry, i dont have any expertise in that area :( Its best to ask this in HttpClient github repo.

[wernerb90]

@davebally - for me running inside docker with HTTP_PROXY and HTTPS_PROXY set as env variables, new HttpClient did NOT respect the proxy (default settings - not using your sample), but a HttpClient created from IHttpClientFactory did.

[cijothomas]

This is not planned for the next release 2.8.

[ArchonMegalon]

Is it planned at all? Having exactly the same problem here...

[cijothomas]

It is, and tagged with Future milestone. This means its not likely to land in next 2 stable releases. Please upvote the issues here so we promote items from Future to nearer milestones.

[mvdburght]

The following worked for us. Run it in the StartUp of your application:

private static void ConfigureProxyForApplicationInsightsHack(IWebProxy proxy)
{
    System.Runtime.CompilerServices.RuntimeHelpers.RunClassConstructor(typeof(Transmission).TypeHandle);
    var field = typeof(Transmission).GetField("client", BindingFlags.Static | BindingFlags.NonPublic);
    field.SetValue(null, new HttpClient(new HttpClientHandler { Proxy = proxy, UseProxy = true }));
}

[cijothomas]

@mvdburght thanks for sharing. The enhancement we are tracking here is to allow users to supply HttpClient for the channel/transmission.

[Swellenator]

I was getting a 407 Proxy Authorisation Required type message. (which by the way I had to use WireShark to discover) I got it to work using @mvdburght 's workaround. My code is using a console app so I have this at the start of my main method:

 var proxy = Settings.GetSection("Proxy").Get<WebProxy>();

            if (!string.IsNullOrWhiteSpace(proxy.Address.ToString()))
            {
                var handler = new HttpClientHandler
                {
                    Proxy = proxy
                };

                System.Runtime.CompilerServices.RuntimeHelpers.RunClassConstructor(typeof(Transmission).TypeHandle);
                var field = typeof(Transmission).GetField("client", BindingFlags.Static | BindingFlags.NonPublic);
                field.SetValue(null, new HttpClient(handler));
            }

and in my config:

  "Proxy": {
    "Address": "http://[Url]:[Port]",
    "UseDefaultCredentials": true
  },

Hope that helps someone. Would be good to have an official way of doing it.

[funnay]

Hi guys.

We're using App Insights in our .NET Core services and these services are running on premises in an Azure Service Fabric backend cluster. This cluster does not have internet/azure direct access.

Right now we're doing a bunch of infrastructure "hacks" using Squid in intercept mode and DNS configs so that the App Insights requests can get to Azure, without affecting the service. It works but it's ugly and it's having unintended consequences, because of the way it's set up.

It would be much easier to be able to specify a proxy exclusive to the App Insights telemetry, without changing the rest of the service.

Do you think we could use @mvdburght suggestion in this scenario?

[pharring]

Instead of reflection, can't you set the default proxy globally for the app via System.Net.WebRequest.DefaultWebProxy ? i.e.

using System.Net;
...
// in Startup
WebRequest.DefaultWebProxy = new WebProxy(proxyUri);

Or do you want to set the value differently for App Insights?

[pharring]

Oh, sorry; that doesn't work for HttpClient. In .NET Core 3.0 and 3.1, there's an HttpClient.DefaultProxy static property: https://docs.microsoft.com/dotnet/api/system.net.http.httpclient.defaultproxy

[funnay]

Or do you want to set the value differently for App Insights?

This. Do you think it's possible?

[funnay]

Hello.

Can someone please confirm if it's possible to specify a proxy only/different for App Insights using the @mvdburght suggestion?

[pharring]

@funnay, I think @Swellenator already confirmed it. This isn't officially supported since it uses reflection to replace a private field value. I also think that it won't work on applications targeting net45.

[funnay]

Thanks @pharring.

This should be a basic feature for on-premises backend services.

[SuberFu]

Oh, sorry; that doesn't work for HttpClient. In .NET Core 3.0 and 3.1, there's an HttpClient.DefaultProxy static property: https://docs.microsoft.com/dotnet/api/system.net.http.httpclient.defaultproxy

Does setting that proxy actually helps? I tried it and Application Insights does not appear to use proxy.

I'm on dotnet core 3.1

[tebeco]

In .NET Core 3.0 and 3.1, there's an HttpClient.DefaultProxy static property

@pharring That's still a big NO You probably don't ever want any proxy to be a global DefaultProxy because if your infra have LAN / Cloud / Local dev laptop All your traffic will implicitly use the "default" which will possible end up in HTTP 502

Any attempt of "Big Bang Default Global" Proxy like

• web.config (only netfx)
• env var
• DefaultProxy
• ServiceEndpointManager etc ...

Are terrible solution because they will only works if literally ALL your code and all your http traffic run in a single place toward one single place If you try to go that way you will have to handle things like NO_PROXY or ByPass or "this needs auth but not this" or "but here I'm on AKS I don't need a proxy" For example, if you have an application on Premise that needs to reach a backend, you're not supposed to use a proxy, but you need to access ApplicaitonInsight And if you have another Service running in AKS you need another network route to be used (possibly another proxy or Express Route + Custom DNS)

The equivalent of PAC script would be a splendid way to handle this because you would have an interceptor of all HttpClient attempt to create a connection (when no explicit proxy specified) and you would be able to inject it properly I think there's an API that look like this, I'm trying to find it back (It was about Proxies and not Proxy IIRC)

Each and every single package dealing with HttpClient must have an extension point allow consumer to use a SPECIFIC proxy for that SPECIFIC package It means "having the choice before the traffic even exist" over "adapt your network traffic with exception rules and hope your topology never change"

As stated in this ticket here we ended up with something very similar to have that "granularity"

        private static void AddApplicationInsightsProxy(this IServiceCollection services, Uri proxyUri)
        {
            try
            {
                var httpClientProxyHandler = new HttpClientHandler
                {
                    Proxy = new WebProxy(proxyUri)
                    {
                        Credentials = CredentialCache.DefaultNetworkCredentials,
                        BypassProxyOnLocal = true,
                    },
                    DefaultProxyCredentials = CredentialCache.DefaultNetworkCredentials,
                };

                /************************************************************************************************************************************************/
                // This is a terrible idea ... but so far the SDK is meh! regarding proxy
                /************************************************************************************************************************************************/
                /************************************************************************************************************************************************/
                // Get the System.Type associated to the Microsoft.ApplicationInsights.Channel.Transmission
                // This class is the one responsible of sending Telemetry data
                var transmissionType = typeof(Transmission);
                // Force static constructor invocation of this class
                RuntimeHelpers.RunClassConstructor(transmissionType.TypeHandle);
                // Get the "private static" field named "client" in that class.
                // see: https://github.com/microsoft/ApplicationInsights-dotnet/blob/cb87710f3c1f369e942becb2d52164f754347b78/BASE/src/Microsoft.ApplicationInsights/Channel/Transmission.cs#L23
                var field = transmissionType.GetField("client", BindingFlags.Static | BindingFlags.NonPublic);

                // In case of refactoring, if a variable named "client" still exists but is not "HttpClient" anymore
                if (field.FieldType == typeof(HttpClient))
                {
                    // Swap the private static field with our custom one, where we injected a proxy
                    // The null here is supposed to indicate the "Instance" of "Transmission" class we assign that too
                    // It is null because the field is "static"
                    // see (for timeout) https://github.com/microsoft/ApplicationInsights-dotnet/blob/cb87710f3c1f369e942becb2d52164f754347b78/BASE/src/Microsoft.ApplicationInsights/Channel/Transmission.cs#L23
                    field?.SetValue(null, new HttpClient(httpClientProxyHandler) { Timeout = System.Threading.Timeout.InfiniteTimeSpan });
                }
                /************************************************************************************************************************************************/
                /************************************************************************************************************************************************/
            }
            catch
            {
#if DEBUG
                // We don't want to throw in RELEASE
                // The idea is to see it throwing only when using DEBUG build
                throw;
#endif
            }
        }

[espenrl]

@cijothomas

Please stop being ignorant of this issue. This is a normal situation for many applications - needing to pass the traffic through a proxy. This issue is three years old and it has not got the attention that it needs. An occasional outreach of it is "tagged with Future milestone" is not sufficient.

[cijothomas]

I can share the obvious bad news for most : This is not planned in the coming SDK release (2.16), and that means this is not going to be solved in 2020.

Tagging @ank3it to to treat this as a top priority issue for Jan-June 2021 SDK releases.

[github-actions[bot]]

This issue is stale because it has been open 300 days with no activity. Remove stale label or comment or this will be closed in 7 days.

[tebeco]

but it's not solved, so not stale I guess

[tebeco]

maybe @cijothomas or @reyang have a way to unstale that issue until the release of that feature ?

[TimothyMothra]

Thank you for commenting, commenting will automatically remove the stale label (when the bot runs this evening).

Please do continue to comment and upvote, this helps our team to prioritize issues!

I understand this is a long open issue and I'm sorry for the added frustration. Our team is aware of this issue and it's on our "want to-do" list but hasn't been committed for the current semester of work.

I believe the workaround that has worked for customers is to have a custom endpoint defined in the firewall that forwards telemetry to our ingestion. The SDK can then be configured to send telemetry to that custom endpoint.

[tebeco]

I believe the workaround that has worked for customers is to have a custom endpoint defined in the firewall that forwards telemetry to our ingestion. The SDK can then be configured to send telemetry to that custom endpoint.

No it's not a working workaround sadly. This issue is about Proxy, not firewall. In order to add that firewall rule you're assuming few things:

• NTLM Proxy are not anonymous proxy and you'd have to setup it explicitly
• Internet access is completly forbidden in some company exception done on very specific infra
• if by any chance that connection happens to be on a network "path" that
  • the sec team allow to use (generally browsing != application)
  • you happen to find the right place for all the layer of possible firewall
  • the sec team allowing AppInsight on that "browsing/outgoing/default path" firewall ... then maybe you're lucky

see answer there: https://github.com/microsoft/ApplicationInsights-dotnet/issues/1554#issuecomment-720450913

also pinging @pharring on that one, as the static DefaultProxy is a non-goal:

The HttpClient causing issue is well identified. It's a private static one, being instanciating ... using the static ctor The fix would be to use the existing Options and let us (consumer) specify an HttpHandler and/or an IWebProxy and pass it down all layer until that one

[cijothomas]

This is how OpenTelemetry has solved this for Zipkin exporter : https://github.com/open-telemetry/opentelemetry-dotnet/tree/main/src/OpenTelemetry.Exporter.Zipkin#configure-httpclient

[molinch]

@cijothomas Is a similar solution to the one done by Zipkin on the roadmap for 2022?

[cijothomas]

@cijothomas Is a similar solution to the one done by Zipkin on the roadmap for 2022?

Not in the plan for next 4 months sure.

it is possible that it'll be solved in the OpenTelemetry based exporter here : https://github.com/Azure/azure-sdk-for-net/tree/main/sdk/monitor/Azure.Monitor.OpenTelemetry.Exporter

[uchitha]

Hi @cijothomas any updates on this? We're using the latest 2.21 libraries.

[johncrim]

There should be a configuration setting for this. My primary use case is to pass it through a dev recording proxy to troubleshoot 400 errors from the app insights endpoint.

[tebeco]

this is still something valuable that most of the Azure SDK took time to setup when ditching MSAL, this would benefits a lot of people here cc @cijothomas

[rusczyk]

Hello there, I am on the same page, will also try to use the mentioned workaraound until there is a way to use the httpclientfactory.

Kind regards