How to find the source of "TelemetryChannel found a telemetry item without an InstrumentationKey"

[rnarayana]

• List of NuGet packages and version that you are using:

Microsoft.ApplicationInsights Version="2.15.0"
Microsoft.ApplicationInsights.AspNetCore Version="2.15.0"
Microsoft.ApplicationInsights.Kubernetes Version="1.1.2"
Microsoft.ApplicationInsights.NLogTarget Version="2.15.0"

• Runtime version (e.g. net461, net48, netcoreapp2.1, netcoreapp3.1, etc. You can find this information from the *.csproj file): .Net Core 3.1.7
• Hosting environment (e.g. Azure Web App, App Service on Linux, Windows, Ubuntu, etc.): Kubernetes (Rancher) and also on AKS on Alpine Linux

What are you trying to achieve? Almost all the telemetry comes through properly, but I still get the trace "AI: TelemetryChannel found a telemetry item without an InstrumentationKey. This is a required field and must be set in either your config file or at application startup." at regular intervals The key is set as the first step in ConfigureServices() itself in Startup.cs:

        public void ConfigureServices(IServiceCollection services)
        {
            var appInsightsKey = this.appConfig.GetConfig("App:ApplicationInsightsKey"); // Get my key
            services.AddAppInsights("LISTENER", appInsightsKey); // See extension method below
            services.AddControllers();
            ....
        }

        public static void AddAppInsights(this IServiceCollection services, string cloudRoleName,
            string instrumentationKey)
        {
            services.AddSingleton<ITelemetryInitializer>(new CloudRoleNameInitializer(cloudRoleName));
            services.AddApplicationInsightsTelemetry(o =>
            {
                o.InstrumentationKey = instrumentationKey;
                o.EnableAdaptiveSampling = false;
            });
            services.ConfigureTelemetryModule<DependencyTrackingTelemetryModule>((module, o) =>
            {
                module.EnableSqlCommandTextInstrumentation = true;
            });
            services.AddApplicationInsightsTelemetryProcessor<HealthCheckExclusionFilter>();
            services.AddApplicationInsightsTelemetryProcessor<CustomPropertyFilter>();
            services.AddApplicationInsightsKubernetesEnricher();
        }

What have you tried so far? I tried adding a TelemetryProcessor to try and inspect each item and figure out this, but did not get any leads.

[Saibamen]

Are you able to run the app locally (without docker or app service), and it still repros?

No. Everything is fine locally (Visual Studio on Windows 10)

[cijothomas]

@Saibamen I do not see a repro with the code shared. If things work locally, but not when you deploy to Docker, can you check if the appsettings.json is copied correctly in docker?

[pushkarajawad]

Any updates on this issue, we are still facing this issue in 2.21.0 version.

[mikeblakeuk]

Any news on this? Also, why add "AddResponseCompression" in the sample? Is that to force the app to emulate being inside a AKS that is not using https?

[kristina-devochko]

@cijothomas Hi, I can confirm that the same issue is happening for .NET 6 worker services and .NET 4.7.2 console apps with NLog provider (Microsoft.ApplicationInsights.NLogTarget 2.21.0) and Application Insights SDK 2.21.0.

Another weird thing is that I can't reproduce it when testing locally - then it works perfectly and data is being sent as expected (both Debug and Release build output), but once it's deployed on the servers, nothing is logged except for the lacking instrumentation key message.

Can you please hint to what workaround can be used for non-web apps? #2070 solved it for ASP.NET Core 6 web apps but the non-web apps are only logging this "TelemetryChannel found a telemetry item without an InstrumentationKey" message. It's non-Azure services, regular Windows services and console apps on-prem that are sending data to an ApplicationInsights instance in Azure.

Can't find any useful info on where to start fixing this as we have followed official guidance on setting up Application Insights with worker services: https://learn.microsoft.com/en-us/azure/azure-monitor/app/worker-service From this thread I see that I'm not the only one experiencing this issue with non-web apps and NLog provider. Would really appreciate help here a lot.

@eriksteinebach maybe you were able to find a solution, ref. your comment https://github.com/microsoft/ApplicationInsights-dotnet/issues/2070#issuecomment-902334050 ? It would be so helpful if you could share how you approached this!

[Saibamen]

@Saibamen I do not see a repro with the code shared. If things work locally, but not when you deploy to Docker, can you check if the appsettings.json is copied correctly in docker?

Yes, it is copied correctly.

[MaxPrisich]

Hi All, I also faced this issue on .NET 7 Worker Service project. Like it was already mentioned by @kristina-devochko @eriksteinebach workaround for web apps doesn't apply for the worker service. So I came up with the following workaround to manually set the connection string for NLog configuration:

private static void AddLoggingAndTelemetry(this IServiceCollection services, IConfiguration configuration)
{
    services.AddApplicationInsightsTelemetryWorkerService();
    services.AddLogging(b =>
    {
        b.ClearProviders();
        b.SetMinimumLevel(LogLevel.Trace);
        b.AddNLog(configuration);
    });

    foreach (var target in NLog.LogManager.Configuration.AllTargets.OfType<ApplicationInsightsTarget>())
    {
        var config = target.GetPrivateField("telemetryClient").GetPrivateField("configuration") as TelemetryConfiguration;
        config.ConnectionString = configuration["ApplicationInsights:ConnectionString"];
    }
}

private static object GetPrivateField(this object o, string field)
    => o.GetType().GetField(field, BindingFlags.NonPublic | BindingFlags.Instance).GetValue(o);

This solution worked for me.

[jannikbeibl]

As for now, this issue still persists for worker services. I found out that the NLog target is using the (deprecated) default constructor of the TelemetryClient class which takes the singleton TelemetryConfiguration.Active as the parameter for the base class constructor. (see here and here)

Although the static TelemetryConfiguration.Active property is also marked as obsolete, you can set the ConnectionString on the Active property which is then taken into account by the NLog target. This may be a bit better than using reflection for setting the property.