PnPCore telemetry changing our ingestion endpoint

hazzinator1 commented 3 weeks ago

Describe the bug

A very strange bug that we've had to liase with Microsoft themselves on. Turns out PnPCore's telemetry was hijacking our ingestion endpoint and making all of our app insights calls route towards you guys'. Below is a transcript from our discussions with them where they debugged it for us:

Connection string for target component: InstrumentationKey={our-key};IngestionEndpoint=https://northeurope-2.in.applicationinsights.azure.com/;LiveEndpoint=https://northeurope.livediagnostics.monitor.azure.com/;ApplicationId={our-app-id}

In the PerfView trace we do see telemetry tied to a different ikey, further checks reveals that the ikey is is for pnpcoresdk.

There is some unknown timing bug involving the PnPCore SDK and it seems you are using that same SDK in your application. When your apps get into this state, if you simply restart the web app the problem will go away and may take awhile to come back if ever.

On perfview trace that there are telemetry records produced where the ikey is set to ffe6116a-bda0-4f0a-b0cf-d26f1b0d84eb. This is an Ikey for App Insights used by pnp core sdk team. See code here: https://github.com/pnp/pnpcore/blob/6d0ed405fa471b8afae350d8d9e5ea94197e1208/src/sdk/PnP.Core/Services/Core/TelemetryManager.cs#L14 . This is some un-reproducible and un-identified bug where a custom web application that uses the PnPCore SDK can randomly see telemetry getting sent from their app insights instance over to wrong endpoint.

Steps to reproduce

Reproduction is very difficult here, as it's clearly only been us that's seen it online! However, it does require that DisableTelemetry is false in the PnPCore config.

Expected behavior

We keep using own own app insights telemetry

Environment details (development & target environment)

SDK version: 1.11.0
OS: Windows 11
SDK used in: ASP.Net Web app
Framework: .NET Core v3.x
Browser(s): Chrome
Tooling: Visual Studio 2022 Professional

jansenbe commented 3 weeks ago

@hazzinator1 : weird, not sure if this is an issue in our code but more of an Azure issue. Anyway, feel free to set DisableTelemetry to true in your setup to avoid this.

jansenbe commented 5 days ago

Closing as there's a workaround by disabling PnP Core SDK telemetry

pnp / pnpcore