Support for non asp.net core workloads

[dolly22]

I have already seen #49. But I'm not asking for nuget package to magically track worker services.

It would be great if you could restructure published nuget packages, so there was one without Microsoft.AspNetCore.* dependencies only on Microsoft.Extensions.* containing all lower level building blocks such as IServiceCollection registration and configuration management. It looks like the only asp.net core specific dependency is application startup hooking.

I would like to be able to initialize ServiceProfiler in non asp.net core application using this package. It's very hard to achieve this today as many core services are marked as internal and it's not possible to reuse service registration logic without taking dependency to Microsoft.AspNetCore.* or replicating whole functionality while battling internal classes.

I do have some use cases that would benefit from this

1. I'm already using application insights to track distributed telemetry when calling Microsoft Orleans grains. Those call are tracked and modeled as DependencyTelemetry on client side and RequestTelementry on silo side. It's very similar to web application workload from appinsights point of view.

2. I also track all Azure ServiceBus message processing as RequestTelemetry in a similar way

Both of those use cases do have some request processing time and would benefit from ServiceProfiler traces and insights.

[xiaomi7732]

Hey @dolly22 thanks for filing the issue. Trying to understand your requirement: are you looking for a package similar to https://www.nuget.org/packages/Microsoft.ApplicationInsights.Profiler.Core that doesn't have ASP.NET Core dependencies?

[pharring]

@xiaomi7732 this seams like a reasonable request. Consider also the worker service scenario. They may reference Microsoft.ApplicationInsights.WorkerService.

[dolly22]

@pharring @xiaomi7732 Yes, worker services are exactly the reason i am proposing this feature. They usually have only Microsoft.Extensions.* dependencies and use generic host, but still produce interesting RequestTelemetry to application insights.

[xiaomi7732]

Thanks all for the input. It is our goal to support various environments anyway. Treat ASP.NET Core header package (Microsoft.ApplicaitonInsights.Profiler.ASPNETCore) as the pioneer package.

Tell me if I am wrong here, my understanding:

• The gap here is that, in our core package(Microsoft.ApplicationInsights.Profiler.Core), we have [InternalVisibleTo] to allow the header package to leverage internal interface & classes in it.
• Even when you want to write some bootstrap code in other type of project (worker services in this case), we are preventing you from doing so because some core classes/interfaces are not exposed properly.

And to address that issue, we will need to:

• [ ] Remove InternalVisibleTo from Core project - no special treaty for ASP.NET Core header project from that point on.
• [ ] Review the public APIs surface to making sure enough and just enough ones are exposed.

Does that make sense? Any early feedback at this point?

[shlomiassaf]

@xiaomi7732 any updates on the roadmap for this?

Our cluster is composed of multiple services, some are AspNetCore WebApi projects but some are worker services to handle service bus messages.

On high workloads we experience issues in all services, we can profile the WebApi services but not the worker service services.

It seems (without extensive knowladge) that this should be similar in principal with AspNetCore doing everything done in the worker service and then some additional things related to the "ASP" part like http request stuff etc...

In it's core, detecting long wait times, blocks etc is similar in all hosting environments....

[xiaomi7732]

@shlomiassaf Thanks for the ask. We are kind of running tight schedule now. That said, would you mind give me some details about your cluster? Like where are you hosting/running your workers? Is it Azure Functions? If it is function, what is host plan? (Consumption, Premium or Dedicated)? Otherwise, I would assume WebJobs, could you please confirm? Depends on the scenarios, I would like to see if there's anything quick that we could do.

[shlomiassaf]

@xiaomi7732 Our service bus services NETCore 3.1 Microsoft.NET.Sdk.Worker SDK projects.

We're using Microsoft.Extensions.Hosting along with other Microsoft.Extensions.XXX packages to load the service with configuration, etc...

We just implement a class (derived from BackgroundService) and register it via IServiceCollection.AddHostedServices and run everything via the host builder.

The service registers to the bus and listen to incoming messages.

It's very similar to the Asp solution just without the Http stuff for incoming/outgoing things

BTW, I didn't find the github repo for the source code, its also not in the main application insights sdk... It will help seeing it, any links?

Thanks!

[xiaomi7732]

@shlomiassaf Thanks for the details. Now, I understand the app is a service that listens to SB messages. That is great. Depends on some other thing, you might be able to leverage the existing implementations.

One thing that might largely impact the capability that I still do not understand: is that where do you deploy your app to? Is it an Azure WebSite? Is it a WebJob? Is it an Azure Function? Is it a in a K8S pod? Could you please help me understand that?

With regarding the code, it is NOT an open source project and there's no source code shared anywhere. That said, if you are interested in anything specific, feel free to post new issues and we will see what we can provide.

[xiaomi7732]

Hey @bruckgt1 , I think you have been looking into the Azure Function support of EventPipe Profiler. Do you think we could combine the effort with this? Far as I know, Azure Function is built upon Workload.

BTW, here is just clue on how application insights works on worker service that might help us form our solution up: https://github.com/microsoft/ApplicationInsights-Kubernetes/tree/develop/examples/WorkerExample#how-is-the-application-built