search

dotnet / orleans

Cloud Native application framework for .NET
https://docs.microsoft.com/dotnet/orleans
MIT License
10.06k stars 2.03k forks source link

Activation does not have an associated Activity #9052

Open rkargMsft opened 3 months ago

rkargMsft commented 3 months ago

Currently the work during grain activation happens outside of the Trace containing the calling client and the message processing on the grain.

It would be useful, if it is feasible, to have an Activate/Activation span to encompass the work being done on grain activation and have that be part of the calling trace.

For example, in the below trace, the two red lines show a point where Activation of the grain happens (I believe that's the correct point in the trace). In this case, there is state hydration that happens (Azure Table Storage Persistence, but it shouldn't matter) but the call to Azure is captured in a separate trace (not pictured) that is unrelated to the calling trace.

There is useful information here that is not correlated, specifically the built-in Persistence hydration. Without this correlated information, it would be difficult to determine why some calls may be slow (if Persistence calls are slow) while others are faster.

image

NzKyle commented 2 months ago

I've just noticed this myself - it would be great to get some traces around what happens when a grain is activated.

rkargMsft commented 1 week ago

I started some exploration of this to better understand where events happen but there have been some changes around lifecycles since starting this that will need a bit of rework. Additionally, there is still question around what to record where. For example, Migrations are something that are queued up and sent in batches so they don't map one-to-one with a grain.

https://github.com/rkargMsft/orleans/tree/lifecycle_activity

ReubenBond commented 1 week ago

We should fix this. We could add explicit support to ActivationData.cs in lieu of a general-purpose activation filter.