WIP: Opentelemetry

[TheAngryByrd]

I used the Aspire dashboard. You can run it via docker:

docker run --rm -it -p 18888:18888 -p 4317:18889 -d --name aspire-dashboard mcr.microsoft.com/dotnet/nightly/aspire-dashboard:8.0.0-preview.4

Then you can run the Expecto.Tests project. It should export OpenTelemetry traces to Aspire.

Example of traces for tests showing up:

Clicking View on one of them with a red dot (red dot indicates there were exceptions/errors):

Clicking on the top span, you can see details about its run.

Clicking on one of the child spans you can see details such as Exception data.

I didn't show it here but you can also export logs that are associated with a test trace.

[farlee2121]

I'm still thinking over generalization approaches. I haven't found a clear nice answer yet.

Have you considered how the current span approach would behave with stress testing?

[TheAngryByrd]

Oh one of the things missing too is how to integrate with dotnet test since we can't call addOpenTelemetry_SpanPerTest, we'd probably need some attribute that would do the call on our behalf when calling into testFromAssembly

[farlee2121]

Good point. That's a problem for shuffle and filter too. I'm not sure why no discovery methods (like testFromThisAssembly) are officially exposed. They are available, if unofficially, under the Expecto.Impl module.

Seems like exposing such a method would be desirable for library extension.

[farlee2121]

Upon review, I think I got wrap up in my own line of thought and didn't interpret your statement correctly.

I'm not sure I see the incompatibility with dotnet test. Doesn't YoloDev.Expecto.TestSdk respect whatever is configured in main?

[TheAngryByrd]

As far as I know, dotnet test doesn't care whats in your main/entrypoint. Yolodev uses Expecto.Impl.testFromAssembly to find tests.

I suppose you can call addOpenTelemetry_SpanPerTest for every testlist that uses [<Tests>]. There might be some tricks required to get ActivitySource to not be null, I'll have to look later.

Specifically the tricks that @baronfel has in https://github.com/baronfel/otel-startup-hook

[farlee2121]

Interesting. I hadn't thought of such a limitation.

Tests names can be listed via dotnet run, but that doesn't provide fully-hydrated Test records. I don't see a clear solution to this either, since I'm not sure anonymous functions could be effectively transferred out-of-process.

[TheAngryByrd]

So creating the TraceProvider statically:

do
    let provider = traceProvider()
    AppDomain.CurrentDomain.ProcessExit.Add(fun _ -> provider.Dispose())

And addOpenTelemetry_SpanPerTest has to be added per [<Tests>]. Which isn't too bad but might get annoying for people that have lots of [<Tests>] instead of a few.

After that, this does work for dotnet test