Assess need to add custom application insights metrics to critical operations

[elsand]

Introduction

In order to track overall performance in Dialogporten, we need to track several metrics

Description

There are several critical operations that we need to measure (time elapsed):

• Initial request handling including authentication
• For both list and detail views
  • Building authorization requests (up to and including serialization of HTTP request)
  • Performing authorization requests (up to excluding deserialization of HTTP response)
  • Building Entity Framework requests
  • Performing database queries

Also, time elapsed on refreshing other dependencies should be tracked

• Resource Registry (both resource list and subjectresources)
• Events

We also need to measure

• Number of Fusion cache hits/misses per cache, for both memory and distributed cache (Redis)

Implementation

Much of this might be possible to implement more or less without any code, given that we get correlations properly set up in Application Insights. We also need to make sure Insights sampling settings are appropiate.

We should also probably look into using https://github.com/serilog-tracing/serilog-tracing

### Tasks
- [ ] Implementation tasks are added here
- [ ] Prepare documentation (if relevant - either update working document, or add a new file in `docs`)
- [ ] Add e2e-test (if relevant)

### Threat modelling
- [ ] Does this change introduce any potential security issues?

Acceptance criteria

GIVEN ... WHEN .... THEN ...

GIVEN ... WHEN .... THEN ...

[arealmaas]

When we re-add instrumentation for Redis, we have most of these traces already, no custom tracing needed.

Building Entity Framework requests

Maybe we also could consider adding the efcore Open Telemetry instrumentation. We would probably get some good metrics for Entity Framework then https://github.com/open-telemetry/opentelemetry-dotnet-contrib/blob/main/src/OpenTelemetry.Instrumentation.EntityFrameworkCore/README.md