Add tracing for incoming requests

[charleskorn]

This PR adds support for tracing incoming HTTP requests for the two webhooks.

It produces traces like the following, with nested spans for each of the HTTP calls:

I've used OpenTracing here over OpenTelemetry in the interests of consistency with Mimir.

I don't love how invasive these changes were - the default OpenTracing instrumentation for net/http requires calling nethttp.TraceRequest() for each outgoing HTTP call, but any other option would require either creating our own OpenTracing instrumentation for net/http, or using OpenTelemetry, which would break with the existing pattern of Mimir. I'm very open to other ideas or suggestions.

I don't believe we can't add tracing for Kubernetes API calls made by the informers used in the reconciliation loop, as these run outside the context of a parent span, so I haven't added tracing to the reconciliation loop beyond a single span.

[andyasp]

Apologies @charleskorn, I merged too early. I reverted the commit and restored the branch so you could respond to the above in a reopened PR.

[charleskorn]

Apologies @charleskorn, I merged too early. I reverted the commit and restored the branch so you could respond to the above in a reopened PR.

No worries, I'll continue in #114.