[Tracing] Allow Client libraries to configure schema URL

[lmolkova]

See https://github.com/Azure/azure-sdk-for-go/pull/23268#discussion_r1702424904 - different SDKs may emit telemetry of slightly different versions and we can't keep them in sync (e.g. servicebus and cosmosdb SDKs are owned by different teams, follow different semantic conventions, and prioritize telemetry differently, i.e. they need to be able to set different schema urls.

We currently hardcode and use schema url here - one for everyone:

https://github.com/Azure/azure-sdk-for-go/blob/969cfcd7a36018cab9461bc768bdd0c8f1ff4e00/sdk/tracing/azotel/otel.go#L31

This is blocker to stabilize tracing plugin (and tracing for anything beyond http).

[kjg]

@lmolkova It seems like current azcore is responsible for tracing the HTTP spans and Client libraries can add additional tracing on top of that. Would it make sense for the Client Libraries to use a separate Tracer than Azcore? That way Azcore could have one schema URL and client libraries could their own?

[lmolkova]

@kjg absolutely! One of the important (from otel perspective) this is that each tracer has a proper instrumentation scope (library name) so that it's clear where the telemetry is coming from and also it's possible to configure it on per-scope level. I.e. we should create a tracer per client.

That's what we do in Java:

• each library creates its own tracer instance with library name, version and, since recently, the schema url it follows. E.g. here Schema URL was added recently - https://github.com/Azure/azure-sdk-for-java/pull/41789. All these are library options - users don't get to change them.
• It also includes user-facing (if any) tracing configuration provided by users on the client level.

[kjg]

Thanks, this is so useful. I see here where the java version of azcore creates its own named tracer. I'll see if I can give a shot updating this go version to do the same!