Add end-user logging and tracing for drivers

[birschick-bq]

What feature or improvement would you like to see?

I'd like to suggest the following plan based on the use of OpenTelemetry

Create a proof-of-concept

1. Target a Go driver like Snowflake
2. Enable the "zero code" automatic instrumentation option

Enable output to

1. Local file logging
2. Console logging
3. Possibly to a remote tracing system

Determine if

1. automatic instrumentation is a viable option
2. if automatic instrumentation is too verbose, exposes private, or personally identifying information.
3. or if we need to use complete manual instrumentation or some combination
4. determine if cross-runtime contexts can be bridged and if so, how best to accomplish that.

[lidavidm]

I'd also be curious if you have ideas about how to bridge OpenTelemetry across runtimes; presumably C#/Python would like to be able to start a span that will properly be the parent span of anything the Go driver does. Possibly we can even do this mostly automatically inside the Go bridge code

[zeroshade]

The easiest way to bridge OpenTelemetry across the runtimes would be to pass metadata containing the headers to identify the context/span. Since they would be key/value pairs. I've done it with C++/Go before, perhaps through options on the connection/statement

[lidavidm]

Yeah - we would need some manual code (possibly, we can do this automatically in the Python driver manager too when OpenTelemetry is installed) and would just extract/inject the context across the FFI boundary.

[lidavidm]

If there's a better way though...

[birschick-bq]

I'd also be curious if you have ideas about how to bridge OpenTelemetry across runtimes; presumably C#/Python would like to be able to start a span that will properly be the parent span of anything the Go driver does. Possibly we can even do this mostly automatically inside the Go bridge code

I've added this to the list of goals for the POC.