search

snowflakedb / snowpark-python

Snowflake Snowpark Python API
Apache License 2.0
263 stars 108 forks source link

SNOW-1462014: Ability to disable OTel span creation even if opentelemetry is installed #1730

Open mikeldking opened 4 months ago

mikeldking commented 4 months ago

What is the current behavior?

If opentelemetry is installed, spans are automatically emitted via https://github.com/snowflakedb/snowpark-python/blob/29b3bba9a7bd13206221b6fc6ad9ed0fae853fe0/src/snowflake/snowpark/_internal/open_telemetry.py

What is the desired behavior?

Add the ability to disable span creation even if opentelemetry is installed so as to keep the trace hierarchy simpler.

How would this improve snowflake-snowpark-python?

Make it more configurable and more composable with other OTel instrumentations

References, Other Background

OTel gives you the ability to turn off different instrumentations like so: https://github.com/snowflakedb/snowpark-python/blob/29b3bba9a7bd13206221b6fc6ad9ed0fae853fe0/src/snowflake/snowpark/_internal/open_telemetry.py

sfc-gh-yuwang commented 4 months ago

Hi, do you just want to be able to disable all snowpark emit spans, or you also want to have control on what function should not emit span? For example, do you want snowpark to emit span on .collect() while not emitting span on .show() ?

mikeldking commented 4 months ago

@sfc-gh-yuwang thanks so much for responding! For now I think it's an overall switch-off (@trevor-laviale-arize might be able to chime in here)

trevor-laviale-arize commented 4 months ago

Yeah I think just the ability to disable in general is good for now :)

sfc-gh-yuwang commented 4 months ago

Got it, will do this recently, I am thinking about having a environment variable to turn on/off snowpark opentelemetry

trevor-laviale-arize commented 4 months ago

Amazing, thanks so much!

mikeldking commented 4 months ago

Thanks @sfc-gh-yuwang - env var makes a ton of sense

sfc-gh-yuwang commented 2 months ago

I want to do a quick update on this issue, we are now discussing with other teams who potentially could have the issue about a uniform solution across snowflake, we should have this solved within this month

sfc-gh-jfreeberg commented 1 month ago

@mikeldking , @trevor-laviale-arize -- Have you tried setting this env var? OTEL_PYTHON_DISABLED_INSTRUMENTATIONS ?

https://opentelemetry.io/docs/zero-code/python/configuration/#disabling-specific-instrumentations

sfc-gh-yuwang commented 1 month ago

I also find that opentelemetry has recently added a feature of trace config: https://github.com/open-telemetry/opentelemetry-specification/pull/3877 , can you also try it?