search

traceloop / openllmetry

Open-source observability for your LLM application, based on OpenTelemetry
https://www.traceloop.com/openllmetry
Apache License 2.0
3.43k stars 674 forks source link

🚀 Feature: allow disabling prompt sending as an argument to Traceloop.init() #137

Open nirga opened 1 year ago

nirga commented 1 year ago

Which component is this feature for?

Traceloop SDK

🔖 Feature description

Add a setting called trace_content (which will default to true) and will control whether all the instrumentations are sending sensitive content (like prompts and completions). This should override the env var behavior (similar to other flags we have).

🎤 Why is this feature needed ?

More control on this sensitive feature

✌️ How do you aim to achieve this?

-

🔄️ Additional Information

No response

👀 Have you spent some time to check if this feature request has been raised before?

Are you willing to submit PR?

None

randolfuy01 commented 7 months ago

is this still available to work on?

nirga commented 7 months ago

Yes @randolfuy01 :)

randolfuy01 commented 7 months ago

perfect I'll look into it!

sanspareilsmyn commented 5 months ago

@nirga @randolfuy01 Hi there! Is it still WIP? If not, I would love to start looking around this issue:)

nirga commented 5 months ago

Not sure what's @randolfuy01's status, but there are other issues that you can also look at that are unassigned @sanspareilsmyn!

liamjdavis commented 3 months ago

@nirga Hi! Is this issue still open? If so, I would be happy to look at it :)

nirga commented 3 months ago

Sounds good @liamjdavis!

liamjdavis commented 3 months ago

@nirga To clarify in greater detail,

  1. The setting is the flag in the sdk's config init.py 'is_content_tracing_enabled

https://github.com/traceloop/openllmetry/blob/9ad18b7067f3036a13268b50366ff4207e32ffa2/packages/traceloop-sdk/traceloop/sdk/config/__init__.py#L8-L9

  1. Then the setting is called in the global sdk init.py

https://github.com/traceloop/openllmetry/blob/9ad18b7067f3036a13268b50366ff4207e32ffa2/packages/traceloop-sdk/traceloop/sdk/__init__.py#L79

  1. And I need to write the functionality that for each instrument,

https://github.com/traceloop/openllmetry/blob/9ad18b7067f3036a13268b50366ff4207e32ffa2/packages/traceloop-sdk/traceloop/sdk/__init__.py#L55

the sensitive content being shared for each individual instrument is controlled.

nirga commented 3 months ago

Yes!

liamjdavis commented 3 months ago

@nirga I've opened #1620 for this issue.