DataDog / datadog-lambda-python

The Datadog AWS Lambda Layer for Python
https://docs.datadoghq.com/integrations/amazon_lambda/#installing-and-using-the-datadog-layer
Apache License 2.0
84 stars 45 forks source link

[MLOB-1009] Auto-Instrument LLM Observability #500

Closed sabrenner closed 3 months ago

sabrenner commented 3 months ago

What does this PR do?

Adds auto-instrumentation to the serverless layer for LLM Observability. Now, in order to enable LLM Observability for auto-instrumented integrations (OpenAI, LangChain, Bedrock, and Anthropic), set DD_LLMOBS_ENABLED=1, along with:

DD_LLMOBS_ML_APP=<YOUR_ML_APP>
DD_API_KEY=<YOUR_DATADOG_API_KEY>
DD_SITE=<YOUR_DATADOG_SITE>

LLM Observability documentation is available for full instrumentation instructions.

Motivation

Enable easy onboarding and auto-instrumentation experience with LLM Observability in a serverless environment.

Testing Guidelines

Built the layer and tested against a lambda handler. Verified that

  1. LLM traces were submitted per invocation of the function
  2. APM traces were still being sent as well

Types of Changes

Check all that apply