NVIDIA / NeMo-Guardrails

NeMo Guardrails is an open-source toolkit for easily adding programmable guardrails to LLM-based conversational systems.
Other
4.21k stars 400 forks source link

Feat/observability support #798

Closed Pouyanpi closed 1 month ago

Pouyanpi commented 1 month ago

Description

Implements a tracing module to assign a trace id to every incoming request and enable detailed tracing of every rail/action/LLM call that is triggered behind the scenes. Can export the traces to different providers like OpenTelemetry, etc.

Example Usage

Consider following config.yml

models:
  - type: main
    engine: openai
    model: gpt-3.5-turbo-instruct

tracing:
  enabled: true
  adapters:
    - name: OpenTelemetry
      service_name: "nemo_guardrails_service"
      exporter: "console"
      resource_attributes:
        env: "production"
    - name: FileSystem
      filepath: "./traces/traces.jsonl"

Now you can also use zipkin as expoerter.

To do so:

pip install opentelemetry-exporter-zipkin
docker run -d -p 9411:9411 openzipkin/zipkin

and set the exporter to zipkin in config.yml :

tracing:
  enabled: true
  adapters:
    - name: OpenTelemetry
      service_name: "nemo_guardrails_service"
      exporter: "zipkin"
      resource_attributes:
        env: "production"

Note:

Test Coverage

File Statements Missed Coverage Missing Lines
tests/test_tracing.py 49 1 98% 128
tests/test_tracing_adapters_filesystem.py 31 0 100% -
tests/test_tracing_adapters_opentelemetry.py 69 5 93% 146-152, 186

Checklist

Pouyanpi commented 1 month ago

@drazvan the changes are done. Currently tracing dependencies are added to pyproject.toml and using defered import like similar situations in the codebase would clutter the adapters.opentelmetry module. Dependency req is included in the documentation.