julep-ai / julep

A new DSL and server for AI agents and multi-step tasks
https://julep.ai
Apache License 2.0
2.67k stars 904 forks source link

Add Documentation for Monitoring #877

Open patcher9 opened 6 days ago

patcher9 commented 6 days ago

Hi team, I've updated the documentation to include both Julep's built-in Prometheus metrics and how to integrate with OpenLIT for better monitoring.

I am one of the maintainers of OpenLIT and we support Obserability for LLM apps built using Julep AI, It is all OpenTelemetry-native so the traces and metrics can be sent to any platform like Grafana or any OSS OTel tools

Although I know Julep has Prometheus metrics inbuilt like various countres, OpenLIT helps when the users are using LLM + VectorDBs + Other framework actions as the consolidated monitoring gives them a complete picture.

Figured the mention here might be useful for developers using Julep hence adding the reference in documentation page.

Let me know your thoughts on this and placement for this!

[!IMPORTANT] Adds docs/monitoring.md with instructions for setting up monitoring using Prometheus and OpenTelemetry with OpenLIT.

  • Documentation:
    • Adds docs/monitoring.md with a guide for setting up monitoring using Prometheus and Grafana.
    • Includes instructions for integrating OpenTelemetry with OpenLIT.
  • Prometheus Setup:
    • Steps to clone the Julep repository and set up Grafana and Prometheus using Docker Compose.
    • Lists available Prometheus metrics for monitoring Julep operations.
  • OpenTelemetry Integration:
    • Instructions for installing openlit and julep.
    • Steps to deploy OpenLIT stack and instrument Julep AI application.
    • Details on configuring OpenTelemetry endpoints and visualizing data.

This description was created by Ellipsis for 7ecb62a0cdbc52ecfa9aa0108e88ca6f126d29de. It will automatically update as commits are pushed.

creatorrr commented 1 day ago

Thanks for the PR, @patcher9 ! I'd love to explore integrating openlit with julep even more closely. We are Apache 2.0 licensed, does that work with you guys?

patcher9 commented 1 day ago

@creatorrr We love Apache 2.0! (OpenLIT is also completely Apache 2.0). And yup would love integrating with you folks!