Canner / WrenAI

🚀 Open-source SQL AI Agent for Text-to-SQL. Make Text2SQL Easy! 🙌
https://getwren.ai/oss
GNU Affero General Public License v3.0
1.73k stars 155 forks source link

feat(wren-ai-service): add metadata into Langfuse trace #566

Closed paopa closed 1 month ago

paopa commented 1 month ago

This PR enhances Langfuse tracing by adding metadata, including the generation model, embedding model, service version, request model deployment ID, project ID, and thread ID. It ensures that all metadata is collected and flushed to Langfuse before the service shuts down, preventing any data loss.

Performance Estimation

I believe enabling Langfuse does not impact performance. The following tables estimate the time cost of handling three questions in both enabled and disabled states.

What is the total value of payments made by customers from each state?

Langfuse Status Round AskService.ask Elapsed Time AskDetailsService.ask_details Elapsed Time
Enable Round 1 10.5156 seconds 6.5332 seconds
Enable Round 2 8.3765 seconds 4.9982 seconds
Enable Round 3 9.2841 seconds 5.2404 seconds
Disable Round 1 8.0913 seconds 6.3799 seconds
Disable Round 2 8.7771 seconds 4.3174 seconds
Disable Round 3 10.7809 seconds 5.3201 seconds

What are the top 3 value for orders placed by customers in each city?

Langfuse Status Round AskService.ask Elapsed Time AskDetailsService.ask_details Elapsed Time
Enable Round 1 6.0459 seconds 4.5500 seconds
Enable Round 2 6.6314 seconds 3.4410 seconds
Enable Round 3 6.9123 seconds 4.1723 seconds
Disable Round 1 5.1141 seconds 3.4092 seconds
Disable Round 2 5.5271 seconds 4.3957 seconds
Disable Round 3 4.9790 seconds 5.1734 seconds

What is the average score of reviews submitted for orders placed by customers in each city?

Langfuse Status Round AskService.ask Elapsed Time AskDetailsService.ask_details Elapsed Time
Enable Round 1 5.9949 seconds 3.8208 seconds
Enable Round 2 5.1165 seconds 6.0028 seconds
Enable Round 3 4.0853 seconds 4.0925 seconds
Disable Round 1 4.1101 seconds 3.6076 seconds
Disable Round 2 7.5454 seconds 6.5127 seconds
Disable Round 3 5.8752 seconds 7.4653 seconds

Screenshots

image image image

See also