krrishdholakia
commented
1 month ago - will need user to tell us their argilla dataset name
- will need user to map our payload to their dataset fields
- need to allow user to set a sampling rate (not all logs should be sent to argilla for annotating)
The Feature
Allow user to sample logs and log them to https://argilla.io/
Also add a hook for allowing user to first run the request through an llm call (llm-as-a-judge)
Motivation, pitch
get better for mlops/ds teams
Twitter / LinkedIn details
No response