search

argilla-io / argilla

Argilla is a collaboration tool for AI engineers and domain experts to build high-quality datasets
https://docs.argilla.io
Apache License 2.0
3.79k stars 354 forks source link

Active listeners for Feedback Dataset #4658

Open vmehta-cohere opened 5 months ago

vmehta-cohere commented 5 months ago

Is your feature request related to a problem? Please describe. For feedback datasets, which carry the ability to carry complex labeling tasks, manual interactions to trigger training/evaluation can be a bottleneck for high frequency models.

Describe the solution you'd like Active listeners for datasets in Argilla to trigger functions on submit, maybe an ability to ex. trigger on submitted 50/60 examples or 60/60 examples etc.

dosubot[bot] commented 5 months ago

It seems like you've got this under control, if you want help or have specific questions, let me know what I can do for you!

#### About Dosu This response is meant to be useful and save you time. It is **not meant to be a precise solution**, but rather a starting point for your own research. Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant. If you want to continue the conversation, start your reply with @dosu-bot.
fozziethebeat commented 4 months ago

I also just encountered this problem and found it very surprising to find out that listeners only work with pre-built template datasets.

From what I could discover it's because /api/datasets endpoint only returns these pre-built template datasets and deep down listeners only check that endpoint.

github-actions[bot] commented 1 month ago

This issue is stale because it has been open for 90 days with no activity.

github-actions[bot] commented 3 weeks ago

This issue was closed because it has been inactive for 30 days since being marked as stale.

nataliaElv commented 1 week ago

Reopening, as we might still want to tackle this one.