argilla-io / argilla

Argilla is a collaboration platform for AI engineers and domain experts that require high-quality outputs, full data ownership, and overall efficiency.
https://docs.argilla.io/en/latest/
3.63k stars 339 forks source link

[ENHANCEMENT] optimise SDK log method and support mapping incoming columns to multiple dataset attributes #5107

Open burtenshaw opened 5 days ago

burtenshaw commented 5 days ago

This PR supports mapping incoming columns/keys to dataset attributes in these two ways:

This PR also optimises the log method so that it takes less time and is easier to work with:

Screenshot 2024-06-26 at 09 09 11

How Has This Been Tested

Checklist

nataliaElv commented 5 days ago

I'd rather have the records ingested and pushed in batches and have an easy way to identify those that threw an error, fix them and try to import those again. Otherwise it can take ages until I see any records in my dataset. Captura de pantalla 2024-06-26 a las 9 23 42