encord-team / text-to-image-eval

Evaluate custom and HuggingFace text-to-image/zero-shot-image-classification models like CLIP, SigLIP, DFN5B, and EVA-CLIP. Metrics include Zero-shot accuracy, Linear Probe, Image retrieval, and KNN accuracy.
https://encord.com
Apache License 2.0
35 stars 1 forks source link

fix: empty dataset splits #73

Closed eloy-encord closed 5 months ago

eloy-encord commented 5 months ago

If a dataset is small enough or the values for the train/val/test splits is very small, it may be possible that no piece of data is assigned to that split. This change enforces that no split is empty, while it may have size 1 and thus not really useful, but it won't break the execution and will leave the responsibility to the user to take action in order to fix the disparity caused by his/her actions or the very small dataset size.