voxel51 / voxelgpt

AI assistant that can query visual datasets, search the FiftyOne docs, and answer general computer vision questions
https://gpt.fiftyone.ai
Apache License 2.0
234 stars 17 forks source link

When to use text similarity search vs label filtering #29

Closed ehofesmann closed 1 year ago

ehofesmann commented 1 year ago

Currently, most examples when querying for a specific object/class type make use of match() or filter_labels(), to the point where even the you can use these and only these label classes: in prompts/dataset_view_generator_prefix.txt is ignored and classes that aren't listed but are in the prompt will be used.

This leads to empty views since filtering on a label class that doesn't exist won't result in anything interesting, even if a text similarity brain run exists which could give very interesting results.

For example, searching the coco_gpt_demo dataset with images with a camel in them should result in a sort by similarity stage, but tried a match instead.