search

visual-layer / fastdup

fastdup is a powerful free tool designed to rapidly extract valuable insights from your image & video datasets. Assisting you to increase your dataset images & labels quality and reduce your data operations costs at an unparalleled scale.
Other
1.52k stars 74 forks source link

Adding Dataset Captioning Function & Notebook #272

Closed guy-singer closed 9 months ago

guy-singer commented 10 months ago

These changes introduce the caption() function into the fastdup controller, allowing the user to caption their entire dataset or a subset of the dataset.

Additionally, an example notebook is added into the examples directory, demonstrating to the user how to conduct captioning.

review-notebook-app[bot] commented 10 months ago

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.

Powered by ReviewNB

dnth commented 10 months ago

Hi @guy-singer thanks for adding a notebook to showcase the captions functionality. The notebook looks good as it is. I have some comments below. It would be great if we had these in the notebook. If not, then it's okay too :)

Typically in a notebook we will include a Download Dataset section where we provide a tiny dataset to minimally reproduce the results in the notebook. This lowers the barrier for users to run this notebook without them having to think of a dataset to use. From a user experience perspective, this increases the chance of them running the notebook. I'd typically drop in a sentence or two to encourage users to run it on their own dataset.

There are readily available datasets in our examples like Oxford Pets, Imagenette, Mini COCO, etc. But if these are too large, then feel free to host a tiny dataset on cloud storage.

Also I'd add a sentence at the top of the notebook to tell users why they need the captioning capability in their workflow. Highlight benefits or include a motivation for the problem at hand.

dnth commented 10 months ago

I feel like we can unlock more capabilities of the VQA model by letting users specify their prompts. Currently, we are only prompting the model to identify indoor/outdoor which only applies to a niche problem.

guy-singer commented 9 months ago

Thanks Dickson, I have rewritten the notebook using the mini coco dataset, and added the description at the top as you have suggested. I will commit an updated notebook shortly.

guy-singer commented 9 months ago

I feel like we can unlock more capabilities of the VQA model by letting users specify their prompts. Currently, we are only prompting the model to identify indoor/outdoor which only applies to a niche problem.

Added capability to let users specify their VQA prompt. committing new notebook shortly with VQA example. Thanks Dickson.

guy-singer commented 9 months ago

all fixes and updates have been made, according to suggestions from @amiralush and @dnth

guy-singer commented 9 months ago

Notebook edited with clustering example as per @amiralush 's request

dnth commented 9 months ago

@guy-singer Not sure if I'm missing something, but why are we zero-ing the distance scores? Since it's an outlier report wouldn't it be useful to know if the image is far apart from all other images in the dataset?

image

amiralush commented 9 months ago

@guy are you ready to deploy this?

guy-singer commented 9 months ago

@guy are you ready to deploy this?

@amiralush yes, it is ready and working well