yoohj0416 / NLSearch

Apache License 2.0
0 stars 0 forks source link

Tag Classification only with Image #7

Open yoohj0416 opened 8 months ago

yoohj0416 commented 8 months ago

To-Do

This is a set of steps for tag classification only with image.

In the tag classification test with one image experiment, classification is performed using only one image per video. The image is selected by the Berkeley DeepDrive team and provided by BDD100K images dataset.

In the tag classification with multiple images experiment, multiple images are extracted from each video and their embedding vectors are calculated. The vectors are then combined (i.e. mean) to perform classification.

Updated To-Do

yoohj0416 commented 8 months ago

Tag classification with one image

We did tag classification with one image per video. We used validation images of BDD100K (they don't provide test labels) and compared with accuracy in the paper. DLA-34 represents results from the Berkeley Deep Drive team (not sure either they used validation set or test set).

Weather classification

Scene classification

yoohj0416 commented 7 months ago

Video & image example of scene classification

Analyze what ImageBind classifies residential images to

yoohj0416 commented 7 months ago

Just some quick notes for idea to utilize mutimodal data..