Custom sensor as one of the multimodality?

facebookresearch / ImageBind

ImageBind One Embedding Space to Bind Them All

Other

8.07k stars 734 forks source link

Custom sensor as one of the multimodality? #104

Open ramkumarkoppu opened 7 months ago

ramkumarkoppu commented 7 months ago

Hi,

If I want to utilize one of the industrial sensors as this multi modality model, how to get started? for example, replacing audio modality by industrial (analog to digital conveted) sensor if I have corresponding images to augment.

StanLei52 commented 4 months ago

Hi there, I recommend you to checkout our project ViT-Lens. We open-sourced the training code and you may take a look at the audio part for your customized application.