kyegomez / PALI

Democratization of "PaLI: A Jointly-Scaled Multilingual Language-Image Model"
https://discord.gg/GYbXvDGevY
MIT License
85 stars 8 forks source link

Image-Text similarity score #10

Open lokesh12345678910 opened 5 months ago

lokesh12345678910 commented 5 months ago

Is it possible to feed in an image and a text into pali to calculate an image-text similarity score? On the readme, I see the prompt is also being fed into the model

Upvote & Fund

Fund with Polar

kyegomez commented 5 months ago

Pretty sure this is how pali was trained in the beginning. But yes you can add this manually by forming the repo

lokesh12345678910 commented 5 months ago

Sorry, may you please provide me "off-the-shelf" code that uses PaLI to compute an image's feature vector and a text's feature vector?