Are there any examples on how to use paligemma model with transformer.js

xenova / transformers.js

State-of-the-art Machine Learning for the web. Run 🤗 Transformers directly in your browser, with no need for a server!

https://huggingface.co/docs/transformers.js

Apache License 2.0

9.87k stars 582 forks source link

Are there any examples on how to use paligemma model with transformer.js #825

Open alextanhongpin opened 4 days ago

alextanhongpin commented 4 days ago

Question

First of all, thanks for this amazing library!

So my questions is, I happened to see this model available on transformers.js: https://huggingface.co/Xenova/paligemma-3b-mix-224

But unfortunately I can't find any example on how to run the image-text-to-text pipeline. Are there are resources you could kindly point me to? Thanks in advance! 🙏🏻

xenova commented 2 days ago

Hi there 👋 PaliGemma is still a work-in-progress (not yet supported), and those weights were for testing. I couldn't get good output when I was testing, but I will hopefully return to it soon :)

alextanhongpin commented 2 days ago

Wokay, thanks for sharing 😄