Can I use Xenova/Phi-3-mini-4k-instruct model server side?

xenova / transformers.js

State-of-the-art Machine Learning for the web. Run 🤗 Transformers directly in your browser, with no need for a server!

https://huggingface.co/docs/transformers.js

Apache License 2.0

9.71k stars 571 forks source link

Can I use Xenova/Phi-3-mini-4k-instruct model server side? #789

Closed StepanKukharskiy closed 3 weeks ago

StepanKukharskiy commented 3 weeks ago

Question

Hey there! I’m trying to run Xenova/Phi-3-mini-4k-instruct model using transformers.js 2.17.2 on the server in my Node.js project, but I get an error saying that Phi-3 is not supported. Can I make it work somehow? Any ideas appreciated

xenova commented 3 weeks ago

Phi-3 models are only supported in the v3 dev branch: https://github.com/xenova/transformers.js/pull/545. You can find example code here. However, we haven't yet finished adding support for server-side usage, since that requires extra consideration for the external data format.

StepanKukharskiy commented 3 weeks ago

Got it! Thanks a lot!