search

fixie-ai / ultravox

MIT License
618 stars 24 forks source link

Smaller model?! #20

Open chukfinley opened 3 weeks ago

chukfinley commented 3 weeks ago

are there plans for a smaller model? if it really is 32.25 GB for the voice model it cant be run on consumer GPU. Also how many parameters is the model? and how to run it?

chukfinley commented 3 weeks ago

also is the 32.25 GB model just voice or also llama3?

juberti commented 3 weeks ago

The 32 GB includes the Llama 3 finetune. With the quantization described in #8 you should be able to run on a consumer GPU.