search

huggingface / candle

Minimalist ML framework for Rust
Apache License 2.0
15.18k stars 887 forks source link

gemma2:2b example #2392

Open evgenyigumnov opened 1 month ago

evgenyigumnov commented 1 month ago

Hello Sir and Madam,

Do you plan to add the gemma2:2b example?

This model is very small and smart.

Best regards, Evgeny

LaurentMazare commented 1 month ago

This should now be available and the default when using the gemma example!

evgenyigumnov commented 2 weeks ago

Thanks a lot! But what about quntinezed mode ? https://github.com/huggingface/candle/issues/2450

gemma2:2b quntinezed version have 1.5Gb size! This is amaziong and high quelity!