search

DenisovAV / flutter_gemma

The Flutter plugin allows running the Gemma AI model locally on a device from a Flutter application.
MIT License
23 stars 5 forks source link

Size concern #4

Open flutter-painter opened 1 month ago

flutter-painter commented 1 month ago

Thank you for bringing AI to flutter in such a simple way, I am really looking forward to this lib and the opportunities it open.

Could you please enlighten me on how much weighs Gemma 2gb model ? I presume this lib is quite light so I am mostly concerned about the model overall weight which, even if downloaded from the app and not embedded, could limit the use on certain device.

Are there also any recommendation regarding device specs to run this efficiently ?

flutter-painter commented 1 month ago

gemma-2b-it-cpu-int4.bin == 1.25 GB (1,346,559,040 bytes)

echogit commented 1 month ago

is that the smallest size? Can it be quantized further?