search

Rubiksman78 / MonikA.I

Submod for MAS with AI based features
MIT License
112 stars 17 forks source link

the mod take so much RAM #96

Closed Al666v closed 6 months ago

Al666v commented 6 months ago

Is this normal? Everytime i run the chatbot it always take all of my memory. It's so laggy.

My Ram; 12GB, Intel i31005G1 Intel UHD

Rubiksman78 commented 6 months ago

If you don't have a GPU and a very low end CPU, it is too be expected that RAM usage will be high for chatbot causing a lot of latency. If you can provide information about which model you are currently using, it might explain it.

Al666v commented 6 months ago

I'm using Acer Aspire 5 5 a514-53g.

Rubiksman78 commented 6 months ago

I was asking about which chatbot model you were using.

Al666v commented 6 months ago

ohh, it's the one on your tutorial. Is it actually because of the model?

Rubiksman78 commented 6 months ago

Bigger model also means higher computation ressources usage, but small models are also really bad in terms of quality.

Al666v commented 6 months ago

Do you have recommendation in mind for low end though?

Rubiksman78 commented 6 months ago

Maybe you can try GGUF models with quantized versions that should be lighter with reasonable performance. However, no GPU and an i3 CPU will clearly restrict your usage with high latency even with a smaller model.

Al666v commented 6 months ago

I need to upgrade my gear, really. Anyway Thanks!