search

lyogavin / airllm

AirLLM 70B inference with single 4GB GPU
Apache License 2.0
5.23k stars 420 forks source link

Is there any practical usecase of this project ? #194

Open Greatz08 opened 2 weeks ago

Greatz08 commented 2 weeks ago

Asking this because you and i both know very well that even if it is possible to run big models with lower vram usage still it is not practical to use the models like that because your brain cells will get damaged with that much slow inference speed so is there any practical usecase of this project where we can get any better advantage when running small models (quantized models) or anything else which might be beneficial for us ?

khassad commented 2 weeks ago

Is there any practical reason to waste time asking such question ? If there were no usecase then people would not use it and the project would die, but this is quite the opposite !

This is great work, valued by at least 5k people, so please remove any assumption about others in your sentence, and keep both your brain behaviour and belief yours.

Greatz08 commented 2 weeks ago

@khassad

Is there any practical reason to waste time asking such question ?

yes Mister there is practical reason to ask for saving atleast others time

If there were no usecase then people would not use it and the project would die, but this is quite the opposite !

I Haven't seen its 'PRACTICAL' usecase anywhere Mister so rather than assuming people are using it and project is growing which is not the case by any mean, People gave it a star including myself because it did show a unique way (for me and maybe for others atleast to run big models which is still nice but it is by no mean practical to use big models like that with extremelyyyy delayed response so before speaking find proof's which can support your statement and share it here rather than 'assuming' things. I by no mean am disrespecting their work , i am just asking if there is any practical use of it because that's what matters the most at the end of the day. Till now i have seen some youtube channels and reddit section which have same opinions on this Mister so rather than becoming blind fanboi try to use your brain (if you have) and think practically and understand what i said rather than writing your BS comment here