llama-cpp-python updates fast, so there would be a constant maintenance grind on having to constantly update the version
gives users a good enough out-of-the-box experience with the latest models
except on LLaMA-2, which at the moment needs -gqa 8 on the arguments to load, but that's temporary, and is a better path to take compared to "wait i now need to update the llama-cpp-python, that's not in the manual" from a user perspective
there isn't anything else on the environment other than llama-cpp-python, no conflicts would be prevented from pinning down versions
all of the reasons would "fall apart" once a v1 is declared of llama-cpp-python, but I have no idea if that will happen or not.
reasonings:
llama-cpp-python
updates fast, so there would be a constant maintenance grind on having to constantly update the version-gqa 8
on the arguments to load, but that's temporary, and is a better path to take compared to "wait i now need to update the llama-cpp-python, that's not in the manual" from a user perspectivellama-cpp-python
, no conflicts would be prevented from pinning down versionsall of the reasons would "fall apart" once a v1 is declared of llama-cpp-python, but I have no idea if that will happen or not.