Open lukestanley opened 1 month ago
I was trying to figure out what regressed with the low level API, and I saw that there was a bit more recent batching notebook. But the Batching notebook itself was broken so I fixed it.
I had to diff changes in llama_cpp.py to find the API changes.
To use GitHub Codespaces after making a Codespace from the branch, I had to:
git submodule update --init --recursive
pip install --upgrade pip
pip install -e .
wget "https://huggingface.co/TheBloke/Mistral-7B-v0.1-GGUF/resolve/main/mistral-7b-v0.1.Q2_K.gguf?download=true"
mv "mistral-7b-v0.1.Q2_K.gguf?download=true" "mistral-7b-v0.1.Q2_K.gguf"
When Codespace offers to install the Jupyter extension, I accepted it. I am then able to run the whole notebook.