Closed saghul closed 10 months ago
This trick works on macOS, as in, the generated libllama does have metal if I build it with pip
with the right env vars. Pl test it on Monday @aaronkvanmeerten / @rpurdel this might save us the libllama build step going forward.
is this still needed? @saghul
I'm going to try and make it work next week, yes.
I confirmed it works on macOS, installing with pip and the right env vars that is, so I want to do the same for Linux / Docker.
I also realized we can greatly simplify the Dockerfile now if that works, so yeah, still relevant.
When installing dependencies with Poetry the env vars that llama-cpp requires seem to not be taken into account, unlike when using pip.
In order to fix this, export the Poetry dependencies into pip format and use that to install them.