jitsi / skynet

AI core services for Jitsi
Apache License 2.0
24 stars 7 forks source link

docker: fix libllama build #23

Closed saghul closed 10 months ago

saghul commented 11 months ago

When installing dependencies with Poetry the env vars that llama-cpp requires seem to not be taken into account, unlike when using pip.

In order to fix this, export the Poetry dependencies into pip format and use that to install them.

saghul commented 11 months ago

This trick works on macOS, as in, the generated libllama does have metal if I build it with pip with the right env vars. Pl test it on Monday @aaronkvanmeerten / @rpurdel this might save us the libllama build step going forward.

quitrk commented 10 months ago

is this still needed? @saghul

saghul commented 10 months ago

I'm going to try and make it work next week, yes.

I confirmed it works on macOS, installing with pip and the right env vars that is, so I want to do the same for Linux / Docker.

I also realized we can greatly simplify the Dockerfile now if that works, so yeah, still relevant.