abdeladim-s
commented
1 year ago Hi @ParisNeo,
I didn't test with pyllamacpp but I tried to compile pywhispercpp a while ago and it wasn't successful as well. I don't even remember what was the problem but it was complicated.
My thoughts are maybe ggml is not optimized to work with ARM architecture.
Another thing is even if we succeeded in compiling the code, the cpu does not have any AVX acceleration. so it will be really really slow I guess. so I don't think it is worth it, that's why I didn't try again.
Hi @abdeladim-s . I am trying to test lollms on a raspberry pi 4 with orca-mini-3b and can't manage to compile your code without errors.
Did you test building the wheels for raspberry pi? it would be cool to have a wheel compatible with raspberry pi 4 because building burns my rasp and takes lonjg time and it fails after more than an hour of compilation. With A raspberry pi 4 we can fuse the whispercpp and the lollms to have a 100% local assistant.
I need the wheel to be built for python 3.10 if possible. This could be a real good challenge and I think your binding is small enough to be used. The other bindings are to complicated.
what do you think?