search

sobelio / llm-chain

`llm-chain` is a powerful rust crate for building chains in large language models allowing you to summarise text and complete complex tasks
https://llm-chain.xyz
MIT License
1.32k stars 130 forks source link

Add gemma.cpp bindings / llm-chain-gemma. #281

Closed jmuk closed 6 months ago

jmuk commented 6 months ago

Hi, can you run the CI tasks again? I think I've addressed them (although I'm not sure if it fixes Windows build as I don't have windows environment)

williamhogman commented 6 months ago

Thanks for that :) seems the CI itself passe

williamhogman commented 6 months ago

Other than windows

jmuk commented 6 months ago

Sigh, sorry about that. I made one more change, I hope it fixes the windows build...

williamhogman commented 6 months ago

Still some weird issue with windows @jmuk

jmuk commented 6 months ago

Looks like an error on compiling C++ code, possibly due to a macro defined in a standard Windows header file? (cf. https://stackoverflow.com/questions/5004858/why-is-stdmin-failing-when-windows-h-is-included). I checked gemma.cpp repository but seemingly their CI includes windows and it looks fine. I'll check how it's resolved there and ports their resolution here -- which is (I hope) simply updating gemme.cpp version referred in the .gitmodule file.

jmuk commented 6 months ago

I checked gemma.cpp and recent version has NOMINMAX macro for windows, which should resolve the build issue. I updated the version so I think it won't get trapped by that error.

However the gemma.cpp interface has changed very significantly. I rebuilt bindings.cc and others to catch up with that and confirmed it's built and works successfully on my local macbook. Sorry for the troubles, but please run the CI checks again!

jmuk commented 6 months ago

Sigh, I just found that gemma.cpp does not support MSBuild / MSVC, it only supports clang-cl for now. I gave up on Windows build; with the new patch, it should pass the build on Windows and CI should pass, but on Windows it will raise ExecutorCreationError on Windows.

Please run the CI again!

jmuk commented 6 months ago

The CI seems retried yesterday but that was on an older revision? Could someone run the CI on the latest one https://github.com/sobelio/llm-chain/pull/281/commits/42932846f87586cfdfd1aea7cdf63ca1d662d8a9

williamhogman commented 6 months ago

All good. Seems like it is ready to merge. Let me know you want me to go ahead and package it

jmuk commented 6 months ago

Yay! Please go ahead anytime you can.