search

withcatai / node-llama-cpp

Run AI models locally on your machine with node.js bindings for llama.cpp. Force a JSON schema on the model output on the generation level
https://withcatai.github.io/node-llama-cpp/
MIT License
760 stars 65 forks source link

Add llama.cpp build number as info #141

Closed nathanlesage closed 6 months ago

nathanlesage commented 6 months ago

Feature Description

The website mentions that each release ships with "the most recent llama.cpp release at the time", offering the possibility to download newer releases as appropriate (with the corresponding caveats).

However, I could not find which release that was. I now manually figured out that release b1618 is the last one that works with the beta, as afterwards there was a breaking API change, but I had to test out quite a few to find "the newest".

It would be great to store this information somewhere – either on the repository (releases section would suffice) or in the distributable.

The Solution

Add the build number of llama.cpp that was used to create a node-llama-cpp release to some public place.

Considered Alternatives

Manually finding out the most recent working build number. None of the files, and not the releases section contain this number.

Additional Context

It doesn't need to be pretty, so there's no need to make it complicated. It's just good if the number is recorded somewhere!

Related Features to This Feature Request

Are you willing to resolve this issue by submitting a Pull Request?

No, I don’t have the time and I’m okay to wait for the community / maintainers to resolve this issue.

giladgd commented 6 months ago

@nathanlesage You can use getReleaseInfo() to find the release that is used in your project.

The stable version updates frequently enough to try to be compatible with the latest version of llama.cpp to make it always possible to download the latest release. However, this is not the case for the beta version.

I've made sure that llama.cpp's release will be mentioned in future GitHub releases.

github-actions[bot] commented 6 months ago

:tada: This issue has been resolved in version 3.0.0-beta.2 :tada:

The release is available on:

Your semantic-release bot :package::rocket:

nathanlesage commented 6 months ago

Amazing, thank you!