search

withcatai / node-llama-cpp

Run AI models locally on your machine with node.js bindings for llama.cpp. Force a JSON schema on the model output on the generation level
https://withcatai.github.io/node-llama-cpp/
MIT License
760 stars 65 forks source link

Grammars folder not found #163

Closed ExcessionEleven closed 5 months ago

ExcessionEleven commented 5 months ago

Issue description

It seems that there are some paths problem

Expected Behavior

Working with grammars json

Actual Behavior

Paths problem

Steps to reproduce

The error is quite simple : Folder grammars are not accessible

My Environment

Dependency Version
Operating System Windows
CPU Intel i9 / Apple M1
Node.js version 21.x
Typescript version x.y.zzz
node-llama-cpp version lastBeta

Additional Context

node-llama-cpp/dist/utils/getGrammarsFolder.js:17 throw new Error("Grammars folder not found"); ^

Error: Grammars folder not found

Relevant Features Used

Are you willing to resolve this issue by submitting a Pull Request?

No, I don’t have the time and I’m okay to wait for the community / maintainers to resolve this issue.

ExcessionEleven commented 5 months ago

But the folder is there, i can see it

ExcessionEleven commented 5 months ago

Ok, i found the error, in the documentation i saw this let grammar = await LlamaGrammar.getFor("json") But you have to use this instead : let grammar = await LlamaGrammar.getFor(llama, "json")

giladgd commented 5 months ago

I haven't updated the documentation website for version 3 beta yet, since the API still changes frequently in the beta. The documentation will be updated with the stable release of version 3

ExcessionEleven commented 5 months ago

Sure, i know what it is, to update the doc of an R&D project. I have one myself XD. Still, your project was one of those that i've waited so long, i play with it as much as my free time allows it.