search

withcatai / node-llama-cpp

Run AI models locally on your machine with node.js bindings for llama.cpp. Force a JSON schema on the model output on the generation level
https://withcatai.github.io/node-llama-cpp/
MIT License
736 stars 63 forks source link

Cannot instantiate new LlamaModel bc class constructor was changed to private in beta #200

Closed convertsee-dev closed 2 months ago

convertsee-dev commented 2 months ago

Issue description

The constructor was changed to private so we cannot use this in TypeScript.

Expected Behavior

I should be able to follow docs:

const model = new LlamaModel({
    modelPath: path.join(__dirname, "models", "codellama-13b.Q3_K_M.gguf")
});

Actual Behavior

I get an error: Constructor of class 'LlamaModel' is private and only accessible within the class declaration.ts(2673)

Steps to reproduce

npm i node-llama-cpp@beta and attempt to follow docs to instantiate a new LlamaModel:

const model = new LlamaModel({
    modelPath: path.join(__dirname, "models", "codellama-13b.Q3_K_M.gguf")
});

My Environment

Dependency Version
Operating System Windows
CPU Intel i9 / Apple M1
Node.js version 20
Typescript version 5.2
node-llama-cpp version beta

Additional Context

No response

Relevant Features Used

Are you willing to resolve this issue by submitting a Pull Request?

No, I don’t have the time and I’m okay to wait for the community / maintainers to resolve this issue.

brandon-e2e commented 2 months ago

This is not a bug. The beta docs specifically state how to create a model: https://github.com/withcatai/node-llama-cpp/pull/105

const llama = await getLlama();
const model = await llama.loadModel({
    modelPath: path.join(__dirname, "models", "dolphin-2.1-mistral-7b.Q4_K_M.gguf")
});
convertsee-dev commented 2 months ago

Thank you for your response, really appreciate it.

On Sat, Apr 13, 2024 at 11:28 AM 'Gilad S.' via Developers < @.***> wrote:

Closed #200 https://github.com/withcatai/node-llama-cpp/issues/200 as completed.

— Reply to this email directly, view it on GitHub https://github.com/withcatai/node-llama-cpp/issues/200#event-12458227632, or unsubscribe https://github.com/notifications/unsubscribe-auth/BHMLWNZNKSYE34NFQHLEA6LY5FFKLAVCNFSM6AAAAABGEPNWQGVHI2DSMVQWIX3LMV45UABCJFZXG5LFIV3GK3TUJZXXI2LGNFRWC5DJN5XDWMJSGQ2TQMRSG43DGMQ . You are receiving this because you authored the thread.Message ID: @.***>

giladgd commented 2 months ago

@convertsee-dev The solution was given by @brandon-e2e in this ticket already, so I closed it as completed