fix(community) : Upgrade node-llama-cpp to be compatible with version 3

rd4cake commented 2 weeks ago

[Work in Progress]

Draft PR to address the following issue: https://github.com/langchain-ai/langchainjs/issues/6994 Using asynchronous function to load models from node-llama-cpp breaks adherence to the default way of instantiating components ( chatmodels, embeddings, llms, etc ).

Here is an example:

before with node-llama-cpp v2

const llamaPath = "/path/to/your/model.gguf"; 

const model = new LlamaCpp({ modelPath: llamaPath });

after with node-llama-cpp v3

const llamaPath = "/path/to/your/model.gguf"; 

const model = await LlamaCpp.llmInit({ modelPath: llamaPath });

Question for the maintainers: Our implementation changes the way LlamaCpp model is instantiated. Is it this fine moving forward?

vercel[bot] commented 2 weeks ago

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
langchainjs-docs	🛑 Canceled (Inspect)			Nov 11, 2024 10:04pm

1 Skipped Deployment

Jacky3003 commented 2 weeks ago

@jacoblee93 was wondering if it was possible for you to take a look at what we have so far for our implementation, it would be greatly appreciated.

jacoblee93 commented 4 days ago

CC @nigel-daniels

jacoblee93 commented 4 days ago

Just one merge to main should be fine! If you keep doing it, it requires CI to run again.

langchain-ai / langchainjs

fix(community) : Upgrade node-llama-cpp to be compatible with version 3 #7135