yorkie-team / codepair

Build your own AI-powered collaborative markdown editor in just 5 minutes
https://codepair.yorkie.dev
Apache License 2.0
49 stars 19 forks source link

Refactor Yorkie Intelligence model specification to improve usability #366

Closed sihyeong671 closed 1 month ago

sihyeong671 commented 1 month ago

What would you like to be added: Currently, specifying models for Yorkie Intelligence directly in the code seems unnecessary. I propose implementing a try-catch mechanism that can handle scenarios where the model is not available, thereby notifying the user of an error or disabling the Yorkie Intelligence feature if necessary.

Considering the rapid development of models like Llama3.2 and Qwen, this change is expected to streamline the code-writing process for users, making it easier to integrate and utilize new models.

Why is this needed: This enhancement aims to improve the usability and flexibility of the Yorkie Intelligence feature, allowing developers to focus on functionality without being constrained by rigid model specifications in the code.

sihyeong671 commented 1 month ago

@devleejb I need a little help.

I remove modelList type and if phase before making ChatOllama. but ChatOllama doesn't check model is really exist in ollama backend. (ex) 113bb, llaaa)

So error works when we actual call model. how to refactor model...? do you have an idea about this?

devleejb commented 1 month ago

@sihyeong671

The issue arises from the implementation in langchain/ollama. Specifically, the pull function is triggered when the user calls generate (see: langchainjs chat_models.ts#L633).

I believe we have three possible approaches:

1. Call the pull function during the initialization of the ChatModel instance.
2. Delay model validation until the user calls generate.
3. Consult with Ollama to determine if there’s a way to validate the model string beforehand. (It doesn’t seem like this is currently supported, so we could consider creating an issue for this on GitHub.)

Personally, I think the third approach is the best. If model validation isn’t feasible, the second option could be a good fallback since many users may not want to pull a large model when the server starts.