[REQUEST]: working tool call configuration with localai

codefromthecrypt commented 2 months ago

Has this been supported or requested before?

[X] I have checked the GitHub README.
[X] I have checked the Qwen documentation.
[X] I have checked the documentation of the related framework.
[X] I have searched the issues and there is not a similar one.

What is this feature about?

tool calling with LocalAI

Proposal

Introduction

I would like to see an example of tool calling with LocalAI, similar to Olllama

Rational

My company has customers that use ollama which works with tool calling, but we also have customers that use LocalAI. I would like to investigate qwen2.5 as an option for those using LocalAI

Anything else

Here is an example pull request which when run with LocalAI, Qwen2.5 cannot complete the same tests that pass with Ollama (both via the OpenAI API). It is my assumption the reason is that Ollama controls the prompt when calling tools https://github.com/square/exchange/pull/51

Contributions are welcomed

[ ] I am willing to help implement this feature.

github-actions[bot] commented 4 weeks ago

This issue has been automatically marked as inactive due to lack of recent activity. Should you believe it remains unresolved and warrants attention, kindly leave a comment on this thread.

codefromthecrypt commented 3 weeks ago

still hoping for an answer

QwenLM / Qwen2.5