.Net: Bug: Microsoft.SemanticKernel.Connectors.Onnx issues exception while using LocalEmbeddings

[arafattehsin]

Describe the bug With the new Onnx package that allows us to use the SLMs like Phi-3 by just loading a model from file, I think there's an issue when I try to use the local embeddings.

Whenever you remove the Onnx package, try to load Phi-3 or any other model using localhost (for example, using LM Studio or Ollama) it just works fine.

To Reproduce Steps to reproduce the behavior:

When you install the mentioned package and try to use LocalEmbeddings with the model, you get an error as shown below.

Expected behavior This should work just fine.

Screenshots

Platform

• OS: Windows
• IDE: Visual Studio
• Language: C#
• Source: 1.16.2 and (1.16.2-alpha)

Additional context I have written a detailed blog post on how I am doing this but still not able to work now. When I tried to use this package, it just works fine. https://github.com/feiyun0112/SemanticKernel.Connectors.OnnxRuntimeGenAI

[Barshan-Mandal]

ITextEmbeddingGenerationService is also not found during using it with phi 3 vision

[arafattehsin]

As a workaround (or an alternate solution), I have fixed it and mentioned the steps here: https://arafattehsin.com/ai-copilot-offline-phi3-semantic-kernel/

[RogerBarreto]

@arafattehsin, Thanks for reaching out, the package is currently 5 months old and do not support the latest versions of Onnx, checking their repo I can see that the LocalEmbedder now implements SemanticKernel embeddings interface.

https://github.com/dotnet/smartcomponents/blob/4dbb671443c84407b598a0104441afd1186d9a3a/src/SmartComponents.LocalEmbeddings/LocalEmbedder.cs#L16

Your alternative solution in my opinion should be the main one so you don't add any other dependencies to the project.

Thanks for putting the Blog in place and was very inspiring also in bringing a Demonstration of ONNX to our demos (WIP).