search

developersdigest / llm-answer-engine

Build a Perplexity-Inspired Answer Engine Using Next.js, Groq, Llama-3, Langchain, OpenAI, Upstash, Brave & Serper
https://developersdigest.tech
MIT License
4.66k stars 742 forks source link

HuggingFaceTransformersEmbeddings does not work #7

Closed taidbui closed 7 months ago

taidbui commented 7 months ago

Hi,

I have tried to use the following for embedding. However, I got an error probably related to using the runtime = "edge"

const embeddings = new HuggingFaceTransformersEmbeddings({modelName: "Xenova/all-MiniLM-L6-v2"});

Here is the link to langchain js documentation: https://js.langchain.com/docs/integrations/text_embedding/transformers

I am wondering if it could be an easy fix for this.

Thanks

developersdigest commented 7 months ago

Yes - this is likely a runtime issue and related to to the X

"Transformers.js uses ONNX Runtime to run models in the browser."

https://www.npmjs.com/package/@xenova/transformers

If you are looking to run open source embeddings models. Right now, I do have support set up for Ollama and I have plans now on the roadmap to also add in AI Gateway to support even more model!