[Request] support other embedding?

[ukhack]

🥰 需求描述

目前除了openai的text embedding，有很多优秀的向量模型。例如silliconcloud支持的bge-m3等 给像我这样的非openai ai(google gemini user)用户有可选择的余地

🧐 解决方案

建议追加一个或两个其他模型，后期可以开放sdk给用户自己对接，例如你规定api的格式，用户自己提供api url。

📝 补充信息

No response

[lobehubbot]

👀 @ukhack

Thank you for raising an issue. We will investigate into the matter and get back to you as soon as possible. Please make sure you have given us as much context as possible.\ 非常感谢您提交 issue。我们会尽快调查此事，并尽快回复您。 请确保您已经提供了尽可能多的背景信息。

[lobehubbot]

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

---

🥰 Description of requirements

Currently, in addition to openai's text embedding, there are many excellent vector models. For example, bge-m3 supported by silliconcloud, etc. Give non-openai ai (google gemini user) users like me a choice

🧐 Solution

It is recommended to add one or two other models. Later, the SDK can be opened for users to connect by themselves. For example, you specify the format of the API and the user provides the API URL.

📝 Supplementary information

No response

[dosubot[bot]]

I found a similar unsolved discussion regarding support for various models and platforms in a plugin format ^[1]. This might be relevant to your request for supporting other embedding models.

Currently, Lobe Chat primarily uses OpenAI's text embeddings, but it does support a variety of AI providers and models, including Google AI (Gemini Pro, Gemini Vision) ^[2].

To address your request for supporting other embedding models, such as those provided by SiliconCloud, and to allow users to integrate their own models via an SDK, you can follow these steps:

1. Define the Base URL: Set up the base URL for the new embedding model's API.
2. Customize Payload Handling: Adjust the payload handling to meet the specific requirements of the new embedding model.
3. API Key Configuration: Ensure that the API key for the new provider can be configured within Lobe Chat.

Here is an example template for integrating a new embedding model:

import OpenAI from 'openai';

import { EmbeddingsPayload, EmbeddingsOptions, EmbeddingItem } from '../types';
import { LobeOpenAICompatibleFactory } from '../utils/openaiCompatibleFactory';

export const LobeNewEmbeddingModel = LobeOpenAICompatibleFactory({
  baseURL: 'https://api.new-embedding-model.com/v1',
  embeddings: {
    handlePayload: (payload: EmbeddingsPayload) => {
      const { user, ...rest } = payload;

      return { 
        ...rest, 
        // Customize payload as needed
      };
    },
  },
  debug: {
    embeddings: () => process.env.DEBUG_NEW_EMBEDDING_MODEL === '1',
  },
  provider: 'NewEmbeddingModel',
});

Replace https://api.new-embedding-model.com/v1 with the actual base URL of the new embedding model and customize the payload handling logic as required by the new provider's API.

Additionally, you can refer to the Plugin Development Guide in the Wiki for more details on how to extend Lobe Chat's functionality with plugins ^[3].

By following these steps, you can integrate and configure new embedding models in Lobe Chat, providing more options for users who prefer non-OpenAI models.

To continue talking to Dosu, mention @dosu.

[arvinxx]

重复 https://github.com/lobehub/lobe-chat/issues/3928

[lobehubbot]

✅ @ukhack

This issue is closed, If you have any questions, you can comment and reply.\ 此问题已经关闭。如果您有任何问题，可以留言并回复。

[lobehubbot]

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

---

Duplicate https://github.com/lobehub/lobe-chat/issues/3928