quackscience / duckdb-extension-openprompt

DuckDB Community Extension to prompt LLMs from SQL
https://duckdb.org/community_extensions/extensions/open_prompt.html
MIT License
24 stars 1 forks source link

Any plans for something like `open_embedding()` support? #16

Open aaronsteers opened 1 week ago

aaronsteers commented 1 week ago

First of all: I love that this extension exists! I just stumbled on it today (shared by MotherDuck team member) and I'll definitely share with others who are interested in this capability.

Do you have any plans to add something like open_embedding() as companion/complement to open_prompt() or are you aware of anyone else looking at adding it?

Thanks for all your work!

lmangani commented 1 week ago

Hello @aaronsteers and thanks for the kind words. The code is just an HTTP wrapper so we can implement new functions pretty fast if you're willing to provide a curl example and a mockup of the function/parameters this would require.

lmangani commented 1 week ago

@aaronsteers another user of our http_client extension came up with this macro:

INSTALL http_client FROM community; LOAD http_client; 
CREATE OR REPLACE MACRO embedding(q) AS ( 
WITH  __input AS (
    SELECT
      http_post(
          'https://api.openai.com/v1/embeddings',
          headers => MAP {
            'Authorization': 'Bearer sk_your_openai_key',
            'Content-Type': 'application/json; charset=UTF-8'
          },
          params =>{
            'model': 'BAAI/bge-large-zh-v1.5',
            'input': q,
            'encoding_format':'float'
          }
      ) AS res
)
-- SELECT (table __input)
SELECT json_extract(((table __input)->>'body')::JSON ,'$.data[0].embedding') as  chat_response
);
select embedding('"DuckDB"') as embedding

If this is what you're after, we can implement it in the open_prompt() extension quite easily