Initial prototype

[simonw]

The initial goal of this plugin is to help me build a prototype of semantic search using embeddings - to work against a table created using this tool:

• https://github.com/simonw/openai-to-sqlite/issues/1

So two SQL functions: openai_embedding(text, api_key) and openai_embedding_similarity(a, b).

[simonw]

This worked:

with query as (
  select
    openai_embedding(:query, :token) as q
),
top_n as (
  select
    id,
    openai_embedding_similarity(query.q, embedding) as score
  from
    blog.embeddings
    ,query
  order by
    score desc
  limit
    10
)
select
  blog_entry.id,
  blog_entry.title,
  top_n.score
from
  simonwillisonblog.blog_entry
  join top_n on blog_entry.id = top_n.id
order by
  score desc

With Datasette run using:

datasette ../openai-to-sqlite/blog.db simonwillisonblog.db \
  --crossdb -p 8003 --setting sql_time_limit_ms 10000