search

eugeneyan / eugeneyan-comments

1 stars 1 forks source link

https://eugeneyan.com/writing/llm-experiments/ #69

Open utterances-bot opened 1 year ago

utterances-bot commented 1 year ago

Experimenting with LLMs to Research, Reflect, and Plan

Also, shortcomings in document retrieval and how to overcome them with search & recsys techniques.

https://eugeneyan.com/writing/llm-experiments/

iandanforth commented 1 year ago

"If we use exact nearest neighbours, we would get perfect recall of 1.0 but with higher latency (think seconds)."

Exact search is performant up to tens of thousands of documents / vectors. Is the document store you're embedding really that large?

eugeneyan commented 1 year ago

Not currently. Nonetheless, this is meant to scale to larger and far more documents, such as books and papers (idea here).

plotti commented 1 year ago

where is the source code for that?

kleneway commented 1 year ago

Great read, nice work on this. Thanks for highlighting the issues with retrieval. I’ve recently started building an LLM-powered assistant type app and was surprised to see how difficult the retrieval step is. There are lots of blog posts out there about how to set up a vector search tool but very few about how to optimize and troubleshoot queries and embeddings.

eugeneyan commented 1 year ago

where is the source code for that?

Currently private. It’s a mess and I’m embarrassed lol 🙈 Also needs to be scrubbed of credentials.