jonathanpv
commented
9 months ago I'm personally working with Lance to build my own product so if I get through that and it becomes an easy step to replicate for vector.dev I can contribute that myself. However if someone else knows about this space I would love to collaborate.
A note for the community
Use Cases
as a user I can ramp up and learn vector quickly
as a user I can search with natural language through vector.dev documentation and tutorials
Attempted Solutions
No response
Proposal
Proposal:
I propose periodic indexing of the entire content vector.dev has from tutorials, code, etc
This can be done with embedding models like text-ada-002, but in order to have a cheap service and not depend on expensive search like algolia, elasticsearch, etc. We can leverage open source tool LanceDB (its in rust).
LanceDB is a disk implementation of ANN (approximate nearest neighbor) which is the search algorithm for anything requiring vectors/matrices. E.g the language that machine learning speaks.
I propose a lightweight playground UI, user's provide their openAI token (we don't store this), we provide the lancefile, may cost $30-$50 to build and if we want to host it in an S3 bucket that will just be another monthly cost requirement.
Result:
User can search the docs through a playground ui, maybe even embed it in VRL playground?
If it's the case Lance doesn't work fully in wasm, then a CLI tool to search the docs of vector will definitely be possible.
This can live in an entirely separate project. Perhaps vectordotdev/vector-docs-search
Flow:
Web version (may be limited since lance is rust->wasm):
[client goes to vector-docs-search site] [vector-docs-search-site provides lancefile through S3 bucket] [client writes a prompt/search query] [prompt/search query goes through client's provided openAI key] [lance.js searches through lancefile (500ms) <- high latency is due to optimizations needing to be done in particular with S3 loading] [client gets a list of most relevant documents] [client can then chain those to openAI and get a natural language response back]
Desktop CLI
The same as the above flow, but instead of hosting the lancefile in S3 we can directly have the user download it, and it will simply be a single file python script to search.
References
https://www.youtube.com/watch?v=cfXpBA-7qIo&t=584s
Version
No response