Added in-memory qdrant vector db to support standardized embedding retrievers
What is a standardized embedding retriever?
inherits from the AbsStandardizedEmbeddingRetriever class
implements the embed_query and embed_corpus functions
embed_query takes in a query string and returns a list of floats (embedding of a query)
embed_corpus takes in a single table string and returns a list of floats (embedding of a table)
Support Implemented
in TARGET.py, you will now see a qdrant vector db being instatiated in memory, and running the benchmark with a standardized embedding retriever will trigger the vector db being automatically popularized with the embeddings of the tables.
in AbsTask.py, you will see the in memory vector db being passed in as a kwarg and used when the table retrieval happens.
OAIEmbedder Class
In order to test the vector db support, I wrote a simple embedder that inherits from the AbsStandardizedEmbeddingRetriever class. It basically just embeds the table string/query with openai text embedding small. This is for testing purposes.
Added in-memory qdrant vector db to support standardized embedding retrievers
What is a standardized embedding retriever?
AbsStandardizedEmbeddingRetrieverclassembed_queryandembed_corpusfunctionsembed_querytakes in a query string and returns a list of floats (embedding of a query)embed_corpustakes in a single table string and returns a list of floats (embedding of a table)Support Implemented
OAIEmbedder Class In order to test the vector db support, I wrote a simple embedder that inherits from the
AbsStandardizedEmbeddingRetrieverclass. It basically just embeds the table string/query with openai text embedding small. This is for testing purposes.