search

Aidenzich / road-to-master

A repo to store our research footprint on AI
MIT License
19 stars 4 forks source link

Vector Database Comparison #62

Open Aidenzich opened 1 month ago

Aidenzich commented 1 month ago

Performance Comparison

Feature Pinecone Weaviate Milvus Qdrant Chroma Elasticsearch PGvector
Is open source ❌ ✅ ✅ ✅ ✅ ❌ ✅
Self-host ❌ ✅ ✅ ✅ ✅ ✅ ✅
Cloud management ✅ ✅ ✅ ✅ ✅ ✅ (✔)
Purpose-built for Vectors ✅ ✅ ✅ ✅ ✅ ❌ ❌
Developer experience 👍👍👍👍 👍👍👍 👍👍👍👍 👍👍👍 👍👍👍 👍  
Community Community page & events 8k⭐ github, 4k slack 23k⭐ github, 4k slack 13k⭐ github, 3k discord 9k⭐ github, 6k discord 23k slack 6k⭐ github
Queries per second (using text nyt-256-angular) 150 *for p2, but more pods can be added 791 2406 326 ? 700-100 *from various reports 141
Latency, ms (Recall/Percentile 95 (millis), nyt-256-angular) 1 *batched search, 0.99 recall, 200k SBERT 2 1 4 ? ? 8
Supported index types ? HNSW Multiple (11 total) HNSW HNSW HNSW HNSW/IVFFlat
Hybrid Search (i.e., scalar filtering) ✅ ✅ ✅ ✅ ✅ ✅ ✅
Disk index support ✅ ✅ ✅ ✅ ✅ ❌ ✅
Role-based access control ❌ ✅ ❌ ❌ ✅ ✅ ❌
Dynamic segment placement vs. static data sharding ? Static sharding Dynamic segment placement Static sharding Dynamic segment placement Static sharding -
Aidenzich commented 1 month ago

Pricing & Compliance

Brand/Property Google Vector Search AWS OpenSearchServerless Pinecorn Qdrant Cloud
BAA Support Yes Yes Yes No
Provider Compliant HIPPA/PHIPA HIPPA/PHIPA HIPPA No
Serverless No Yes Yes No
Free Tier No No Free 2GB Free 1GB
Example. Pricing for 2GB Vector Store (Monthly) 196.96 USD 776.77 USD Free About 26.1 USD