qdrant / qdrant

Qdrant - High-performance, massive-scale Vector Database for the next generation of AI. Also available in the cloud https://cloud.qdrant.io/
https://qdrant.tech
Apache License 2.0
19.25k stars 1.31k forks source link

Qdrant Cloud: can't create snapshot #3942

Open Morriz opened 4 months ago

Morriz commented 4 months ago

It seems impossible to create a snapshot of a free quadrant cloud node with the SDK. I don't know if that is intended, but I designate it to be a bug. Please move accordingly...

I followed the instructions from these locations:

Current Behavior

I get a 404 when trying to create a snapshot.

Steps to Reproduce

  1. create a free node
  2. create a collection
  3. use the sdk to create a snapshot
# list the collections:
from qdrant_client import QdrantClient

QDRANT_MAIN_URL = "xx.us-east-1-0.aws.cloud.qdrant.io:6333"
QDRANT_API_KEY = "xx"

client = QdrantClient(QDRANT_MAIN_URL, api_key=QDRANT_API_KEY)
resp = client.get_collections()
collections = [c.name for c in resp.collections]

# create snapshots and keep urls
QDRANT_NODES = ["node-0-xx.us-east-1-0.aws.cloud.qdrant.io:6333"]

snapshot_urls = {}
for collection in collections:
    urls = []
    for node_url in QDRANT_NODES:
        node_client = QdrantClient(node_url, api_key=QDRANT_API_KEY, port=6334)
        snapshot_info = node_client.create_snapshot(collection_name=collection)

        snapshot_url = f"{node_url}/collections/{collection}/snapshots/{snapshot_info.name}"
        urls.append(snapshot_url)

    snapshot_urls[collection] = urls

Expected Behavior

snapshot to be created

generall commented 3 months ago

Hey @Morriz, could you please make sure that you are requesting individual nodes and not hitting the load balancer?

In qdrant cloud you can connect to individual node by using prefix node-X- to the node URL.

It is important because qdrant creates per-node snapshots.

Morriz commented 3 months ago

Hi @generall, my code shows I am doing this...

But is the free plan blocking those endpoints?

generall commented 3 months ago

no, free plan doesn't block any requests

generall commented 3 months ago

Do you see snapshots in the UI?

Morriz commented 3 months ago

no, it says snapshot only available in paid version...but how can we get our data out otherwise?

generall commented 3 months ago

no, it says snapshot only available in paid version...but how can we get our data out otherwise?

This can't be true. Probably you are mixing up snapshots and backups, those are two difference machanisms

Morriz commented 3 months ago

Do you see snapshots in the UI?

No I don't see those either...