qdrant / rust-client

Rust client for Qdrant vector search engine
https://crates.io/crates/qdrant-client
Apache License 2.0
222 stars 45 forks source link

Bad request: Empty update request when points.len() < chunk_size missing points bug #174

Closed lijingrs closed 1 month ago

lijingrs commented 2 months ago
    pub async fn upsert_points_chunked(
        &self,
        request: impl Into<UpsertPoints>,
        chunk_size: usize,
    ) -> QdrantResult<PointsOperationResponse> {
        let mut request = request.into();
        let points = std::mem::take(&mut request.points);

        if points.len() < chunk_size {
            return self.upsert_points(request).await;
        }

        let request = &request;
        let points = &points;

        self.with_points_client(|mut points_api| async move {
            let mut resp = PointsOperationResponse {
                result: None,
                time: 0.0,
            };

            for chunk in points.clone().chunks(chunk_size) {
                let mut chunked_request = request.clone();
                chunked_request.points = chunk.to_vec();

                let PointsOperationResponse { result, time } =
                    points_api.upsert(chunked_request).await?.into_inner();

                resp.result = result;
                resp.time += time;
            }

            Ok(resp)
        })
        .await
    }
timvisee commented 2 months ago

What is the exact problem here?

Are you saying that this _chunked function doesn't work properly if the given set of points is smaller than the chunk size? If so, could you provide an reproducible example for this.

lijingrs commented 1 month ago

What is the exact problem here?

Are you saying that this _chunked function doesn't work properly if the given set of points is smaller than the chunk size? If so, could you provide an reproducible example for this.

let upsert_points = UpsertPointsBuilder::new( text_collection_name.to_string(), text_point_structs.clone(), ); let _text_result = qdrant_client.upsert_points_chunked(upsert_points,300).await; Executing this code will throw an exception when the length of upsert_points is less than 300.

image

let points = std::mem::take(&mut request.points);

The implementation of this code snippet looks problematic.

timvisee commented 1 month ago

You're totally right. Thank you for diving into it. I'll make sure to release a patch shortly.

timvisee commented 1 month ago

We've fixed this in the 1.11.1 patch release, which you can find here: https://github.com/qdrant/rust-client/releases/tag/v1.11.1

Thanks again for the report!