milvus-io / pymilvus

Python SDK for Milvus.
Apache License 2.0
983 stars 312 forks source link

[Bug]: Can't create partition in Milvus running on Docker Compose #2178

Closed lukezx3 closed 1 week ago

lukezx3 commented 1 month ago

Is there an existing issue for this?

Describe the bug

I am new to Milvus so I'm getting familiar with syntax and functions. I tried to create_partition but I run into an error: grpc RpcError: [create_partition], <_MultiThreadedRendezvous: StatusCode.UNIMPLEMENTED, >, <Time:{'RPC start': '2024-07-15 23:45:59.373525', 'gRPC error': '2024-07-15 23:45:59.375271'}> Traceback (most recent call last): File "/var/www/html/milvus-docker-test/milvus_docker_intro.py", line 60, in <module> client.create_partition( File "/home/vagrant/.local/lib/python3.10/site-packages/pymilvus/milvus_client/milvus_client.py", line 776, in create_partition conn.create_partition(collection_name, partition_name, timeout=timeout, **kwargs) File "/home/vagrant/.local/lib/python3.10/site-packages/pymilvus/decorators.py", line 161, in handler raise e from e File "/home/vagrant/.local/lib/python3.10/site-packages/pymilvus/decorators.py", line 143, in handler return func(*args, **kwargs) File "/home/vagrant/.local/lib/python3.10/site-packages/pymilvus/decorators.py", line 182, in handler return func(self, *args, **kwargs) File "/home/vagrant/.local/lib/python3.10/site-packages/pymilvus/decorators.py", line 91, in handler raise e from e File "/home/vagrant/.local/lib/python3.10/site-packages/pymilvus/decorators.py", line 87, in handler return func(*args, **kwargs) File "/home/vagrant/.local/lib/python3.10/site-packages/pymilvus/client/grpc_handler.py", line 403, in create_partition response = rf.result() File "/home/vagrant/.local/lib/python3.10/site-packages/grpc/_channel.py", line 883, in result raise self grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with: status = StatusCode.UNIMPLEMENTED details = "" debug_error_string = "UNKNOWN:Error received from peer {created_time:"2024-07-15T23:45:59.374735814+00:00", grpc_status:12, grpc_message:""}"

My code:

import json

client = MilvusClient(
    uri="localhost",
    token="root:Milvus"
)

if client.has_collection(collection_name="dockerDB"):
    client.drop_collection(collection_name="dockerDB")

testSchema = MilvusClient.create_schema(
    enable_dynamic_field=True
)

testSchema.add_field(field_name="Auto_id", datatype=DataType.INT64, description="The Primary Key", is_primary=True,
                     auto_id=True)
testSchema.add_field(field_name="vector", datatype=DataType.FLOAT_VECTOR, metric_type="COSINE", dim=5)
testSchema.add_field(field_name="text", datatype=DataType.VARCHAR, max_length=8192)
testSchema.add_field(field_name="sku", datatype=DataType.VARCHAR, max_length=32)
testSchema.add_field(field_name="metadata", datatype=DataType.JSON)

indexParams = MilvusClient.prepare_index_params()
indexParams.add_index(
    field_name="vector",
    metric_type="COSINE",
    index_name="test-index"
)

client.create_collection(
    collection_name="dockerDB",
    schema=testSchema,
    index_params=indexParams,
    auto_id=False
)

collection = client.describe_collection(
    collection_name="dockerDB",
)

print("TEST COLLECTION DESCRIPTION:\n\n", collection)
print("\n\n\n")

try:
    client.create_partition(
        collection_name="dockerDB",
        partition_name="partitionA"
    )
except MilvusException as e:
    print(f"An error `occurred:")`

### Steps/Code To Reproduce behavior
My docker-compose.yml:
services:
  etcd:
    container_name: milvus-etcd
    image: quay.io/coreos/etcd:v3.5.5
    environment:
      - ETCD_AUTO_COMPACTION_MODE=revision
      - ETCD_AUTO_COMPACTION_RETENTION=1000
      - ETCD_QUOTA_BACKEND_BYTES=4294967296
      - ETCD_SNAPSHOT_COUNT=50000
    volumes:
      - /var/www/html/milvus-docker-test/milvus.yaml:/milvus/configs/milvus.yaml
      - ${DOCKER_VOLUME_DIRECTORY:-.}/volumes/etcd:/etcd
    command: etcd -advertise-client-urls=http://127.0.0.1:2379 -listen-client-urls http://0.0.0.0:2379 --data-dir /etcd
    healthcheck:
      test: ["CMD", "etcdctl", "endpoint", "health"]
      interval: 30s
      timeout: 20s
      retries: 3

  minio:
    container_name: milvus-minio
    image: minio/minio:RELEASE.2023-03-20T20-16-18Z
    environment:
      MINIO_ACCESS_KEY: minioadmin
      MINIO_SECRET_KEY: minioadmin
    ports:
      - "9001:9001"
      - "9000:9000"
    volumes:
      - /var/www/html/milvus-docker-test/milvus.yaml:/milvus/configs/milvus.yaml
      - ${DOCKER_VOLUME_DIRECTORY:-.}/volumes/minio:/minio_data
    command: minio server /minio_data --console-address ":9001"
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"]
      interval: 30s
      timeout: 20s
      retries: 3

  standalone:
    container_name: milvus-standalone
    image: milvusdb/milvus:v2.4.4
    command: ["milvus", "run", "standalone"]
    security_opt:
    - seccomp:unconfined
    environment:
      ETCD_ENDPOINTS: etcd:2379
      MINIO_ADDRESS: minio:9000
    volumes:
      - /var/www/html/milvus-docker-test/milvus.yaml:/milvus/configs/milvus.yaml
      - ${DOCKER_VOLUME_DIRECTORY:-.}/volumes/milvus:/var/lib/milvus
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:9091/healthz"]
      interval: 30s
      start_period: 90s
      timeout: 20s
      retries: 3
    ports:
      - "19530:19530"
      - "9091:9091"
    depends_on:
      - "etcd"
      - "minio"

networks:
  default:
    name: milvus

```markdown
running on vmware ubuntu 22.04, pymilvus==2.4.4, docker compose v2.28.1
XuanYang-cn commented 1 month ago

@lukezx3 The uri you passing in is not a valid uri, so PyMilvus thought you're trying to connecting to Milvus-lite.

Try call ls in the running dir, you'll find a localhost file, which means you're using Milvus-lite. And Milvus-lite doesn't support partition yet.

Try to connect to Milvus using correct uri with scheme like: uri="http://localhost:19530", for host only, just use host: host="localhost"

XuanYang-cn commented 1 month ago

/assign @lukezx3

XuanYang-cn commented 1 week ago

closing idle questions