知识库构建索引出错

与 #32 情况不一样，服务器资源充足。

milvus 版本 2.3.3 ，使用 docker 部署。

存储情况如下：

[root@server volumes]# du ./ --max-depth=1 -h
367M    ./etcd
8.9M    ./minio
888M    ./milvus
1.3G    ./

运行情况如下：

[root@gpu-server milvus]# docker-compose ps
NAME                IMAGE                                      COMMAND                   SERVICE      CREATED       STATUS                 PORTS
attu                zilliz/attu:v2.3.2                         "docker-entrypoint.s…"   attu         6 hours ago   Up 6 hours             0.0.0.0:8000->3000/tcp, :::8000->3000/tcp
milvus-etcd         quay.io/coreos/etcd:v3.5.5                 "etcd -advertise-cli…"   etcd         6 hours ago   Up 6 hours (healthy)   2379-2380/tcp
milvus-minio        minio/minio:RELEASE.2023-03-20T20-16-18Z   "/usr/bin/docker-ent…"   minio        6 hours ago   Up 6 hours (healthy)   0.0.0.0:9000-9001->9000-9001/tcp, :::9000-9001->9000-9001/tcp
milvus-standalone   milvusdb/milvus:v2.3.3                     "/tini -- milvus run…"   standalone   6 hours ago   Up 6 hours (healthy)   0.0.0.0:19530->19530/tcp, :::19530->19530/tcp

错误日志如下：

[2024-03-21 20:43:06,020: ERROR/MainProcess] RPC error: [insert_rows], <DataNotMatchException: (code=1, message=Attempt to insert an unexpected field to collection without enabling dynamic field)>, <Time:{'RPC start': '2024-03-21 20:43:06.017347', 'RPC error': '2024-03-21 20:43:06.020646'}>
[2024-03-21 20:43:06,021: ERROR/MainProcess] Failed to insert batch starting at entity: 0/11
[2024-03-21 20:43:06,021: ERROR/MainProcess] Failed to insert batch starting at entity: 0/11
[2024-03-21 20:43:06,021: ERROR/MainProcess] consume document failed
Traceback (most recent call last):
  File "/app/api/core/indexing_runner.py", line 70, in run
    self._load(
  File "/app/api/core/indexing_runner.py", line 667, in _load
    index_processor.load(dataset, chunk_documents)
  File "/app/api/core/rag/index_processor/processor/paragraph_index_processor.py", line 59, in load
    vector.create(documents)
  File "/app/api/core/rag/datasource/vdb/vector_factory.py", line 121, in create
    self._vector_processor.create(
  File "/app/api/core/rag/datasource/vdb/milvus/milvus_vector.py", line 75, in create
    self.add_texts(texts, embeddings)
  File "/app/api/core/rag/datasource/vdb/milvus/milvus_vector.py", line 101, in add_texts
    raise e
  File "/app/api/core/rag/datasource/vdb/milvus/milvus_vector.py", line 95, in add_texts
    ids = self._client.insert(collection_name=self._collection_name, data=batch_insert_list)
  File "/usr/local/lib/python3.10/site-packages/pymilvus/milvus_client/milvus_client.py", line 206, in insert
    raise ex from ex
  File "/usr/local/lib/python3.10/site-packages/pymilvus/milvus_client/milvus_client.py", line 198, in insert
    res = conn.insert_rows(collection_name, insert_batch, timeout=timeout)
  File "/usr/local/lib/python3.10/site-packages/pymilvus/decorators.py", line 127, in handler
    raise e from e
  File "/usr/local/lib/python3.10/site-packages/pymilvus/decorators.py", line 123, in handler
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/pymilvus/decorators.py", line 162, in handler
    return func(self, *args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/pymilvus/decorators.py", line 102, in handler
    raise e from e
  File "/usr/local/lib/python3.10/site-packages/pymilvus/decorators.py", line 68, in handler
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/pymilvus/client/grpc_handler.py", line 501, in insert_rows
    request = self._prepare_row_insert_request(
  File "/usr/local/lib/python3.10/site-packages/pymilvus/client/grpc_handler.py", line 482, in _prepare_row_insert_request
    return Prepare.row_insert_param(
  File "/usr/local/lib/python3.10/site-packages/pymilvus/client/prepare.py", line 422, in row_insert_param
    return cls._parse_row_request(request, fields_info, enable_dynamic, entities)
  File "/usr/local/lib/python3.10/site-packages/pymilvus/client/prepare.py", line 370, in _parse_row_request
    raise DataNotMatchException(message=ExceptionsMessage.InsertUnexpectedField)
pymilvus.exceptions.DataNotMatchException: <DataNotMatchException: (code=1, message=Attempt to insert an unexpected field to collection without enabling dynamic field)>
[2024-03-21 20:43:06,028: INFO/MainProcess] Processed dataset: bd66a1d2-d871-42c4-8fe7-4275be32a591 latency: 6.201149852946401
[2024-03-21 20:43:06,064: INFO/MainProcess] Task tasks.document_indexing_task.document_indexing_task[0df93ed7-9cae-4670-842d-8032b3d5852b] succeeded in 6.23699377477169s: None

wxywb / history_rag

知识库构建索引出错 #60