Error sending result: '<multiprocessing.pool.ExceptionWithTraceback object

[pauldintel]

I have launched the server using the docker image and vector DB launched fine : Same machine in other session I am using the vector benchmark run . I am using python and pip3.10 .

[root@9049fa05600b ~]# docker run -p 6333:6333 qdrant/qdrant _ _| | | | / |/ _ | '/ ` | ' | | | (| | (| | | | (| | | | | | _, |_,|| _,|| ||_| ||

Access web UI at http://localhost:6333/dashboard

[2023-07-18T22:34:56.204Z INFO storage::content_manager::consensus::persistent] Initializing new raft state at ./storage/raft_state [2023-07-18T22:34:56.236Z INFO qdrant] Distributed mode disabled [2023-07-18T22:34:56.236Z INFO qdrant] Telemetry reporting enabled, id: a96d4b56-dba5-4f4c-9332-04ab9c9033eb [2023-07-18T22:34:56.238Z INFO qdrant::tonic] Qdrant gRPC listening on 6334 [2023-07-18T22:34:56.238Z INFO qdrant::tonic] TLS disabled for gRPC API [2023-07-18T22:34:56.251Z INFO qdrant::actix] TLS disabled for REST API [2023-07-18T22:34:56.251Z INFO qdrant::actix] Qdrant HTTP listening on 6333 [2023-07-18T22:34:56.251Z INFO actix_server::builder] Starting 7 workers [2023-07-18T22:34:56.251Z INFO actix_server::server] Actix runtime found; starting in Actix runtime [2023-07-18T22:35:04.526Z INFO actix_web::middleware::logger] 172.17.0.1 "DELETE /collections/benchmark HTTP/1.1" 200 72 "-" "python-httpx/0.24.1" 0.000457 [2023-07-18T22:35:04.530Z INFO actix_web::middleware::logger] 172.17.0.1 "DELETE /collections/benchmark HTTP/1.1" 200 69 "-" "python-httpx/0.24.1" 0.000119 [2023-07-18T22:35:05.562Z INFO actix_web::middleware::logger] 172.17.0.1 "PUT /collections/benchmark HTTP/1.1" 200 71 "-" "python-httpx/0.24.1" 1.031083

When I am running benchmark test getting below error :

[root@9049fa05600b vector-db-benchmark]# python3.10 -m run --engines -m-16- --datasets glove-* Running experiment: qdrant-mmap-m-16-ef-128 - glove-25-angular Downloading http://ann-benchmarks.com/glove-25-angular.hdf5... Moving: /tmp/tmp6q2343mx -> /root/vectordb/vector-db-benchmark/datasets/glove-25-angular/glove-25-angular.hdf5 Experiment stage: Configure Experiment stage: Upload 1343it [00:00, 18694.49it/s] Experiment qdrant-mmap-m-16-ef-128 - glove-25-angular interrupted Traceback (most recent call last): File "/root/vectordb/vector-db-benchmark/run.py", line 52, in run client.run_experiment(dataset, skip_upload, skip_search) File "/root/vectordb/vector-db-benchmark/engine/base_client/client.py", line 70, in run_experiment upload_stats = self.uploader.upload( File "/root/vectordb/vector-db-benchmark/engine/base_client/upload.py", line 56, in upload latencies = list( File "/usr/local/lib/python3.10/multiprocessing/pool.py", line 873, in next raise value multiprocessing.pool.MaybeEncodingError: Error sending result: '<multiprocessing.pool.ExceptionWithTraceback object at 0x7f5fc488fe20>'. Reason: 'TypeError("cannot pickle '_thread.RLock' object")' Traceback (most recent call last):

File "/usr/local/lib/python3.10/runpy.py", line 196, in _run_module_as_main return _run_code(code, main_globals, None,

File "/usr/local/lib/python3.10/runpy.py", line 86, in _run_code exec(code, run_globals)

File "/root/vectordb/vector-db-benchmark/run.py", line 79, in app()

File "/root/vectordb/vector-db-benchmark/run.py", line 74, in run raise e

File "/root/vectordb/vector-db-benchmark/run.py", line 52, in run client.run_experiment(dataset, skip_upload, skip_search)

File "/root/vectordb/vector-db-benchmark/engine/base_client/client.py", line 70, in run_experiment upload_stats = self.uploader.upload(

File "/root/vectordb/vector-db-benchmark/engine/base_client/upload.py", line 56, in upload latencies = list(

File "/usr/local/lib/python3.10/multiprocessing/pool.py", line 873, in next raise value

multiprocessing.pool.MaybeEncodingError: Error sending result: '<multiprocessing.pool.ExceptionWithTraceback object at 0x7f5fc488fe20>'. Reason: 'TypeError("cannot pickle '_thread.RLock' object")'

Also there is no content generated inside results dir. Any pointer how to resolve this issue ? thanks

[chenbt-hz]

I'm facing the same issue here. My system environment is CentOS Linux release 7.4.1708 (Core), and I'm using Python 3.8.1.

Have you found any solution or workaround for this problem?

[pauldintel]

I'm facing the same issue here. My system environment is CentOS Linux release 7.4.1708 (Core), and I'm using Python 3.8.1.

Have you found any solution or workaround for this problem?

I am using CentOS 8 stream and Python 3.10 ... no solution yet .waiting for the repo owners to comment .

[chenbt-hz]

I'm facing the same issue here. My system environment is CentOS Linux release 7.4.1708 (Core), and I'm using Python 3.8.1. Have you found any solution or workaround for this problem?

I am using CentOS 8 stream and Python 3.10 ... no solution yet .waiting for the repo owners to comment .

Hello, I referred to the testing steps in this repository (https://github.com/myscale/vector-db-benchmark/blob/master/docs/step-by-step-guide-for-benchmark.md#qdrant) and made some minor modifications to the QdrantUploader.init_client method in qdrant/engine/clients/qdrant/upload.py. I understand that the program is now able to run, but it appears to be hanging during runtime and there are anomalies detected in the collection status. ` curl 'http://{ip}:{port}/collections/benchmark'

{"result":{"status":"red","optimizer_status":{"error":"Service internal error: IO Error: Function not implemented (os error 38)"},"vectors_count":12ctors_count":0,"points_count":1240620,"segments_count":24,"config":{"params":{"vectors":{"size":100,"distance":"Cosine"},"shard_number":3,"replicatite_consistency_factor":1,"on_disk_payload":true},"hnsw_config":{"m":16,"ef_construct":100,"full_scan_threshold":10000,"max_indexing_threads":0,"on_dizer_config":{"deleted_threshold":0.2,"vacuum_min_vector_number":1000,"default_segment_number":0,"max_segment_size":null,"memmap_threshold":10000,"i":20000,"flush_interval_sec":5,"max_optimization_threads":1},"wal_config":{"wal_capacity_mb":32,"wal_segments_ahead":0},"quantization_config":null},{}},"status":"ok","time":0.001639665} `

[qbx2]

You forgot to add -p 6334:6334

[KShivendu]

This was fixed in https://github.com/qdrant/vector-db-benchmark/commit/9b22a842b9a3a42ada68c28938447cebed61f696

Feel free to let us know if you continue to face the error.

[gingerwizard]

@KShivendu still facing this when running the qdrant tests, is a specific version of Python needed?

Traceback (most recent call last):

  File "<frozen runpy>", line 198, in _run_module_as_main

  File "<frozen runpy>", line 88, in _run_code

  File "/opt/vector-db-benchmark/run.py", line 91, in <module>
    app()

  File "/opt/vector-db-benchmark/run.py", line 86, in run
    raise e

  File "/opt/vector-db-benchmark/run.py", line 59, in run
    client.run_experiment(

  File "/opt/vector-db-benchmark/engine/base_client/client.py", line 115, in run_experiment
    upload_stats = self.uploader.upload(
                   ^^^^^^^^^^^^^^^^^^^^^

  File "/opt/vector-db-benchmark/engine/base_client/upload.py", line 56, in upload
    latencies = list(
                ^^^^^

  File "/home/ubuntu/.pyenv/versions/3.11.10/lib/python3.11/multiprocessing/pool.py", line 873, in next
    raise value

multiprocessing.pool.MaybeEncodingError: Error sending result: '<multiprocessing.pool.ExceptionWithTraceback object at 0x710345e4a950>'. Reason: 'TypeError("cannot pickle '_thread.RLock' object")

[glorinli]

@KShivendu still facing this when running the qdrant tests, is a specific version of Python needed?

Traceback (most recent call last):

  File "<frozen runpy>", line 198, in _run_module_as_main

  File "<frozen runpy>", line 88, in _run_code

  File "/opt/vector-db-benchmark/run.py", line 91, in <module>
    app()

  File "/opt/vector-db-benchmark/run.py", line 86, in run
    raise e

  File "/opt/vector-db-benchmark/run.py", line 59, in run
    client.run_experiment(

  File "/opt/vector-db-benchmark/engine/base_client/client.py", line 115, in run_experiment
    upload_stats = self.uploader.upload(
                   ^^^^^^^^^^^^^^^^^^^^^

  File "/opt/vector-db-benchmark/engine/base_client/upload.py", line 56, in upload
    latencies = list(
                ^^^^^

  File "/home/ubuntu/.pyenv/versions/3.11.10/lib/python3.11/multiprocessing/pool.py", line 873, in next
    raise value

multiprocessing.pool.MaybeEncodingError: Error sending result: '<multiprocessing.pool.ExceptionWithTraceback object at 0x710345e4a950>'. Reason: 'TypeError("cannot pickle '_thread.RLock' object")

Same issue

[glorinli]

In my case, the unpicklable is not in the passed-in parameters but seems the result, because I use wrong port.