search

opensearch-project / opensearch-py

Python Client for OpenSearch
https://opensearch.org/docs/latest/clients/python/
Apache License 2.0
316 stars 161 forks source link

[DOC] Is OpenSearch python client thread safe? #753

Open Shodded opened 1 month ago

Shodded commented 1 month ago

Versions Client - opensearch-py = "2.5.0" AWS serverless = 2.0.x

Describe the issue: Is the OpenSearch client with connection_class=Urllib3HttpConnection thread-safe?

As part of adopting OpenSearch in our Python repository, we aim to verify whether the client configured with connection_class=Urllib3HttpConnection is thread-safe. We have noted that perform_request utilizes Urllib3's connection pool, which is known to be thread-safe. However, we have found no additional documentation that clarifies this issue further.

Configuration:

OpenSearch(
            hosts=[(host=host, port=port)],
            http_auth=Urllib3AWSV4SignerAuth(credentials, AWS_REGION, "aoss"),
            use_ssl=True,
            verify_certs=True,
            connection_class=Urllib3HttpConnection,
            retry_on_timeout=True
        )
dblock commented 1 month ago

This is a good question. AFAIK it is, I believe it was used across a thread pool in https://github.com/opensearch-project/OpenSearch-benchmark, but I am not 100% sure. How about we clarify this? Want to write a mutlithreaded sample and contribute it to https://github.com/opensearch-project/opensearch-py/tree/main/samples?