elastic / elasticsearch-py

Official Python client for Elasticsearch
https://ela.st/es-python
Apache License 2.0
4.17k stars 1.17k forks source link

Setting custom node_pool_class does not work and seems to be a typo #2578

Closed tallakh closed 1 month ago

tallakh commented 1 month ago

Elasticsearch version (bin/elasticsearch --version): 8.13.0

elasticsearch-py version (elasticsearch.__versionstr__): 8.13.0

Please make sure the major version matches the Elasticsearch server you are running.

Description of the problem including expected versus actual behavior: I want to override the mark_dead/mark_live functionality in the NodePool class since it does not make sense in our kubernetes setup which I've simply setup like this:

class NoTimeoutConnectionPool(NodePool):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)

    def mark_dead(self, connection):
        # Don't fail a node since we only have a single kubernetes endpoint for our cluster
        pass

    def mark_live(self, connection):
        # Override to do nothing on mark_live
        pass

es: AsyncElasticsearch = AsyncElasticsearch(
    hosts=[settings.elastic_host],
    sniff_on_start=False,
    sniff_before_requests=False,
    sniff_on_node_failure=False,
    request_timeout=20,
    node_pool_class=NoTimeoutConnectionPool,
)

But this code simply fails with the errors:

  File "/dev/salsa/salsa/services/elastic.py", line 47, in <module>
    es: AsyncElasticsearch = AsyncElasticsearch(
                             ^^^^^^^^^^^^^^^^^^^
  File "/dev/salsa/.venv/lib/python3.11/site-packages/elasticsearch/_async/client/__init__.py", line 399, in __init__
    _transport = transport_class(
                 ^^^^^^^^^^^^^^^^
  File "/dev/salsa/.venv/lib/python3.11/site-packages/elastic_transport/_async_transport.py", line 141, in __init__
    super().__init__(
  File "/dev/salsa/.venv/lib/python3.11/site-packages/elastic_transport/_transport.py", line 237, in __init__
    self.node_pool: NodePool = node_pool_class(
                               ^^^^^^^^^^^^^^^^
TypeError: 'DefaultType' object is not callable

Which i think is because of a minor typo in https://github.com/elastic/elasticsearch-py/blob/9a650e67dd1ffd84a6de3484dc87c056e190da13/elasticsearch/_async/client/__init__.py#L355

Where

            if node_pool_class is not DEFAULT:
                transport_kwargs["node_pool_class"] = node_class

Should be

            if node_pool_class is not DEFAULT:
                transport_kwargs["node_pool_class"] = node_pool_class
pquentin commented 1 month ago

Thank you, great find! Are you interested in putting up a pull request?

tallakh commented 1 month ago

Sure, I can create a pull request 👍