search

python-zk / kazoo

Kazoo is a high-level Python library that makes it easier to use Apache Zookeeper.
https://kazoo.readthedocs.io
Apache License 2.0
1.3k stars 386 forks source link

KazooClient thread-safety and instance reuse #728

Closed bossie closed 11 months ago

bossie commented 1 year ago

I couldn't immediately find an answer in the docs re: thread-safety and reuse of KazooClient.

I would like to know if an instance of KazooClient is supposed to be short-lived and limited to a single thread or if a shared instance can be used from multiple threads.

In particular, is there any problem with:

  1. instantiate + start() a single instance of KazooClient at the startup of a web application and reusing it everywhere;
  2. stop() and close() said instance at shutdown?

Thanks!

ceache commented 1 year ago

Hi,

It can safely be used across multiple threads, but do not use it across processes (forks).

Cheers,

On Wed, Sep 6, 2023, 10:37 Jan Van den bosch @.***> wrote:

I couldn't immediately find an answer in the docs re: thread-safety and reuse of KazooClient.

I would like to know if an instance of KazooClient is supposed to be short-lived and limited to a single thread or if a shared instance can be used from multiple threads.

In particular, is there any problem with:

  1. instantiate + start() a single instance of KazooClient at the startup of a web application and reusing it everywhere;
  2. stop() and close() said instance at shutdown?

Thanks!

— Reply to this email directly, view it on GitHub https://github.com/python-zk/kazoo/issues/728, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAIFTHW7WQJGDOH3IXGPXV3XZCDEFANCNFSM6AAAAAA4NOM42E . You are receiving this because you are subscribed to this thread.Message ID: @.***>

bossie commented 1 year ago

Thanks. So would you recommend using a single KazooClient instance across the entire application? Are there any drawbacks, considerations?

ceache commented 12 months ago

None, we have been doing single kazoo client shared by multiple threads for years now. Only advice is make sure you run a little code on the watchers' thread as possible (there is only one per KazooClient) since it will be shared. Create a worker thread and pass it all the events for processing using a queue.Queue.

Cheers,

On Wed, Sep 6, 2023, 11:19 Jan Van den bosch @.***> wrote:

Thanks. So would you recommend using a single KazooClient instance across the entire application? Are there any drawbacks, considerations?

— Reply to this email directly, view it on GitHub https://github.com/python-zk/kazoo/issues/728#issuecomment-1708588496, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAIFTHS4WPODHYISAQJYRE3XZCIBNANCNFSM6AAAAAA4NOM42E . You are receiving this because you commented.Message ID: @.***>

bossie commented 11 months ago

That's clear, thanks.