python-zk / kazoo

Kazoo is a high-level Python library that makes it easier to use Apache Zookeeper.
https://kazoo.readthedocs.io
Apache License 2.0
1.3k stars 386 forks source link

Do not know how to use election function. #432

Open xiawang opened 7 years ago

xiawang commented 7 years ago

According to the given example in the document, I could not get election to work. I could not find any external code example as well. Could I get an example that shows at least 2 nodes in the election?

The only post on stackoverflow regarding this topic is: http://stackoverflow.com/questions/39125064/how-to-use-kazoo-client-for-leader-election/40219590#40219590

However, this does not show anything new, and I do not know how this can be used for election with 2 or more nodes. There are no other git repositories containing code that use kazoo as well.

Many thanks.

adityagujral commented 6 years ago

@xiawang where you able to resolve the issue? I would appreciate if you could share your findings.

xiawang commented 6 years ago

@adityagujral I was not be able to solve the problem at the time (for my class project), so my partner and I wrote one on our own.

mschirbel commented 5 years ago

I don't know if this is correct, but I did in this way:

zk.ensure_path("kingdom/election")
        election = zk.Election("/kingdom/election", "ruler")
        zk.create("/kingdom/election/second", b"Scar", ephemeral=True, sequence=False)
        zk.create("/kingdom/election/first", bytes(name, encoding='utf-8'), ephemeral=True, sequence=False)
election.run(whoIsKing)
newKing = election.contenders()

and the function:

def whoIsKing():
    ans = input('''
        I am the king, I can do whatever I want. - Scar

        ---
        Everything you see exists together in a delicate balance. 
        As king, you need to understand that balance and respect all the creatures
        from the crawling ant to the leaping antelope. - Mufasa
        Who do you agree with?
    ''')
    if(ans == 'Mufasa' or 'mufasa'):
        return True
    else:
        return False
Abby3017 commented 2 years ago

@mschirbel why you need to keep sequence False, as sequence number would be used to get smallest znode to elect leader.

I created modified version of @mschirbel code and that wasn't working for me. I agree with @xiawang there is not much present how to use leader election recipe of kazoo for more than one node.

My way of doing this (which didn't worked):

def create_path_zk(zk):
    path = get_election_path()
    zk.ensure_path(path)

def get_election_path():
    path = config['zk-election']['node_path']
    path = "/" + path
    return path

def start_zk():
    host_ip = config['zk-connection']['host']
    port = config['zk-connection']['port']
    host_address = host_ip + ":" + port
    zk = KazooClient(hosts=host_address)
    zk.start()
    # zk.add_listener(my_listener)
    return zk

def print_callback(async_obj):
    try:
        val = async_obj.get()
        print('va', val)
        z_node_name = val
    except (ConnectionLossException, NoAuthException):
        sys.exit(1)

def create_znode(zk):
    ele_path = get_election_path()
    ele_path_node = ele_path + "/test3"
    as_ob = zk.create_async(ele_path_node, ephemeral=True, sequence=True)
    as_ob.rawlink(print_callback)
def leader_election():
    print("Election completed, I won ")

def election_call(zk):
    ele_path = get_election_path()
    identifier_name = config['zk-election']['identifier_name']
    election = zk.Election(ele_path, identifier_name)
    return election

def run_election(ele):
    ele.run(leader_election)
    zk = start_zk()
    create_path_zk(zk)
    ele = election_call(zk)
    ele_path = get_election_path()
    create_znode(zk)
    run_election(ele)
    contenders = ele.contenders()
    print('contenders', contenders)
    print('election done')
    zk.stop()

I created three files with differen znode name and tried to contest for election and my leader_election func ran for all three which is confusing and not clear of Leader election recipe of zookeeper. Thanks for your time.

SaurabhMittal1989 commented 1 day ago

@mschirbel : improved you case to have practical cases for illustration


import time

from kazoo.client import KazooClient

def leader_callback():
    """Blocking call, until an input "y" is provided"""
    #Do the leader stuff"""
    ans = None
    while ans != "y":
        time.sleep(1)  # to let others join the leadership race
        print(election.contenders())
        ans = input('''I am the king now, Shall I abdicate the throne? (y/n)\n''')

    return

def do_follower_stuff():
    # poll to check if leader or follower
    # blocks, and if not leader, execute follower stuff
    # instead use locks to see if the leader stuff is done, and then can do follower stuff rather than waiting
    contenders = None
    while True:
        try:
            contenders = election.contenders()
        except:
            # retry in case election is not defined
            time.sleep(1)
        if contenders and contenders[0] == id:
            print(f"I {id} am the leader now!")
            break
        elif contenders:
            # Do the Follower stuff
            print(f"I {id} am the follower. {contenders}")
        time.sleep(5)  # Check every 5 seconds

zk = KazooClient(hosts='127.0.0.1:2181')
zk.start()
zk.ensure_path("kingdom/election")
id = "node" + str(random.randint(1, 1000))

# Start the leader check in a non-blocking way, and execute follower code if leader check is false
import threading

leader_thread = threading.Thread(target=do_follower_stuff)
leader_thread.start()

while True:
    # election.run blocks the thread, until elected
    # Once elected, the thread becomes leader until the func=whoIsKing returns,
    # this thread surrenders the leadership, once the func=whoIsKing returns
    # rejoin the leadership race, after surrendering the leadership, perpetually

    election = zk.Election("/kingdom/election", id)
    election.run(func=leader_callback)```