teamclairvoyant / airflow-scheduler-failover-controller

A process that runs in unison with Apache Airflow to control the Scheduler process to ensure High Availability
Apache License 2.0
232 stars 58 forks source link

a bytes-like object is required, not 'str' 1.0.4 #20

Closed HWTsang closed 5 years ago

HWTsang commented 5 years ago

I tried use this in airflow-1.10 with 4 machine.After I install the zip packages in one machine, I use scheduler_failover_controller init,then I change the airflow.cfg scheduler_nodes_in_cluster = slave2,slave3,and start it in the installed machine,then it is wrong with a bytes-like object is required, not 'str'.

HWTsang commented 5 years ago

[2019-08-01 08:56:50,830] {failover_controller.py:37} INFO - Active Failover Node: slave3 [2019-08-01 08:56:50,830] {failover_controller.py:38} INFO - Active Scheduler Node: None [2019-08-01 08:56:50,830] {failover_controller.py:39} INFO - Last Failover Heartbeat: 2019-08-01 08:40:42. Current time: 2019-08-01 08:56:50.830097. [2019-08-01 08:56:50,830] {failover_controller.py:43} INFO - This Failover Controller is on Standby. [2019-08-01 08:56:50,830] {failover_controller.py:46} INFO - There already is an active Failover Controller 'slave3' [2019-08-01 08:56:50,830] {failover_controller.py:48} WARNING - Discovered this Failover Controller should be active because Active Failover Node is this nodes host [2019-08-01 08:56:50,830] {failover_controller.py:208} WARNING - Setting this Failover Node to ACTIVE [2019-08-01 08:56:50,839] {failover_controller.py:213} WARNING - This Failover Controller is now ACTIVE. [2019-08-01 08:56:50,839] {failover_controller.py:71} INFO - This Failover Controller is ACTIVE [2019-08-01 08:56:50,839] {failover_controller.py:79} INFO - Setting Failover Heartbeat [2019-08-01 08:56:50,841] {failover_controller.py:83} WARNING - Active Scheduler is None [2019-08-01 08:56:50,842] {failover_controller.py:184} INFO - Starting to search for current Active Scheduler Node... [2019-08-01 08:56:50,842] {failover_controller.py:143} INFO - Starting to Check if Scheduler on host 'slave3' is running... [2019-08-01 08:56:50,859] {failover_controller.py:166} INFO - Finished Checking if Scheduler on host 'slave3' is running. is_running: True [2019-08-01 08:56:50,859] {failover_controller.py:143} INFO - Starting to Check if Scheduler on host 'slave2' is running... Traceback (most recent call last): File "/root/anaconda3/bin/scheduler_failover_controller", line 6, in exec(compile(open(file).read(), file, 'exec')) File "/mnt/data/airflow-scheduler-failover-controller-master/scheduler_failover_controller/bin/scheduler_failover_controller", line 7, in args.func(args) File "/mnt/data/airflow-scheduler-failover-controller-master/scheduler_failover_controller/bin/cli.py", line 98, in start logger=logger File "/mnt/data/airflow-scheduler-failover-controller-master/scheduler_failover_controller/app.py", line 35, in main failover_controller.poll() File "/mnt/data/airflow-scheduler-failover-controller-master/scheduler_failover_controller/failover/failover_controller.py", line 84, in poll active_scheduler_node = self.search_for_active_scheduler_node() File "/mnt/data/airflow-scheduler-failover-controller-master/scheduler_failover_controller/failover/failover_controller.py", line 187, in search_for_active_scheduler_node if self.is_scheduler_running(node): File "/mnt/data/airflow-scheduler-failover-controller-master/scheduler_failover_controller/failover/failover_controller.py", line 155, in is_scheduler_running if line.strip() != "" and process_check_command not in line and grep_command not in line and grep_command_no_quotes not in line and full_status_check_command not in line: TypeError: a bytes-like object is required, not 'str'

HWTsang commented 5 years ago

i start webserver and scheduler in slave3 ,worker in slave2. airflow-scheduler-failover-controller-master is just installed in slave3 .

HWTsang commented 5 years ago

[root@slave3 airflow-scheduler-failover-controller-master]# scheduler_failover_controller test_connection [2019-08-01 08:59:06,495] {settings.py:182} INFO - settings.configure_orm(): Using pool settings. pool_size=5, pool_recycle=1800, pid=16715 Testing Connection for host 'slave3' (True, ['Connection Succeeded', '']) Testing Connection for host 'slave2' (True, [b'Connection Succeeded\n'])

sudhakar-nallam commented 5 years ago

@WtyFile Please look at the #19 for a quick fix. We will update the master soon.

HWTsang commented 5 years ago

i am not very clear. i change command_runner.py as the #15 ,and then pip install it. still has the wrong. please update it soon. Thanks very much ;-)

sudhakar-nallam commented 5 years ago

@WtyFile Please try the latest release and let us know if you still have any issues.