Fault tolerance works with checkpointing.
It works with both zookeeper and without. Though, there is no zookeeper, job master failure is fatal.
Faulty workers are restarted.
Faults are broadcasted to all healthy workers. They stop execution and return to WorkerManager as soon as possible in the case of failures in the job.
Workers try to restart and re-execute 5 times by default (this can be changed by config parameters.) They fail after that.
Implemented worker restart determination using ConfigMaps in K8s when no zookeeper is used.
Implemented fault notifications in jobs without zookeeper.
Fault tolerance works with checkpointing. It works with both zookeeper and without. Though, there is no zookeeper, job master failure is fatal. Faulty workers are restarted. Faults are broadcasted to all healthy workers. They stop execution and return to WorkerManager as soon as possible in the case of failures in the job. Workers try to restart and re-execute 5 times by default (this can be changed by config parameters.) They fail after that. Implemented worker restart determination using ConfigMaps in K8s when no zookeeper is used. Implemented fault notifications in jobs without zookeeper.
things to do: