cylondata / twister2

A composable framework for fast and scalable data analytics
https://twister2.org
Apache License 2.0
57 stars 32 forks source link

Client Logger at @k8s #872

Closed ahmet-uyar closed 4 years ago

ahmet-uyar commented 4 years ago

Implemented logging at client at k8s. A configuration parameter added: kubernetes.log.in.client JobLogger runs when this parameter is true. Job log files are saved in the directory: $HOME/.twister2/jobID Log files names are in the form of: worker${ID}-ip{workerIP}-${logFileIndex}.log Updated k8s documentation for logging. Handled scaling up of workers. Handled restarted workers. New log files are generated for restarted workers with new counter index.