Currently, TonY only get the executor's exit code and then attach on the Yarn diagnostics panel. If we could get the detailed python exit detailed diagnostics message, it will be better to statistical analysis and debug.
Maybe the first step is to separate the python's process error and standard output to different files.
And then we could get the error msg if exit code is non-zero.
TODO
[x] Separate the python subprocess log from task executor log. #652
[x] Collect the detailed execution error log to Yarn diagnostics. #653
Why
Currently, TonY only get the executor's exit code and then attach on the Yarn diagnostics panel. If we could get the detailed python exit detailed diagnostics message, it will be better to statistical analysis and debug.
Maybe the first step is to separate the python's process error and standard output to different files.
And then we could get the error msg if exit code is non-zero.
TODO