Open francescobianco opened 3 years ago
Hi @francescobianco
Sure, it's easy to do. I intentionally patch airflow.cfg
to split the logs into two files to keep airflow and cwl logs separately. You can see how I do it here
https://github.com/Barski-lab/cwl-airflow/blob/9c2c8522c0969458915123e501591ebf15cc69f5/cwl_airflow/components/init/config.py#L105
If you want to to keep both of the logs in the same file, set the default value for logging_config_class
in airflow.cfg
.
Let me know if you have any other questions.
I want collect all generated output by every task into standard Airflow task log file, generally is
./logs/{dag_id}/{task_id}/{run_stamp}/1.log
Furthermore I would get all logs for every task into single file, I want collapse all log into file
1.log
and not into1.log.cwl