Does volcano support log aggregation?

[melin]

Does volcano support log aggregation? Similar to yarn log aggregation feature

[xiao-jay]

Can you describe the scenario where log aggregation is required？volcano using klog now ,klog don't have log aggregation.

[melin]

类似hadoop yarn 日志聚合功能，例如spark 运行结果后，如果yarn开启日志聚合，yarn logs applicationId 命令非常方便查看任务运行。运行在k8s，如果任务挂掉，查看日志不方便，需要主动去收集，增加使用成本。

[lowang-bh]

You means collect the job's logs for it when its log aggregation enabled?

[melin]

You means collect the job's logs for it when its log aggregation enabled?

yes，

k8s is the most troublesome to collect logs, if volcano completes this part of the work, it is the most convenient for users, collected logs can be stored to nfs, ftp, s3, hdfs, etc., support plugin storage.

Another question is whether a running job can switch queues. Similar to yarn: yarn queue-submit application-id-change new-queue

[melin]

类emr serverless spark 能够自动把日志搜集到s3，任务运行中和结束能够看到日志，

[wangyang0616]

I understand that log aggregation should be the capability of the cloud platform rather than the core of the scheduling component.

[melin]

I understand that log aggregation should be the capability of the cloud platform rather than the core of the scheduling component.

yarn 日志聚合能力，就是调度引擎提供的。这样能力需要调度引擎结合才能实现，平台只是一个展现能力

[melin]

https://mp.weixin.qq.com/s/az-PIMtGNlxgT5P4KUYmBQ

[stale[bot]]

Hello 👋 Looks like there was no activity on this issue for last 90 days. Do you mind updating us on the status? Is this still reproducible or needed? If yes, just comment on this PR or push a commit. Thanks! 🤗 If there will be no activity for 60 days, this issue will be closed (we can always reopen an issue if we need!).