search

Tencent / caelus

Set of Kubernetes solutions for reusing idle resources of nodes by running extra batch jobs
Other
344 stars 83 forks source link

Whether LinuxContainerExecutor could be supported on NM runs in Docker #47

Closed zuston closed 2 years ago

zuston commented 2 years ago

Thanks for your great work on this project, especially for elastic Yarn with K8S.

Sorry i'm not familiar with K8S and so confused whether Hadoop LinuxContainerExecutor could be supported on NM runs in Docker natively.

If you have any ideas on it, please share it with me.

ddongchen commented 2 years ago

hi, @zuston. It works with mounting container cgroup path on host into docker container(the docker has already done itself), and also you should give the container the write privileges for cgroup path. If you have any questions, please ask me again.

zuston commented 2 years ago

@ddongchen Thanks for your reply. I found that the Yarn NodeManager in docker works fine when running docker instance with the privilege in my local MAC. But how to enable write privileges for cgroup path in k8s? Could be specified in yaml?

Sorry I'm not familiar with K8S and interested in this project, and i'm also focusing on the elastic resource manager like Yarn on K8S. Looking forward to your reply.

Hope that i can connect you offline, such as developer groups like wechat or slack.

ddongchen commented 2 years ago

@zuston sorry for long time to response，you can send me your wechat number by email, and I will invite you into the group!

zuston commented 2 years ago

Thanks. I have solved it by using kata runtime. @ddongchen