search

kube-HPC / hkube

🐟 High Performance Computing over Kubernetes - Core Repo 🎣
http://hkube.io
MIT License
305 stars 20 forks source link

Add ability to run algorithms with hostPID option under kubernetess #1166

Closed mattanf2 closed 2 years ago

mattanf2 commented 3 years ago

Hi,

I would like the ability to understand how much GPU memory an algorithm is using as it is running (per running algorithm). This ability will be used for debugging purposes. However, under kubernetes this can only be achieved if the pod is running with a hostPID option. Please add the ability to run with hostPID. At least for debug purposes.

Other alternatives might be hkube itself logging gpu usage per algorithm, however I would still prefer my solution as I can report other statistics along with GPU usage to monitoring systems

stale[bot] commented 2 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.