search

apache-spark-on-k8s / spark

Apache Spark enhanced with native Kubernetes scheduler back-end: NOTE this repository is being ARCHIVED as all new development for the kubernetes scheduler back-end is now on https://github.com/apache/spark/
https://spark.apache.org/
Apache License 2.0
612 stars 118 forks source link

Add debugging tools in base image #489

Closed duyanghao closed 7 years ago

duyanghao commented 7 years ago

@mccheah At present, there are few tools in base image that we can use for debug, and the internal base tools like ps and top could do little.

For instance, i want to list all threads of a process,which is pretty meaningful in java debug,but it becomes hard to achieve it as top -Hp pid and ps -Lfp pid are invalid in alpine.

So i guess we could add more powerful base tools(such as enhanced ps and top) in base image for java debug.

Addition: As for list all threads of a process in alpine, i have googled a lot to find the answer and found nothing.

ash211 commented 7 years ago

@duyanghao what additional alpine packages would you find useful to include in the base image?

duyanghao commented 7 years ago

@ash211 I've been looking at necessary and useful tools in alpine but unfortunately found nothing.

But i guess these tools should provide the basic functionality of debugging such as listing all threads of a process.

tangzhankun commented 7 years ago

Add the debug script at the beginning to run these tools in the background is one side, on the other side, you may also need to leave the debug log in a host volume for future analysis. I used to change the code to achieve this in order to debug an issue easily.

duyanghao commented 7 years ago

@tangzhankun What we are talking about here is base-debug tool in alpine.

tangzhankun commented 7 years ago

@duyanghao oh, sorry. I'm talking about when we got the tool, we also need a way to run it and persistent the log.

duyanghao commented 7 years ago

@ash211 @mccheah i will close this issue since different users will have different needs, and many users have no need for those debugging tools at all.

As for me,i will replace alpine with centos as it contains almost everything i need.