kubectl shovel

Plugin for kubectl that will help you to gather diagnostic info from running in Kubernetes dotnet applications. It can work with .NET Core 3.0+ applications and Kubernetes clusters with docker or containerd runtime.

At the moment the following diagnostic tools are supported:

• dotnet-gcdump
• dotnet-trace
• dotnet-dump
• createdump

Inspired by kubectl-flame.

Installation

Krew

You can install kubectl shovel via krew. At first install krew if you don't have it yet following the guide - Installing. Then you will be able to install shovel plugin:

kubectl krew install shovel

Precompiled binaries

You can find latest release on repository release page. Once you download compatible with your OS binary, move it to any directory specified in your $PATH.

Usage

Feel free to use it as a kubectl plugin or standalone executable (kubectl shovel/kubectl-shovel)

Get gcdump:

kubectl shovel gcdump --pod-name pod-name-74df554df7-qldq7 -o ./dump.gcdump

Or trace:

kubectl shovel trace --pod-name pod-name-74df554df7-qldq7 -o ./trace.nettrace

Or get full managed memory dump:

kubectl shovel dump --pod-name pod-name-74df554df7-qldq7 -o ./memory.dump --type Full

Or get full (managed and unmanaged) memory dump with createdump utility:

kubectl shovel coredump --pod-name pod-name-74df554df7-qldq7 -o ./coredump.dump --type Full

Most of dotnet tools flags supported as well to use, e.g --duration and --format for trace. You can find more info and examples in cli documentation or by using -h/--help flag.

How it works

• For dotnet-trace, dotnet-gcdump, dotnet-dump commands it runs job on pod's node and mounts
  • /tmp folder with dotnet-diagnostic socket
  • docker or containerd runtime folders (/var/lib/docker or /run/containerd)
• For coredump command it runs privileged job (with SYS_PTRACE capability) on pod's node and mounts
  • /tmp folder with dotnet-diagnostic socket
  • docker or containerd runtime folders (/var/lib/docker or /run/containerd)
  • /proc folder mounted from host to find host process id for container

Development

To run all kinds of checks and generators please use:

make prepare

Prerequisites

• golang
• docker
• kind
• dotnet sdk

Testing

Unit tests

make test-unit

Integration tests

kind-clusters use containerd as container runtime, so functionality with docker-runtime won't be covered.

• Integration tests require running kind-cluster. You can create it with make test-integration-setup. Also you can specify some version for cluster: kind create cluster --image=kindest/node:<version>, e.g v1.19.1 version.
• Then run integration tests with make test-integration. It will:
  • Build docker image for dumper
  • Upload it to kind-cluster
  • Build CLI
  • Run tests using these artifacts

All in one

make test