build: enable building debug binaries and images

[askervin]

Builds debug versions of binaries with "make DEBUG=1" and images with "make DEBUG=1 images". Debug images contain dlv and source files. If a NRI resource policy has been deployed using the debug image, you can attach debugger to it with:

kubectl exec -n kube-system -it nri-resource-policy-POD -- dlv attach 1

For now, the intention is to provide an easy way to build and deploy debuggable images to local e2e test vms. Therefore, debug images are tagged exactly like normal images, and e2e tests use them as is.

[fmuyassarov]

@askervin can you please sign the commit ?

[askervin]

@askervin can you please sign the commit ?

Thanks @fmuyassarov, fixed!

Added the same debug changes to the topology-aware Dockerfile, too. In other words, forgot two things from this PR... :-/

[klihub]

Looking at this, I think it would then be better to either

• generate Dockerfiles from Dockerfile.in's (so we could inject debug-specific additions), or
• have a separate Dockerfile.debug which is then manually kept in sync with the main one

Bet even if we have something like this as a interim solution, I'd prefer to instead emulate a conditional COPY with something like this:

ARG GO_VERSION=1.22

FROM golang:${GO_VERSION}-bullseye AS builder

ARG IMAGE_VERSION
ARG BUILD_VERSION
ARG BUILD_BUILDID
ARG DEBUG=0

WORKDIR /go/builder

# Fetch go dependencies in a separate layer for caching
COPY go.mod go.sum ./
COPY pkg/topology/ pkg/topology/
RUN go mod download

# Build nri-resource-policy
COPY . .

RUN make clean
RUN make IMAGE_VERSION=${IMAGE_VERSION} BUILD_VERSION=${BUILD_VERSION} BUILD_BUILDID=${BUILD_BUILDID} PLUGINS=nri-resource-policy-balloons DEBUG=$DEBUG V=$DEBUG build-plugins-static

RUN mkdir -p /debug-extras
RUN if [ "$DEBUG" = 1 ]; then \
        mkdir -p /debug-extras/bin; \
          GOBIN=/debug-extras/bin go install -tags osusergo,netgo -ldflags "-extldflags=-static" github.com/go-delve/delve/cmd/dlv@latest; \
        mkdir -p /debug-extras/go/builder; \
          mv /go/builder/pkg /debug-extras/go/builder; \
          mv /go/builder/cmd /debug-extras/go/builder; \
        mkdir -p /debug-extras/usr/local/go; \
          mv /go/pkg /debug-extras/go/pkg; \
        mkdir -p /debug-extras/usr/local/go; \
          mv /usr/local/go/src /debug-extras/usr/local/go; \
    fi

FROM gcr.io/distroless/static

COPY --from=builder /go/builder/build/bin/nri-resource-policy-balloons /bin/nri-resource-policy-balloons
COPY --from=builder /debug-extras /

ENTRYPOINT ["/bin/nri-resource-policy-balloons"]

[fmuyassarov]

Looking at this, I think it would then be better to either

* generate Dockerfiles from Dockerfile.in's (so we could inject debug-specific additions), or

* have a separate Dockerfile.debug which is then manually kept in sync with the main one

+1 I think it would make it cleaner.

[askervin]

Thanks @klihub! Definitely cleaner without empty "placeholder files" in production images.

Adopted the debug-extras directory.

Changed the logic so that only go/asm source files are copied, and filtering out obvious test files. This resulted in 500 MB -> 300 MB debug image size reduction.

Motivated by your example, I also benchmarked mv instead of cp. However, as mv lacks "--parents" functionality, running it in a loop for every file to be moved slowed down the source-copy-step from 70 s to 220 s in my build env. And that happened even if I separated directory structure creation under /debug-extras so that the loop did not need to run mkdir for each file. That's the reason for sticking with cp here.

Furthermore, added missing sources from the vendor dir. Now one can see containerd/nri and ttrpc code lines, too, in numerous goroutines....

[klihub]

Motivated by your example, I also benchmarked mv instead of cp. However, as mv lacks "--parents" functionality, running it in a loop for every file to be moved slowed down the source-copy-step from 70 s to 220 s in my build env. And that happened even if I separated directory structure creation under /debug-extras so that the loop did not need to run mkdir for each file. That's the reason for sticking with cp here.

Sure, that mv was just there as the simplest hack to demonstrate the overall idea of the approach. I also think that selective copying is the right way to go.