Provide container image for customers

[theBNT]

Hi,

not sure where to post this but it seems somewhat related, feel free to move around ;)

Idea would be to provide a docker image like devctl but for customers where all required dependencies are installed/maintained already.

This is especially useful for customers that are not natively using linux.

There is a Dockerfile for kubectl-gs and one for kubectl itself (redundant?) so one could base a gsctl image on either of them like

FROM quay.io/giantswarm/kubectl-gs

RUN --mount=type=cache,target=/var/cache/apk/,sharing=locked \
    apk update && apk add --no-cache \
      ca-certificates \
      curl \
      jq \
      yq \
      bash \
      bash-completion \
      git \
      vim \
      tar \
      gnupg \
      apache2-utils \
      kustomize

# install packages from other sources
RUN curl -sS https://webinstall.dev/k9s | bash
RUN curl -s https://fluxcd.io/install.sh | FLUX_VERSION=2.0.0 bash
RUN curl -LO https://github.com/getsops/sops/releases/download/v3.9.0/sops-v3.9.0.linux.amd64 && mv sops-v3.9.0.linux.amd64 /usr/local/bin/sops && chmod +x /usr/local/bin/sops
RUN ( \
    set -x; cd "$(mktemp -d)" && \
    OS="$(uname | tr '[:upper:]' '[:lower:]')" && \
    ARCH="$(uname -m | sed -e 's/x86_64/amd64/' -e 's/\(arm\)\(64\)\?.*/\1\2/' -e 's/aarch64$/arm64/')" && \
    KREW="krew-${OS}_${ARCH}" && \
    curl -fsSLO "https://github.com/kubernetes-sigs/krew/releases/latest/download/${KREW}.tar.gz" && \
    tar zxvf "${KREW}.tar.gz" && \
    ./"${KREW}" install krew neat \
  )

ENV PATH="/root/.krew/bin:$PATH"

RUN kubectl-gs completion bash > /etc/bash_completion.d/kubectl-gs

# enable alias and autocomplete
RUN echo "source <(kubectl completion bash)" >> ~/.bashrc && echo "alias k=kubectl" >> ~/.bashrc && echo "complete -o default -F __start_kubectl k"  >> ~/.bashrc && echo "source ~/.config/envman/load.sh" >> ~/.bashrc && echo "export GPG_TTY=${tty}" >> ~/.bashrc

ENV TERM=linux

RUN mkdir -p /giantswarm
WORKDIR /giantswarm

SHELL ["/bin/bash", "-l", "-c"]

ENTRYPOINT ["/bin/bash"]

Then build this container

docker build -t gsctl .

The customers would then only need to run this container like this (mounting the gnupg folder is important to have the keys persistent between container restarts)

run -it --rm -v ${PWD}/data:/giantswarm -v ${PWD}/.gnupg:/root/.gnupg gsctl

to operate against the management clusters.

What do you think?

[puja108]

This is a nice idea, so thank you creating this.

I also like at least considering some additional tooling (like SOPS) that could help the customer's user experience when working with our API and GitOps.

@weatherhog will take this into our DevEx team and discuss priority and ideas there. They'll also know best, where to move this issue.

[weatherhog]

@marians can you think about this?

[marians]

Hi @theBNT, thanks for sending in your idea!

I was quite confused at first, as gsctl is the name of an old CLI of ours for our Rest API, which we have phased out. I coudn't figure out why you would what that in the mix.

So I f I get this right, you are expecting to run this container as your shell environment for as long as you are working with the management cluster? (As opposed to executing it once for every command, e. g. docker run --rm -ti gsctl kubectl-gs get clusters -- probably via some alias to make it shorter).

[theBNT]

Sorry @marians for reviving old memories by accident ;)

I think its a personal preference how to interact with the container, i prefer the "stay inside" mode, so i have all the additional tools available from the commandline, but i guess that could be solved with an additional entrypoint to bash.

main advantage i see in a centrally maintained image is that the tools one needs to interact with kubectl-gs but also other tools for the gitops workflow (sops, gnupg, kustomize, yp, ...) are simply available, because i struggled a bit to get the "customer dev environment" set up (especially under windows, which route do you go? mingw + windows ports? vm? WSL?).

[puja108]

I personally think the idea of having the "customer dev environment" at least from the GS (GitOps) side in a container sounds nice, as end users would not need to have as many things set up locally.

However, for the team this would definitely include some added maintenance (and take ownership of some additional CLIs at least on a "keep them updated and CVE-free, but compatible"), so I can't say how it fits into their roadmap and plans.

As for Windows, I don't use it regularly, but I have a backup ops environment on my Windows Desktop in WSL2 (Ubuntu-based) just in case my Macbook is broken.

[theBNT]

i totally understand that this is an extra part of GS ecoverse which would need maintenance.

maybe be the opensource hub and host a repository but do not officially support it and have it community maintained? this would also be a good test balloon to see if it finds adoption.

WSL in corporate environments has some additional hiccups like problems with VPN (see https://github.com/microsoft/WSL/issues/1350 which needs something like https://github.com/sakai135/wsl-vpnkit to workaround) but its possible yes.

yet the benefit of having a script to set up the required tools is then still missing. Plus it depends on your WSL distro ;)

so this approach is a twofold gain: use an established common abstraction layer (docker) and provide guidance on how to install GS specific tools (via Dockerfile/shell script).

[puja108]

Indeed, I had a similar issue with VPN, but somehow when I moved to WSL2 it solved itself as long as I initiated the VPN in Windows, I was quite surprised it worked tbh.

And I fully agree, having the docker container would make a lot of things nicer, I've sometimes wanted to be able to do oncalls from any machine I have and could tailscale to, but I don't wanna set up all my tooling everytime.