brigadecore / brigade

Event-driven scripting for Kubernetes
https://brigade.sh/
Apache License 2.0
2.4k stars 247 forks source link

builds of windows components are broken #1846

Closed krancour closed 2 years ago

krancour commented 2 years ago

This isn't really a bug, but it's a problem for sure and I'm tagging it as a bug mainly to denote urgency.

Brigade supports Windows-based jobs if your cluster has Windows nodes. This means two specific components have Windows-based images in addition to the usual Linux-based ones: the logger and the git-initializer. These are the only components that potentially run on a Windows node.

We use Brigade to build itself, but the exception is for those two images. Why? Because currently, it's not yet possible to build a Windows image inside a Windows container, so we need to rely on a VM to host those build jobs. VMs aren't Brigade's jam, so we fall back on GitHub Actions for this.

Our actions for building the Windows-based images use a "windows-latest" VM. It would appear that within the last few days, "windows-latest" has been upgraded and its kernel is no longer compatible with the kernel of the base images we were using. This leads to error messages like the following:

The container operating system does not match the host operating system.

The options for getting these builds working again would seem to be rolling back the version of the VM or rolling forward the version of the base images. I'm not sure where, if anywhere, in the marketplace Windows nodes are available in a k8s cluster other than AKS, so I believe which of the two options we pursue probably depends on what AKS supports. It's even possible we need to do both.