Dynamically load list of Windows base images

[luc122c]

I'm currently running an insider build of Windows so I can't seem to build Windows containers as the latest available tag is 20H2. The list of valid tags seems to be defined here. My OS version is 2009 which seems to be mapped to 20H2 here, however when I try to build, I get the following errors:

[ue4-docker build] Error: building images with a different kernel version than the host, 
[ue4-docker build] but a custom DLL directory has not specified via the `-dlldir=DIR` arg.
[ue4-docker build] The DLL files will be the incorrect version and the container OS will 
[ue4-docker build] refuse to load them, preventing the built Engine from running correctly.

Microsoft publish a list of their tags in JSON format so it should be fairly straight forward to pull this in and parse it i'd imagine. I'd make a PR if I knew Python! The list can be found here: https://mcr.microsoft.com/v2/windows/tags/list. It does include tags for 2009:

• 2009
• 2009-KB4579311
• 2009-KB4579311-amd64
• 2009-amd64

Output of the ue4-docker info command:

$ ue4-docker info
ue4-docker version:         0.0.81 (latest available version is 0.0.81)
Operating system:           Windows 10 Pro for Workstations Version 2009 (OS Build 21332.1010)
Docker daemon version:      20.10.5
NVIDIA Docker supported:    No
Maximum image size:         20GB
Available disk space:       18.74 GiB
Total system memory:        31.92 GiB physical, 36.67 GiB virtual
Number of processors:       4 physical, 8 logical

[slonopotamus]

Okay, there is a dynamic list of Windows tags. But how ue4-docker would understand which Windows Server Core version to map to?

[luc122c]

This list is from that page. Presumably for testing the user could specify with the base tag flag? -basetag BASETAG Windows Server Core base image tag to use for Windows containers (default is the host OS version)

[TBBle]

The base image is not https://hub.docker.com/_/microsoft-windows-servercore, it's https://hub.docker.com/_/microsoft-dotnet-framework-sdk/ tag 4.8-windowsservercore-XXX, see https://github.com/adamrehn/ue4-docker/blob/76fefc7b741827a51f65873ad52e81ea031e1db5/ue4docker/infrastructure/BuildConfiguration.py#L234.

XXX here is currently one of the MS architecture tags, ['ltsc2016', '1709', '1803', 'ltsc2019', '1903', '1909', '2004', '20H2']

There is code here that if your host is running newer than 10.19024.x.x (i.e. newer than 20H2) it's considered an Insider build and uses the latest stable tag.

As it happens, mcr.microsoft.com/dotnet/framework/sdk:4.8 is a multi-arch image which supports ltsc2016, ltsc2019, 1909, 2004 and 20H2, so if we dropped the Windows 10 Home/Pro EOL versions from the currently-supported list, then we could just pull that multi-arch image and it'd get the right thing on the right platforms. (You can see this in the source for those container images too).

> docker run --rm mplatform/mquery:v0.3 mcr.microsoft.com/dotnet/framework/sdk:4.8
Image: mcr.microsoft.com/dotnet/framework/sdk:4.8 (digest: sha256:6259543b9b5f39f4990d3e1fefeb351706f96f0fa6a5e820fafabec2f4aef42a)
 * Manifest List: Yes (Image type: application/vnd.docker.distribution.manifest.list.v2+json)
 * Supported platforms:
   - windows/amd64:10.0.14393.4283
   - windows/amd64:10.0.17763.1817
   - windows/amd64:10.0.18363.1440
   - windows/amd64:10.0.19041.867
   - windows/amd64:10.0.19042.867

I don't know of a repository of builds of the .NET Framework SDK images or .NET Framework Runtime images for Windows Insider builds, even though they exist for the underlying Server Core Insider repository. I assume that the team maintaining the .NET container images aren't interested in insider builds, or at least not interested enough to maintain them.

And right now, there doesn't appear to be a 10.0.21332.1010 tag in the mcr.microsoft.com/windows/servercore/insider tag list, so you're ahead of MS's container team themselves.

[slonopotamus]

I still can't understand what this issue is about. You want ue4-docker to stop using a hardcoded mapping of "os version -> docker image tag"? But how? Yes, we have a list of tags. Yes, we can query host os for its version. But now ue4-docker has to somehow understand that if host os is 2009 then it should use 20H2 base image.

Or you want ue4-docker to allow arbitrary values for -basetag? That is possible, but ue4-docker will most likely have to default to HyperV isolation in such cases (because it doesn't know whether arbitrary tag is compatible with host os).

Or you want something else and I completely missed the point?

[TBBle]

I think the core need here is for https://github.com/adamrehn/ue4-docker/blob/d832c11d8017fcca2f2894b44a1421152aab51db/ue4docker/infrastructure/WindowsUtils.py#L179-L184 to recognise any actual base tag supported by https://hub.docker.com/_/microsoft-windows-servercore, since #144 means we now have a base image where MS are publishing Insider base images, so it's reasonable for users to use that. It would also make it reasonable for environments where there are multiple tags for a particular base image, and for consistency, a different tag is preferred than the current hardcoded list, e.g. 1809 instead of ltsc2019.

And it means, for example, ltsc2022 would usable on release-day via -basetag, before ue4-docker is updated to recognise it as a release.

And testing on Windows Server LTSC 2022 pre-releases could be done now, without patching ue4-docker.

It probably should also improve the logic of the auto-isolation-mode check https://github.com/adamrehn/ue4-docker/blob/d832c11d8017fcca2f2894b44a1421152aab51db/ue4docker/infrastructure/BuildConfiguration.py#L250-L257 to not always use Hyper-V on Insiders, but really be able to match MS's container tag schema against host OS version.

I don't think it makes sense for https://github.com/adamrehn/ue4-docker/blob/d832c11d8017fcca2f2894b44a1421152aab51db/ue4docker/infrastructure/WindowsUtils.py#L150-L158 to change, but am open to discussion; I think to build on an Insider base image, it should be deliberate (via -basetag) rather than automatic (when the currently running OS is not a recognised Windows 10/Windows Server release) but I'm not sure which is the least-surprising result.

[slonopotamus]

Okay, understood. I started working on this issue.

ltsc2022 would usable on release-day

I'm not sure this will work. We need to construct ltsc2016/ltsc2019/ltsc2022 strings somehow in order to use LTSC base images. But what we actually know about host OS is that it is 1607/1809/whatever-number-2022-would-get.

UPD: Or maybe we don't need that logic actually. MS still continues to update 1607 tag in sync with ltsc2016.

[TBBle]

We might need to change how we match OS versions once ltsc2022 is on the table anyway; right now we're using Windows 10 client versions, and mapping those to container tags, but there won't be a Windows Client version that matches ltsc2022.

https://github.com/adamrehn/ue4-docker/blob/d832c11d8017fcca2f2894b44a1421152aab51db/ue4docker/infrastructure/WindowsUtils.py#L160-L170

based on

https://github.com/adamrehn/ue4-docker/blob/d832c11d8017fcca2f2894b44a1421152aab51db/ue4docker/infrastructure/WindowsUtils.py#L93-L98

I guess there will be some value in ReleaseId for LTSC 2022, but I would also assume that value is there in insider builds too; right now we skirt around that by rejecting insider builds based on being newer than 20H2.

So we might need to be matching build release numbers instead, e.g., the lookup table becomes based on WindowsUtils.getWindowsVersion()['patch'], something like

        # This lookup table is based on the list of valid tags from <https://hub.docker.com/r/microsoft/windowsservercore/>
        return {
            '14393': 'ltsc2016',
            '16299': '1709',
            '17134': '1803',
            '17763': 'ltsc2019',
            '18362': '1903',
            '18363': '1909',
            '19041': '2004',
            '19042': '20H2',
            '19043': '21H1'
        # Default value is the current host version, which will work for both regular and insider base images.
        }.get(release, platform.win32_ver()[1])

That said, I had overlooked in my earlier comment that insiders are also a different image, so to build for insiders, both the tag and the repository must be overridden. In the insider case, there are only tags matching platform.win32_ver()[1].

We couldn't use platform.win32_ver()[1] as a tag before #144, because the .NET Framework SDK images weren't tagging that way, only by the ReleaseId.

[slonopotamus]

there won't be a Windows Client version that matches ltsc2022

What do you mean by "matches"? There won't be a Windows Client that can run ltsc2022 images in process isolation mode?

[TBBle]

Yeah, pretty much. I expect that there won't be a Windows 10 (client) release of 10.0.20345.x (or whatever the final build turns out to be, 10.0.20344.x is the latest preview) because the Windows 10 21H1 release is a feature update 10.0.19043.x, and Windows 10 21H2 insider preview is already up to 10.0.21382.x, and they would have to roll back a long way to do a client release to match the Windows Server LTSC 2022 kernel.

There were previously releases of the Windows container image to match the LTSC 2022 kernel up to 10.0.20324.1, which makes sense since one expects to support the full range of Windows containers images (Windows, Server Core, and Nanoserver) hosted on Windows Server LTSC in process isolation mode, but MS's support for the other way 'round has so far seemed to rely on having had matching Windows 10 client and Server release kernels.

That said, it looks like for Server LTSC 2022 onwards, the Windows container has been replaced with a Server container starting with 10.0.20329.1, so the "full" container image is bifurcated along the same lines as Windows Server LTSC/Windows 10, suggesting this isn't a temporary situation.

This is the first Server LTSC release where it was out-of-step with Windows 10 client (and the latter remains in-step with Windows Server SAC), so I might be wrong, and they may have something in the pipeline to address this, but I've not seen any such suggestion around, beyond https://github.com/microsoft/Windows-Containers/issues/36 (which hasn't shown any movement publicly since agreeing that it'd be nice to have in July 2020).

[slonopotamus]

Wow. Thanks for all those links. I need some timeout to grok it.

[slonopotamus]

I wonder what we're still missing in order to mark this as resolved. Ability to use insider images?