runtime:4.8 image does not work with gMSA

[gpgarcia]

Describe the Bug

Trying to run a framework app using mcr.microsoft.com/dotnet/framework/runtime:4.8 as a base image. Need to run container with a gSMA, but it fails to startup.

Steps to Reproduce

Just try to run base container image.

docker run --rm -it --security-opt "credentialspec=file://svc_acct.json" --user "NT AUTHORITY\SYSTEM" mcr.microsoft.com/dotnet/framework/runtime:4.8

svc_acct.json is the credential spec file for the group managed Service Account.

docker run --rm -it --security-opt "credentialspec=file://svc_acct.json" --user "NT AUTHORITY\SYSTEM" mcr.microsoft.com/windows/servercore:1809

works well

Other Information

Error Message docker: Error response from daemon: container a90eccedf6d4391cbfe747d87ffe7a9b20fc6d550bf3f7885e189fa4cc58ae07 encountered an error during hcsshim::System::CreateProcess: failure in a Windows system call: The user name or password is incorrect. (0x52e) [Event Detail: Provider: 00000000-0000-0000-0000-000000000000].

I tried building the runtime:4.8 image locally, but skipping the ngen steps. That appears to work and it is a smaller image.

Output of docker version

docker version Client: Cloud integration: 1.0.17 Version: 20.10.8 API version: 1.41 Go version: go1.16.6 Git commit: 3967b7d Built: Fri Jul 30 19:58:50 2021 OS/Arch: windows/amd64 Context: default Experimental: true

Server: Docker Engine - Community Engine: Version: 20.10.8 API version: 1.41 (minimum version 1.24) Go version: go1.16.6 Git commit: 75249d8 Built: Fri Jul 30 19:54:29 2021 OS/Arch: windows/amd64 Experimental: false

Output of docker info

docker info Client: Context: default Debug Mode: false Plugins: buildx: Build with BuildKit (Docker Inc., v0.6.1-docker) compose: Docker Compose (Docker Inc., v2.0.0-rc.3) scan: Docker Scan (Docker Inc., v0.8.0)

Server: Containers: 10 Running: 0 Paused: 0 Stopped: 10 Images: 85 Server Version: 20.10.8 Storage Driver: windowsfilter Windows: Logging Driver: json-file Plugins: Volume: local Network: ics internal l2bridge l2tunnel nat null overlay private transparent Log: awslogs etwlogs fluentd gcplogs gelf json-file local logentries splunk syslog Swarm: active NodeID: d6ke8qyah2peuxgonbga3c3c3 Is Manager: true ClusterID: y7dxk3ttdffeoilns7z286dzg Managers: 1 Nodes: 1 Default Address Pool: 10.0.0.0/8 SubnetSize: 24 Data Path Port: 4789 Orchestration: Task History Retention Limit: 5 Raft: Snapshot Interval: 10000 Number of Old Snapshots to Retain: 0 Heartbeat Tick: 1 Election Tick: 10 Dispatcher: Heartbeat Period: 5 seconds CA Configuration: Expiry Duration: 3 months Force Rotate: 0 Autolock Managers: false Root Rotation In Progress: false Node Address: 10.223.227.95 Manager Addresses: 10.223.227.95:2377 Default Isolation: hyperv Kernel Version: 10.0 19043 (19041.1.amd64fre.vb_release.191206-1406) Operating System: Windows 10 Pro Version 2009 (OS Build 19043.1288) OSType: windows Architecture: x86_64 CPUs: 8 Total Memory: 15.88GiB Name: ggarcia1-Deskto ID: X2F3:OGWQ:YNLW:RNPS:VNDQ:HKQH:XNL2:WZWQ:6FBR:NH2C:6Z5R:IINV Docker Root Dir: C:\ProgramData\Docker Debug Mode: false Registry: https://index.docker.io/v1/ Labels: Experimental: false Insecure Registries: 127.0.0.0/8 Live Restore Enabled: false Product License: Community Engine

[mthalman]

I've been unable to repro this. I don't know if this is related to your particular AD or account but the configuration that I have is working fine with the runtime:4.8 image.

docker run --security-opt "credentialspec=file://gmsa.json" --user "NT AUTHORITY\SYSTEM" --rm -it mcr.microsoft.com/dotnet/framework/runtime:4.8

== In container ==
C:\>nltest /sc_verify:<domain>.com
Flags: b0 HAS_IP  HAS_TIMESERV
Trusted DC Name \\<dc-name>.<domain>.com
Trusted DC Connection Status Status = 0 0x0 NERR_Success
Trust Verification Status = 0 0x0 NERR_Success
The command completed successfully

Make sure to review the content at https://docs.microsoft.com/virtualization/windowscontainers/manage-containers/manage-serviceaccounts.

[gpgarcia]

I reviewed the link, and followed the instruction on troubleshooting. The credentialspec file is correct. I have used the gMSA account with the nanoserver image with dotnet core applications previously -- no problems.

Was your configuration a windows 10 pro container host?

I will try to have some one run the test on a windows server 2019 standard container host to see if that is a factor.

[mthalman]

Was your configuration a windows 10 pro container host?

I ran it on a Windows Server 2019 host.

[gpgarcia]

Yes I am on a Windows 10 Pro host. I found some one to run it on a Windows Server 2019 host and it worked.

So the problem is in the Win 10 Host. Would the isolation mode have anything to do with it? Win 10 uses hyper-V isolation and server 2019 uses process, could that be contributing?

This is a pain point because I cannot test the framework apps on my desktop, like I do for the dotnet core apps.

[mthalman]

Have you tried running it with the --isolation=hyperv option?

This appears to be specific to the Windows container implementation so I think you'll have better luck and more expertise in troubleshooting this if you post to the https://github.com/microsoft/Windows-Containers repo.

[gpgarcia]

The win 10 Host defaults to isolation hyperv. That is the only way to run anything on win 10.

As stated above, it runs if I use the servercore image. The problem occurs when using dotnet framework images provide by this project. Further more the problem can be eliminated by removing the ngen step in the image creation.

[mthalman]

I think I might know what the issue is here.

Please run the following command: docker run --rm mcr.microsoft.com/dotnet/framework/runtime:4.8 cmd /c ver

If it returns Version 10.0.14393, then this is a Windows Server 2016 image. Not what you want, I suspect.

This will happen on versions of Win 10 that are above 20H2 that are targeting a multi-arch tag like 4.8 instead of a Windows version-specific tag. Version 20H2 was the last SAC (semi-annual channel) release for Windows Server and so there have been no newer releases that correspond to the Windows Client versions, like 21H2. When you pull a multi-arch tag like 4.8, the Docker client attempts to resolve the image to one whose version matches the host system. If you're running a version newer than Windows 10 version 20H2, it won't find a match. Unfortunately, Docker's default behavior is to fallback to the earliest version that's associated with the tag. In this case, that's Windows Server 2016.

This explains why your colleague was able to get it to work since they were running on a Windows Server 2019 machine which would have resolved 4.8 to a Windows Server 2019 image, because there was a matching version defined in the tag's manifest.

I suspect that if you update the tag you're targeting to be 4.8-windowsservercore-ltsc2019, your issue will be resolved.

[gpgarcia]

Yes, that resolved the observed problem. I used the 4.8-windowsservercore-ltsc2019 tag and things worked as expected.

Thank you for your assistence.