search

microsoft / dotnet-framework-docker

The repo for the official docker images for .NET Framework on Windows Server Core.
https://hub.docker.com/_/microsoft-dotnet-framework
MIT License
710 stars 335 forks source link

Container creation fails #856

Closed ramakrpr closed 2 years ago

ramakrpr commented 2 years ago

Describe the Bug

Hi,

Has there been any recent changes to the image mcr.microsoft.com/dotnet/framework/sdk:4.8-windowsservercore-ltsc2019 I see that suddenly the container creation fails. I thought initially that it might be related to the packeges we add on top of this image for our deployment to AKS, but then even with just a plain image reference i see that the container creation breaks. Sample YAML reference below. And i am doing this in AKS

Steps to Reproduce


kind: Deployment
metadata:
  name: testimagedeployment
  labels:
    app: tstimgdepapp
spec:
  replicas: 1
  selector:
    matchLabels:
      app: tstimgdepapp
  template:
    metadata:
      labels:
        app: tstimgdepapp
    spec:
      containers:
      - name: tstimgdepapp
        image: mcr.microsoft.com/dotnet/framework/sdk:4.8-windowsservercore-ltsc2019
      nodeSelector:
        agentpool: winpl1

### Other Information

i jsust get the below details from the container and nothing specific from the logs

Name:         testimagedeployment-6676bc94d-hsj65
Namespace:    default
Priority:     0
Node:         akswinpl1000007/10.130.30.132
Start Time:   Thu, 25 Nov 2021 21:16:16 +0100
Labels:       app=tstimgdepapp
              pod-template-hash=6676bc94d
Annotations:  <none>
Status:       Running
IP:           10.130.30.162
IPs:
  IP:           10.130.30.162
Controlled By:  ReplicaSet/testimagedeployment-6676bc94d
Containers:
  tstimgdepapp:
    Container ID:   docker://330efdf066490a7ec4c3f871d7dc647e44cb96353e15e468b021199b761304a5
    Image:          mcr.microsoft.com/dotnet/framework/sdk:4.8-windowsservercore-ltsc2019
    Image ID:       docker-pullable://mcr.microsoft.com/dotnet/framework/sdk@sha256:f1915874869d31d4ba9002478809248b7661937d28a01156b12e4c72558ec8f2
    Port:           <none>
    Host Port:      <none>
    State:          Waiting
      Reason:       CrashLoopBackOff
    Last State:     Terminated
      Reason:       Completed
      Exit Code:    0
      Started:      Thu, 25 Nov 2021 21:16:43 +0100
      Finished:     Thu, 25 Nov 2021 21:16:43 +0100
    Ready:          False
    Restart Count:  2
    Environment:    <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-mf7hl (ro)
Conditions:
  Type              Status
  Initialized       True
  Ready             False
  ContainersReady   False
  PodScheduled      True
Volumes:
  default-token-mf7hl:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-mf7hl
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  agentpool=winpl1
Tolerations:     node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                 node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type     Reason     Age                From               Message
  ----     ------     ----               ----               -------
  Normal   Scheduled  57s                default-scheduler  Successfully assigned default/testimagedeployment-6676bc94d-hsj65 to akswinpl1000007
  Warning  BackOff    16s (x4 over 44s)  kubelet            Back-off restarting failed container
  Normal   Pulled     5s (x4 over 55s)   kubelet            Container image "mcr.microsoft.com/dotnet/framework/sdk:4.8-windowsservercore-ltsc2019" already present on machine
  Normal   Created    5s (x4 over 55s)   kubelet            Created container tstimgdepapp
  Normal   Started    3s (x4 over 53s)   kubelet            Started container tstimgdepapp
Warning  BackOff    36s (x25 over 5m32s)  kubelet            Back-off restarting failed container

### Output of `docker version`

docker://20.10.6
mthalman commented 2 years ago

What do you consider to be "recent"? That tag was last updated on Nov 9. These are the two key changes to the image aside from any updates to Windows itself:

ramakrpr commented 2 years ago

What do you consider to be "recent"? That tag was last updated on Nov 9. These are the two key changes to the image aside from any updates to Windows itself:

Well we have the same image being used in PROD and the pods are running since 26 days , so i believe an issue related to the Nov 9 update that we cannot bring our pods up in Test environment anymore

mthalman commented 2 years ago

Just to be certain, could you please verify that the previous version of the image continues to work? You can use the timestamp tag for the previous version: 4.8-20211012-windowsservercore-ltsc2019

ramakrpr commented 2 years ago

Just to be certain, could you please verify that the previous version of the image continues to work? You can use the timestamp tag for the previous version: 4.8-20211012-windowsservercore-ltsc2019

This one 4.8-20211012-windowsservercore-ltsc2019 as well breaks with the same error

mthalman commented 2 years ago

Ok, then the issues I linked to above aren't relevant here. It seems possible there was a change in your environment or configuration that caused this to no longer work, rather than the image.

If you'd like to still check whether it was a change to the image that caused this, here are the most recent tags for each past month's releases.

If all of these tags yield the same error in your environment, then something must have changed in your environment or configuration.

mthalman commented 2 years ago

@ramakrpr - Any follow up on narrowing down which tag caused the regression?

ramakrpr commented 2 years ago

@ramakrpr - Any follow up on narrowing down which tag caused the regression?

yes, i managed to figure out that the issue is something related to the way the VC++ build tools needs to be installed. if i follow for example the below guidance - it is no more working. As i mentioned earlier, it used to work before.

https://docs.microsoft.com/en-us/visualstudio/install/build-tools-container?view=vs-2022

I see that the installation never happens even though there is a 'start /wait' and no downloads as well. Hence the file mentioned in the ENTRYPOINT - VsDevCmd.bat is missing and the container fails to initialize. I managed to fix this by an alternative approach which seems working every time as below;

RUN start /w C:\Temp\vs_buildtools.exe --quiet --wait --norestart --nocache --installPath C:\BuildTools --channelUri C:\Temp\VisualStudio.chman --installChannelUri C:\Temp\VisualStudio.chman --add Microsoft.VisualStudio.Workload.VCTools --includeRecommended --add Microsoft.Component.MSBuild IF "%ERRORLEVEL%"=="3010" EXIT 0

offcource after downloading the vs_buildtools.exe and VisualStudio.chman explicitly via powershell using invoke-webrequest. Maybe its good to revisit this document, the example is for Microsoft.VisualStudio.Workload.AzureBuildTools and in my case it was for Microsoft.VisualStudio.Workload.VCTools

mthalman commented 2 years ago

Glad to hear you got things working. I'll follow-up on that documentation.