microsoft / dotnet-framework-docker

The repo for the official docker images for .NET Framework on Windows Server Core.
https://hub.docker.com/_/microsoft-dotnet-framework
MIT License
692 stars 333 forks source link

Can no longer add Microsoft.VisualStudio.Component.SQL.SSDTBuildSku component to image #1050

Closed westy closed 1 year ago

westy commented 1 year ago

Describe the Bug

I have been using the 4.8 SDK image to successfully build our .NET Framework application and supporting framework for a couple of months now. Given that we use SSDT for our database projects my docker file has to add the Microsoft.VisualStudio.Component.SQL.SSDTBuildSku component to the SDK image. I found a post recommending how to install the build tools here, and extended it accordingly.

I just added the following to my dockerfile, although I have now recreated in a simple extension of the SDK image to can ask for help.

RUN curl -SL --output vs_buildtools.exe https://aka.ms/vs/17/release/vs_buildtools.exe && \
(start /w vs_buildtools.exe --quiet --wait --norestart --nocache --add Microsoft.VisualStudio.Component.SQL.SSDTBuildSku --installPath "%ProgramFiles(x86)%\Microsoft Visual Studio\2022\BuildTools" || IF "%ERRORLEVEL%"=="3010" EXIT 0) && \
del /q vs_buildtools.exe

This stopped working around a week ago, and I have been trying various methods to get it working again, such as trying lots of different vs_BuildTools downloads, using the various VS installers to add the component, all with no success.

This morning I have been trying to extend your SDK image to add the component there, rather than add the component in my dockerfile, but am still getting issues (hence me finally raising this issue).

Please help!

Steps to Reproduce

The following dockerfile builds (despite appearing to return a response code of 1), but does not add the desired component:

#escape=`

FROM mcr.microsoft.com/dotnet/framework/sdk:4.8-windowsservercore-ltsc2019

SHELL ["cmd", "/S", "/C"]

# Install VS components
RUN `
    # Install SSDT Build Tools
    curl -fSLo vs_BuildTools.exe https://aka.ms/vs/17/release/vs_BuildTools.exe `
    && start /w vs_BuildTools ^ `
        --installPath "%ProgramFiles(x86)%\Microsoft Visual Studio\2022\BuildTools" ^ `
        --add Microsoft.VisualStudio.Component.SQL.SSDTBuildSku ^ `
        --quiet --norestart --nocache --wait `
    && powershell -Command "if ($err = dir $Env:TEMP -Filter dd_setup_*_errors.log | where Length -gt 0 | Get-Content) { throw $err }" `
    && del vs_BuildTools.exe
    #  `
    # `
    # Cleanup
    # && (for /D %i in ("%ProgramFiles(x86)%\Microsoft Visual Studio\Installer\*") do rmdir /S /Q "%i") `
    # && (for %i in ("%ProgramFiles(x86)%\Microsoft Visual Studio\Installer\*") do if not "%~nxi" == "vswhere.exe" del "%~i") `
    # && powershell Remove-Item -Force -Recurse "%TEMP%\*" `
    # && rmdir /S /Q "%ProgramData%\Package Cache"

SHELL ["powershell", "-Command", "$ErrorActionPreference = 'Stop'; $ProgressPreference = 'SilentlyContinue';"]

I am expecting to see an SSDT directory here: C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\MSBuild\Microsoft\VisualStudio\v17.0

Other Information

ddinstaller{timestamp}.log:

C:\Users\ContainerAdministrator\AppData\Local\Temp>type dd_installer_20230306103419.log
[0274:0001][2023-03-06T10:34:19] Setting the locale to en-US.
[0274:0001][2023-03-06T10:34:19] Saving the current locale (en-US) to user.json.
[0274:0001][2023-03-06T10:34:19] Setting the telemetry services
[0274:0003][2023-03-06T10:34:19] Creating a new telemetry service.
[0274:0001][2023-03-06T10:34:19] Visual Studio Installer Version: 3.5.2145
[0274:0001][2023-03-06T10:34:19] Raw Command line: "C:\Program Files (x86)\Microsoft Visual Studio\Installer\setup.exe" /finalizeInstall install --in "C:\ProgramData\Microsoft\VisualStudio\Packages\_bootstrapper\vs_setup_bootstrapper_202303061034086149.json" --installPath "C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools" --add Microsoft.VisualStudio.Component.SQL.SSDTBuildSku --quiet --norestart --nocache --locale en-US --activityId "8905cd9c-4e48-42fc-a3e7-2d9c6de8f38e"
[0274:0001][2023-03-06T10:34:19] Parsed command line options: install --activityId 8905cd9c-4e48-42fc-a3e7-2d9c6de8f38e --add Microsoft.VisualStudio.Component.SQL.SSDTBuildSku --channelId VisualStudio.17.Release --channelUri https://aka.ms/vs/17/release/channel --installPath "C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools" --locale en-US --nocache --norestart --productId Microsoft.VisualStudio.Product.BuildTools --quiet
[0274:0003][2023-03-06T10:34:19] Telemetry session ID: 4d9fc44f-bf05-4580-b87a-c60a3f6b3a15
[0274:0003][2023-03-06T10:34:19] Creating new ExperimentationService
[0274:0003][2023-03-06T10:34:19] Telemetry property VS.ABExp.Flights : lazytoolboxinit;fwlargebuffer;refactoring;spmoretempsbtn1;asloff;keybindgoldbarext;asynccsproj;vsfricheditor;completionapi;typeimportcompletion:30562486;multitenanttasmigration_002:30108253;vsttk:30514303;vsttkdebugcheckerexpcf:30634194;1b1ac847:30622683;87ce9466:30635376;fijh5155:30675898
[0274:0001][2023-03-06T10:34:19] Using registry value of (N/A) to theme installer.
[0274:0001][2023-03-06T10:34:19] Getting singleton lock. Mutex name: DevdivInstallerUI
[0274:0001][2023-03-06T10:34:19] Getting singleton lock succeed.
[0274:0003][2023-03-06T10:34:19] Telemetry property vs.willow.source : WPF
[0274:0003][2023-03-06T10:34:19] Telemetry property vs.willow.locale : en-US
[0274:0003][2023-03-06T10:34:19] Telemetry property vs.willow.installerversion : 3.5.2145.59678
[0274:0003][2023-03-06T10:34:19] Telemetry property vs.willow.startmethod : indirect
[0274:0003][2023-03-06T10:34:19] Telemetry property vs.willow.activityid : 8905cd9c-4e48-42fc-a3e7-2d9c6de8f38e
[0274:0003][2023-03-06T10:34:19] Telemetry property vs.willow.campaign :
[0274:0003][2023-03-06T10:34:19] Telemetry property vs.willow.passive : False
[0274:0003][2023-03-06T10:34:19] Telemetry property vs.willow.quiet : True
[0274:0003][2023-03-06T10:34:19] Telemetry property vs.willow.processtype : ui
[0274:0003][2023-03-06T10:34:19] Telemetry property vs.willow.force : False
[0274:0003][2023-03-06T10:34:19] Telemetry property vs.willow.noweb : False
[0274:0001][2023-03-06T10:34:20] Navigation requested from ApplicationViewModel to MainPageViewModel
[0274:000d][2023-03-06T10:34:22] Telemetry property VS.SetupEngine.ChannelUpdateDisabled : False
[0274:000d][2023-03-06T10:34:22] Trying to remove channel manifest: C:\Users\ContainerAdministrator\AppData\Local\Microsoft\VisualStudio\Packages\_Channels\59f4be23\installChannelManifest.json
[0274:000d][2023-03-06T10:34:22] Trying to remove product manifest: C:\Users\ContainerAdministrator\AppData\Local\Microsoft\VisualStudio\Packages\_Channels\59f4be23\install_catalog.json
[0274:000d][2023-03-06T10:34:22] Authentication Environment variable setting value: null
[0274:000e][2023-03-06T10:34:23] Uri 'https://go.microsoft.com/fwlink/?linkid=857708' redirected to 'https://sendvsfeedback2.azurewebsites.net/api'
[0274:000d][2023-03-06T10:34:23] Authentication remote setting value: False
[0274:000d][2023-03-06T10:34:23] Download requested: https://aka.ms/vs/17/release/channel
[0274:0010][2023-03-06T10:34:23] Attempting download 'https://aka.ms/vs/17/release/channel' using engine 'WebClient'
[0274:000a][2023-03-06T10:34:23] Telemetry property vs.setup.WorkloadOverrides.DidReceiveOverride : False
[0274:000a][2023-03-06T10:34:23] Telemetry property vs.setup.WorkloadOverrides.RuleId : None
[0274:0010][2023-03-06T10:34:23] Uri 'https://aka.ms/vs/17/release/channel' redirected to 'https://download.visualstudio.microsoft.com/download/pr/07db0e25-01f0-4ac0-946d-e03196d2cc8b/79eac302f46c986ed0f5371b9b15546945fb938910e00fc4a27f67f9e8e4cb56/VisualStudio.17.Release.chman'
[0274:0010][2023-03-06T10:34:23] ManifestVerifier Result: Success
[0274:0010][2023-03-06T10:34:23] Download of 'https://aka.ms/vs/17/release/channel' succeeded using engine 'WebClient'
[0274:000f][2023-03-06T10:34:23] The downloaded channel C:\Users\ContainerAdministrator\AppData\Local\Microsoft\VisualStudio\Packages\_Channels\59f4be23\channelManifest.json is older than the existing one.
[0274:000f][2023-03-06T10:34:23] Channel manifest: C:\Users\ContainerAdministrator\AppData\Local\Microsoft\VisualStudio\Packages\_Channels\59f4be23\catalog.json was being deleted because it is outdated.
[0274:000f][2023-03-06T10:34:23] Downloading installable manifest from "https://download.visualstudio.microsoft.com/download/pr/07db0e25-01f0-4ac0-946d-e03196d2cc8b/952ccbe7cd37352fe5537063c91cd3194b394faf64d87e8d685cfb9177376771/VisualStudio.vsman" and using the channel URI "https://aka.ms/vs/17/release/channel".
[0274:000f][2023-03-06T10:34:23] Download requested: https://download.visualstudio.microsoft.com/download/pr/07db0e25-01f0-4ac0-946d-e03196d2cc8b/952ccbe7cd37352fe5537063c91cd3194b394faf64d87e8d685cfb9177376771/VisualStudio.vsman
[0274:0011][2023-03-06T10:34:23] Attempting download 'https://download.visualstudio.microsoft.com/download/pr/07db0e25-01f0-4ac0-946d-e03196d2cc8b/952ccbe7cd37352fe5537063c91cd3194b394faf64d87e8d685cfb9177376771/VisualStudio.vsman' using engine 'WebClient'
[0274:0011][2023-03-06T10:34:25] ManifestVerifier Result: Success
[0274:0011][2023-03-06T10:34:25] Download of 'https://download.visualstudio.microsoft.com/download/pr/07db0e25-01f0-4ac0-946d-e03196d2cc8b/952ccbe7cd37352fe5537063c91cd3194b394faf64d87e8d685cfb9177376771/VisualStudio.vsman' succeeded using engine 'WebClient'
[0274:000a][2023-03-06T10:34:25] Download requested: https://go.microsoft.com/fwlink/?linkid=2066144
[0274:0011][2023-03-06T10:34:25] Attempting download 'https://go.microsoft.com/fwlink/?linkid=2066144' using engine 'WebClient'
[0274:000f][2023-03-06T10:34:25] Created shortcut: C:\ProgramData\Microsoft\Windows\Start Menu\Programs\Visual Studio Installer.lnk
[0274:000f][2023-03-06T10:34:25] Starting C:\Program Files (x86)\Microsoft Visual Studio\Installer\vs_installer.windows.exe with arguments /finalizeinstall 6F320B93-EE3C-4826-85E0-ADF79F8D4C61 "Visual Studio Installer" "Microsoft Visual Studio Installer" 3.5.2145.59678 0 "C:\Program Files (x86)\Microsoft Visual Studio\Installer\setup.exe"
[0274:000f][2023-03-06T10:34:25] Finalizer completed successfully
[0274:0011][2023-03-06T10:34:25] Uri 'https://go.microsoft.com/fwlink/?linkid=2066144' redirected to 'https://vsstartpagenewsfeed.azureedge.net/news/vs'
[0274:0011][2023-03-06T10:34:25] Download of 'https://go.microsoft.com/fwlink/?linkid=2066144' succeeded using engine 'WebClient'
[0274:000d][2023-03-06T10:34:26] Warning: No previous catalog found at 'C:\ProgramData\Microsoft\VisualStudio\Packages\_Instances\5fd55259\catalog.previous.json'
[0274:000d][2023-03-06T10:34:27] Warning: No previous catalog found at 'C:\ProgramData\Microsoft\VisualStudio\Packages\_Instances\fec4c9b5\catalog.previous.json'
[0274:000a][2023-03-06T10:34:27] Status changed to UpdateAvailable
[0274:0011][2023-03-06T10:34:27] Download requested: https://aka.ms/vs/17/release/channel
[0274:0013][2023-03-06T10:34:27] Attempting download 'https://aka.ms/vs/17/release/channel' using engine 'WebClient'
[0274:0013][2023-03-06T10:34:27] Uri 'https://aka.ms/vs/17/release/channel' redirected to 'https://download.visualstudio.microsoft.com/download/pr/07db0e25-01f0-4ac0-946d-e03196d2cc8b/79eac302f46c986ed0f5371b9b15546945fb938910e00fc4a27f67f9e8e4cb56/VisualStudio.17.Release.chman'
[0274:0013][2023-03-06T10:34:27] ManifestVerifier Result: Success
[0274:0013][2023-03-06T10:34:27] Download of 'https://aka.ms/vs/17/release/channel' succeeded using engine 'WebClient'
[0274:000d][2023-03-06T10:34:27] Channel manifest for https://aka.ms/vs/17/release/channel is already up to date.
[0274:000d][2023-03-06T10:34:27] Warning: Shutting down the application with exit code 1
[0274:000d][2023-03-06T10:34:27] Warning: Visual Studio Build Tools 2022 (2) is already installed.
[0274:0001][2023-03-06T10:34:27] Releasing singleton lock.
[0274:0001][2023-03-06T10:34:27] Releasing singleton lock succeed.
[0274:0001][2023-03-06T10:34:27] Releasing singleton lock.
[0274:0001][2023-03-06T10:34:27] Singleton lock does not exist. Releasing singleton lock skipped.
[0274:0001][2023-03-06T10:34:27] Closing the installer with exit code 1
[0274:0001][2023-03-06T10:34:27] Exit Code: 1
[0274:0001][2023-03-06T10:34:27] Cleared previous session ID.
[0274:000f][2023-03-06T10:34:27] Download requested: https://aka.ms/vs/17/release/channel
[0274:0014][2023-03-06T10:34:27] Attempting download 'https://aka.ms/vs/17/release/channel' using engine 'WebClient'
[0274:0001][2023-03-06T10:34:27] Trying to remove channel manifest: C:\Users\ContainerAdministrator\AppData\Local\Microsoft\VisualStudio\Packages\_Channels\59f4be23\installChannelManifest.json
[0274:0001][2023-03-06T10:34:27] Trying to remove product manifest: C:\Users\ContainerAdministrator\AppData\Local\Microsoft\VisualStudio\Packages\_Channels\59f4be23\install_catalog.json
[0274:0014][2023-03-06T10:34:27] Download of 'https://aka.ms/vs/17/release/channel' succeeded using engine 'WebClient'
[0274:0014][2023-03-06T10:34:27] Channel 'https://aka.ms/vs/17/release/channel' was not modified. IfModifiedSince: 2/25/2023 1:45:59 AM +00:00, LastModifiedDate: 2/25/2023 1:45:59 AM +00:00
[0274:0014][2023-03-06T10:34:27] Channel manifest for https://aka.ms/vs/17/release/channel is already up to date.
[0274:0014][2023-03-06T10:34:27] ManifestVerifier Result: Success
[0274:0014][2023-03-06T10:34:27] Download requested: https://aka.ms/vs/channels
[0274:0013][2023-03-06T10:34:27] Attempting download 'https://aka.ms/vs/channels' using engine 'WebClient'
[0274:0013][2023-03-06T10:34:27] Uri 'https://aka.ms/vs/channels' redirected to 'https://download.visualstudio.microsoft.com/download/pr/6e42d917-9e3e-40f2-9e07-7016118ad88c/ed6b5bdbd24dc3b12d60cb246ae56b37/channels.json'
[0274:0013][2023-03-06T10:34:27] ManifestVerifier Result: Success
[0274:0013][2023-03-06T10:34:27] Download of 'https://aka.ms/vs/channels' succeeded using engine 'WebClient'
[0274:0013][2023-03-06T10:34:27] ManifestVerifier Result: Success
[0274:0013][2023-03-06T10:34:27] ManifestVerifier Result: Success
[0274:000d][2023-03-06T10:34:27] Status changed to UpdateAvailable

Output of docker version

Client:
 Version:           23.0.1
 API version:       1.42
 Go version:        go1.19.5
 Git commit:        a5ee5b1
 Built:             Thu Feb  9 19:50:24 2023
 OS/Arch:           windows/amd64
 Context:           default

Server: Docker Engine - Community
 Engine:
  Version:          23.0.1
  API version:      1.42 (minimum version 1.24)
  Go version:       go1.19.5
  Git commit:       bc3805a0
  Built:            Thu Feb  9 19:46:43 2023
  OS/Arch:          windows/amd64
  Experimental:     false

Output of docker info

Client:
 Context:    default
 Debug Mode: false

Server:
 Containers: 7
  Running: 1
  Paused: 0
  Stopped: 6
 Images: 1112
 Server Version: 23.0.1
 Storage Driver: windowsfilter
  Windows:
 Logging Driver: json-file
 Plugins:
  Volume: local
  Network: ics internal l2bridge l2tunnel nat null overlay private transparent
  Log: awslogs etwlogs fluentd gcplogs gelf json-file local logentries splunk syslog
 Swarm: inactive
 Default Isolation: hyperv
 Kernel Version: 10.0 19045 (19041.1.amd64fre.vb_release.191206-1406)
 Operating System: Microsoft Windows Version 22H2 (OS Build 19045.2604)
 OSType: windows
 Architecture: x86_64
 CPUs: 8
 Total Memory: 15.74GiB
 Name: A00173
 ID: XTSX:MQ6I:2CUH:JSRM:JCNP:KINQ:4O4F:HO7L:7YEH:I4TG:4LHD:ENCG
 Docker Root Dir: D:\docker-data
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false
 Product License: Community Engine
westy commented 1 year ago

This here appears to work though!

#escape=`

FROM mcr.microsoft.com/dotnet/framework/sdk:4.8-windowsservercore-ltsc2019

SHELL ["cmd", "/S", "/C"]

# Install VS components
RUN `
    # Install SSDT Build Tools
    curl -fSLo vs_BuildTools.exe https://aka.ms/vs/17/release/vs_BuildTools.exe `
    && start /w vs_BuildTools ^ `
        --installPath "%ProgramFiles(x86)%\Microsoft Visual Studio\2022\BuildTools" ^ `
        --add Microsoft.VisualStudio.Component.SQL.SSDTBuildSku ^ `
        --quiet --norestart --nocache --wait `
    && powershell -Command "if ($err = dir $Env:TEMP -Filter dd_setup_*_errors.log | where Length -gt 0 | Get-Content) { throw $err }" `
    && del vs_BuildTools.exe

# Try and force install using now installed vs_installer
WORKDIR "C:/Program Files (x86)/Microsoft Visual Studio/installer"
RUN vs_installer.exe `
    modify --quiet --nocache --add Microsoft.VisualStudio.Component.SQL.SSDTBuildSku --installPath "%ProgramFiles(x86)%\Microsoft Visual Studio\2022\BuildTools"

RUN `
    # Cleanup
    (for /D %i in ("%ProgramFiles(x86)%\Microsoft Visual Studio\Installer\*") do rmdir /S /Q "%i") `
    && (for %i in ("%ProgramFiles(x86)%\Microsoft Visual Studio\Installer\*") do if not "%~nxi" == "vswhere.exe" del "%~i") `
    && powershell Remove-Item -Force -Recurse "%TEMP%\*"
    # `
    # && rmdir /S /Q "%ProgramData%\Package Cache"

WORKDIR /
SHELL ["powershell", "-Command", "$ErrorActionPreference = 'Stop'; $ProgressPreference = 'SilentlyContinue';"]

So, at least I can progress. Be interested to know why the vs_buildtools route does not work though!

mthalman commented 1 year ago

This is how I typically add more components: https://github.com/microsoft/dotnet-framework-docker/issues/979#issuecomment-1208413978. I notice you don't use the modify in the first install command. Can you try that?

Also, there are a lot of log files output to the TEMP directory. When there's an install failure, I recommend looking through all of them.

westy commented 1 year ago

Thanks for the response. I'll try it with modify. I was using that when trying the various VS installers (but no luck there either), but didn't see it as an option for vs_BuildTools.

I have captured a LOT of logs from %TEMP% and diffed them many times. Nothing much useful that I can see other than the installer one I have posted.

I'll report back with progress.

Cheers

westy commented 1 year ago

Oh blimey, thank you, that was it!

Very odd that was working fine for ages without it. Thanks, I'll close this now.

Wish I'd asked a week ago now :D