Windows image build failed because MS removed directx_redist.exe

[thisislj]

Output of the ue4-docker info command:

ue4-docker version:         0.0.77 (latest available version is 0.0.77)
Operating system:           Windows 10 Enterprise Version 2004 (OS Build 19041.746)
Docker daemon version:      20.10.2
NVIDIA Docker supported:    No
Maximum image size:         400GB
Available disk space:       340.34 GiB
Total system memory:        31.95 GiB physical, 37.45 GiB virtual
Number of processors:       6 physical, 12 logical

Additional details:

• Are you accessing the network through a proxy server? No

Problem description: ue4-docke build failed due to being unable to run downloaded %TEMP%\directx_redist.exe It reported The system cannot execute the specified program.

It seems to me that the directx redist has been removed by Microsoft. So https://download.microsoft.com/download/8/4/A/84A35BF1-DAFE-4AE8-82AF-AD2AE20B6B14/directx_Jun2010_redist.exe is no longer available. The curl command that downloads it didn't report error, so when we tried to execute the redist exe, it reported this unhelpful "The system cannot execute the specified program."

Output of ue4-docker build

ue4-docker build 4.24.3 --no-engine --verbose
[ue4-docker build] COMMAND-LINE INVOCATION:
[ue4-docker build] ['ue4-docker', '4.24.3', '--no-engine', '--verbose']

[ue4-docker build] UNREAL ENGINE VERSION SETTINGS:
[ue4-docker build] Custom build:  No
[ue4-docker build] Release:       4.24.3
[ue4-docker build] Repository:    https://github.com/EpicGames/UnrealEngine.git
[ue4-docker build] Branch/tag:    4.24.3-release

[ue4-docker build] WINDOWS CONTAINER SETTINGS
[ue4-docker build] Isolation mode:               process
[ue4-docker build] Base OS image tag:            2004 (host OS is Windows 10 version 2004)
[ue4-docker build] Memory limit:                 No limit
[ue4-docker build] Detected max image size:      400GB
[ue4-docker build] Directory to copy DLLs from:  C:\WINDOWS\System32

[ue4-docker build] GENERAL SETTINGS
[ue4-docker build] Not excluding any Engine components.

Retrieving the Git credentials that will be used to clone the UE4 repo
Username: thisislj@gmail.com
Password:

[ue4-docker build] Building image "adamrehn/ue4-build-prerequisites:2004"...
Sending build context to Docker daemon  1.863MB
Step 1/27 : ARG BASEIMAGE
Step 2/27 : FROM ${BASEIMAGE} AS dlls
 ---> 683b4603fa04
Step 3/27 : SHELL ["cmd", "/S", "/C"]
 ---> Using cache
 ---> bfbf7edf7025
Step 4/27 : LABEL com.adamrehn.ue4-docker.sentinel="1"
 ---> Using cache
 ---> 6e212a8953d4
Step 5/27 : RUN mkdir C:\GatheredDlls && echo. && echo.RUN directive complete. Docker will now commit the filesystem layer to disk. && echo.Note that for large filesystem layers this can take quite some time. && echo.Performing filesystem layer commit... && echo.
 ---> Using cache
 ---> d8da744d5175
Step 6/27 : RUN powershell -NoProfile -ExecutionPolicy Bypass -Command "iex ((New-Object System.Net.WebClient).DownloadString('https://chocolatey.org/install.ps1'))" && echo. && echo.RUN directive complete. Docker will now commit the filesystem layer to disk. && echo.Note that for large filesystem layers this can take quite some time. && echo.Performing filesystem layer commit... && echo.
 ---> Using cache
 ---> 1e7b333b0dac
Step 7/27 : RUN choco install -y 7zip curl && choco install -y python --version=3.7.5 && echo. && echo.RUN directive complete. Docker will now commit the filesystem layer to disk. && echo.Note that for large filesystem layers this can take quite some time. && echo.Performing filesystem layer commit... && echo.
 ---> Using cache
 ---> b2fd5b1e59ad
Step 8/27 : COPY *.dll C:\GatheredDlls\
 ---> Using cache
 ---> ccdf401a3f50
Step 9/27 : ARG HOST_VERSION
 ---> Using cache
 ---> 4067d26ef2e9
Step 10/27 : RUN pip install pywin32 && echo. && echo.RUN directive complete. Docker will now commit the filesystem layer to disk. && echo.Note that for large filesystem layers this can take quite some time. && echo.Performing filesystem layer commit... && echo.
 ---> Using cache
 ---> f47e8c878152
Step 11/27 : COPY copy.py verify-host-dlls.py C:\
 ---> Using cache
 ---> 697e6ddb95dc
Step 12/27 : RUN C:\copy.py "C:\GatheredDlls\*.dll" C:\Windows\System32\ && echo. && echo.RUN directive complete. Docker will now commit the filesystem layer to disk. && echo.Note that for large filesystem layers this can take quite some time. && echo.Performing filesystem layer commit... && echo.
 ---> Using cache
 ---> 117131c99044
Step 13/27 : RUN python C:\verify-host-dlls.py %HOST_VERSION% C:\GatheredDlls && echo. && echo.RUN directive complete. Docker will now commit the filesystem layer to disk. && echo.Note that for large filesystem layers this can take quite some time. && echo.Performing filesystem layer commit... && echo.
 ---> Using cache
 ---> 9e8411c42878
Step 14/27 : RUN curl --progress -L "https://download.microsoft.com/download/8/4/A/84A35BF1-DAFE-4AE8-82AF-AD2AE20B6B14/directx_Jun2010_redist.exe" --output %TEMP%\directx_redist.exe && echo. && echo.RUN directive complete. Docker will now commit the filesystem layer to disk. && echo.Note that for large filesystem layers this can take quite some time. && echo.Performing filesystem layer commit... && echo.
 ---> Using cache
 ---> fbcf40683717
Step 15/27 : RUN start /wait %TEMP%\directx_redist.exe /Q /T:%TEMP% &&  expand %TEMP%\APR2007_xinput_x64.cab -F:xinput1_3.dll C:\GatheredDlls\ &&       expand %TEMP%\Jun2010_D3DCompiler_43_x64.cab -F:D3DCompiler_43.dll C:\GatheredDlls\ &&  expand %TEMP%\Feb2010_X3DAudio_x64.cab -F:X3DAudio1_7.dll C:\GatheredDlls\ &&   expand %TEMP%\Jun2010_XAudio_x64.cab -F:XAPOFX1_5.dll C:\GatheredDlls\ &&      expand %TEMP%\Jun2010_XAudio_x64.cab -F:XAudio2_7.dll C:\GatheredDlls\ && echo. && echo.RUN directive complete. Docker will now commit the filesystem layer to disk. && echo.Note that for large filesystem layers this can take quite some time. && echo.Performing filesystem layer commit... && echo.
 ---> Running in f891e96330cb
The system cannot execute the specified program.
The command 'cmd /S /C start /wait %TEMP%\directx_redist.exe /Q /T:%TEMP% &&    expand %TEMP%\APR2007_xinput_x64.cab -F:xinput1_3.dll C:\GatheredDlls\ &&       expand %TEMP%\Jun2010_D3DCompiler_43_x64.cab -F:D3DCompiler_43.dll C:\GatheredDlls\ &&  expand %TEMP%\Feb2010_X3DAudio_x64.cab -F:X3DAudio1_7.dll C:\GatheredDlls\ &&   expand %TEMP%\Jun2010_XAudio_x64.cab -F:XAPOFX1_5.dll C:\GatheredDlls\ &&       expand %TEMP%\Jun2010_XAudio_x64.cab -F:XAudio2_7.dll C:\GatheredDlls\ && echo. && echo.RUN directive complete. Docker will now commit the filesystem layer to disk. && echo.Note that for large filesystem layers this can take quite some time. && echo.Performing filesystem layer commit... && echo.' returned a non-zero code: 1

[TBBle]

Sadly, the _directx_Jun2010redist.exe was part of the removal of SHA-1-signed content.

The winetricks resolution is to pull from a mirror, and validate the file with a checksum before running it. That should be doable here, since PowerShell can do the checksumming, I believe, but we'll need to get the checksum from either trusting winetricks's SHA256 value or some other existing source, or someone checksumming a known-good installer in an existing prereqs image's layer.

[adamrehn]

I've just attempted to reproduce this issue, but everything seems to work correctly and the required DLL files are extracted successfully. Manually downloading and inspecting directx_Jun2010_redist.exe on the host system, I can see that its size is reported as 95.6 MB (100,275,120 bytes) and it includes a SHA-256 digital signature from Microsoft Corporation dated ‎Thursday, ‎18 ‎February ‎2021 7:01:21 AM. Presumably Microsoft realised that they had caused problems for software that relied on the DirectX redistributable package and re-signed it to fix the issue.

[TBBle]

Yeah, makes sense. https://www.microsoft.com/en-us/download/details.aspx?id=8109 is back, and shows a published date of 20th February 2021, version 9.29.1974.1. At the time of my last comment, this link was redirecting to https://www.microsoft.com/en-us/download/404Error.aspx, and the package wasn't showing up in MS's site-search at all. The previous version was showing a published date of 18th April 2011, version 9.29.1974.

So they must have just republished it this last weekend. I'm not seeing this picked up by "news" sites that were carrying the removal story yet.

Side-trivia, it looks like they also republished/resigned the relevant DirectX June2010 SDK, but not the DirectX 9.0c End-User Runtime. The latter makes sense, as that was only needed by Windows XP or older, later versions included the DirectX runtime as an OS component.