Large >2GB Installer created with WiX 3.14.1 with <MediaTemplate EmbedCab="yes"/> fails during install with "The system cannot open the device or file specified."

[david-sitsky]

WiX Version

3.14.1

.NET or MSBuild or Visual Studio Version

Calling light/candle directly

HeatWave Version

n/a

Windows Version

Win10 22H2

Repro Repo

No response

Repro Steps

I have a (complex) WiX file that has been used for many years, but has recently exceeded the 2GB limit I assume due to the single CAB file being created from the single <Media Id="1" Cabinet="file1.cab" EmbedCab="yes"/>. I decided to change it to <MediaTemplate EmbedCab="yes"/> to handle this.

Actual Result

The MSI is successfully created however it fails when run. Here is output from msiexec /i file.msi /L*V output.log:

=== Verbose logging started: 28/06/2024  17:21:14  Build type: SHIP UNICODE 5.00.10011.00  Calling process: C:\WINDOWS\system32\msiexec.exe ===
MSI (c) (08:2C) [17:21:14:370]: Resetting cached policy values
MSI (c) (08:2C) [17:21:14:370]: Machine policy value 'Debug' is 0
MSI (c) (08:2C) [17:21:14:370]: ******* RunEngine:
           ******* Product: c:\work\...\file.msi
           ******* Action: 
           ******* CommandLine: **********
MSI (c) (08:2C) [17:21:14:370]: Client-side and UI is none or basic: Running entire install on the server.
MSI (c) (08:2C) [17:21:14:370]: Grabbed execution mutex.
MSI (c) (08:2C) [17:21:14:385]: Cloaking enabled.
MSI (c) (08:2C) [17:21:14:385]: Attempting to enable all disabled privileges before calling Install on Server
MSI (c) (08:2C) [17:21:14:385]: Incrementing counter to disable shutdown. Counter after increment: 0
MSI (s) (E8:28) [17:21:14:385]: Running installation inside multi-package transaction c:\work\...\file.msi
MSI (s) (E8:28) [17:21:14:385]: Grabbed execution mutex.
MSI (s) (E8:1C) [17:21:14:385]: Resetting cached policy values
MSI (s) (E8:1C) [17:21:14:385]: Machine policy value 'Debug' is 0
MSI (s) (E8:1C) [17:21:14:385]: ******* RunEngine:
           ******* Product: c:\work\...\file.msi
           ******* Action: 
           ******* CommandLine: **********
MSI (s) (E8:1C) [17:21:14:401]: Machine policy value 'DisableUserInstalls' is 0
MSI (s) (E8:1C) [17:21:14:417]: Note: 1: 2203 2: C:\WINDOWS\Installer\inprogressinstallinfo.ipi 3: -2147287038 
MSI (s) (E8:1C) [17:21:14:417]: SRSetRestorePoint skipped for this transaction.
MSI (s) (E8:1C) [17:21:14:417]: Note: 1: 1402 2: HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Policies\Explorer 3: 2 
MSI (s) (E8:1C) [17:21:14:417]: Access database with Impersonation
MSI (s) (E8:1C) [17:21:14:417]: File will have security applied from OpCode.
MSI (s) (E8:1C) [17:21:14:417]: Note: 1: 1310 2: 131 3: c:\WINDOWS\Installer\2c6b089f.msi 
MSI (s) (E8:1C) [17:21:14:417]: MainEngineThread is returning 110
MSI (s) (E8:28) [17:21:14:417]: No System Restore sequence number for this installation.
The system cannot open the device or file specified.
MSI (s) (E8:28) [17:21:14:417]: User policy value 'DisableRollback' is 0
MSI (s) (E8:28) [17:21:14:417]: Machine policy value 'DisableRollback' is 0
MSI (s) (E8:28) [17:21:14:417]: Incrementing counter to disable shutdown. Counter after increment: 0
MSI (s) (E8:28) [17:21:14:417]: Note: 1: 1402 2: HKEY_LOCAL_MACHINE\Software\Microsoft\Windows\CurrentVersion\Installer\Rollback\Scripts 3: 2 
MSI (s) (E8:28) [17:21:14:417]: Note: 1: 1402 2: HKEY_LOCAL_MACHINE\Software\Microsoft\Windows\CurrentVersion\Installer\Rollback\Scripts 3: 2 
MSI (s) (E8:28) [17:21:14:417]: Decrementing counter to disable shutdown. If counter >= 0, shutdown will be denied.  Counter after decrement: -1
MSI (c) (08:2C) [17:21:14:417]: Decrementing counter to disable shutdown. If counter >= 0, shutdown will be denied.  Counter after decrement: -1
MSI (c) (08:2C) [17:21:14:417]: MainEngineThread is returning 110
=== Verbose logging stopped: 28/06/2024  17:21:14 ===

If I change the WiuX file to use <MediaTemplate EmbedCab="no"/>, then I see a number of .cab files created outside the .msi file, and the installer does work. However, I want to have a single .msi for ease of deployment.

Expected Result

No error.

Acknowledgements

• [X] I acknowledge that this is a fully completed bug report. It is not a question or attempt to get help debugging my issue (because those should be sent to Discussions).

[david-sitsky]

It is also worth stating that I originally tried using multiple <media> elements and the same thing happened. An .msi file was built but the same error during installation was reported.

[bevanweiss]

This is more a Microsoft Installer limitation, so there's not much Wix can do about the issue. https://stackoverflow.com/questions/9373988/what-is-the-largest-size-that-a-single-msi-windows-package-installer-file-can-b

Have you tried opening your big MSI with Orca? It might reveal something that could allow you to shrink things down (i.e. duplicated copies of source files etc)

If you're close to the edge, you could potentially try alternative CompressionLevelType settings. I've never used them myself, and they may or may not do good things. But it's worth a crack. https://wixtoolset.org/docs/v3/xsd/wix/media/

[david-sitsky]

Thanks.. I mistakingly thought the 2GB limit just applied for individual CAB files and using multiple CABs would circumvent that, but it seems the MSI format itself also has that limitation anyway.

[bevanweiss]

Thanks.. I mistakingly thought the 2GB limit just applied for individual CAB files and using multiple CABs would circumvent that, but it seems the MSI format itself also has that limitation anyway.

Multiple CABs (external to the MSI) would work, as you've mentioned

If I change the WiuX file to use , then I see a number of .cab files created outside the .msi file, and the installer does work. However, I want to have a single .msi for ease of deployment.

You could use a Burn Bundle which has all the CABs embedded within it, and then at run-time extracts them to a temporary location for the MSI to reference. That would let you ship a single 'package', that is >2GB. If your installer size is really getting up there, then you may want to consider splitting your 'deployment' into multiple packages / MSIs already, and then chaining them together with a Burn Bundle (so that it feels like one installation, but with multiple individually versioned packages)

[david-sitsky]

Thanks for the reply. I created a simple "Burn Bundle" WiX file that references the MSI and the external CAB payloads, but it ultimated still failed with the following from the light/burn command (I have about 8000 files):

light.exe : error LGHT0306 : An error (E_FAIL) was returned while finalizing a CAB file. This most commonly happens when creating a CAB file with more than 65535 files in it. Either reduce the number of files in your installation package or split your installation package's files into more than one CAB file using the Media element.

This seems to match with what is written hee: https://stackoverflow.com/a/55317746, which seems to indicate that there is still a 2GB limit regardless. :(

[david-sitsky]

FWIW.. this is what I used:

<?xml version="1.0" encoding="UTF-8"?>
<Wix xmlns="http://schemas.microsoft.com/wix/2006/wi"
     xmlns:bal="http://schemas.microsoft.com/wix/BalExtension">

  <Bundle Name="Product Name"
          Version="1.0"
          Manufacturer="Company Limited."
          UpgradeCode="6CCC7A29-FAAA-4445-A955-47234546466A">

    <BootstrapperApplicationRef Id="WixStandardBootstrapperApplication.RtfLicense" />

    <Chain>
      <MsiPackage SourceFile="build\releases\installer.msi">
        <Payload SourceFile="build\releases\cab1.cab" />
        <Payload SourceFile="build\releases\cab2.cab" />
        <Payload SourceFile="build\releases\cab3.cab" />
        <Payload SourceFile="build\releases\cab4.cab" />
        <Payload SourceFile="build\releases\cab5.cab" />
        <Payload SourceFile="build\releases\cab6.cab" />
        <Payload SourceFile="build\releases\cab7.cab" />
        <Payload SourceFile="build\releases\cab8.cab" />
        <Payload SourceFile="build\releases\cab9.cab" />
        <Payload SourceFile="build\releases\cab10.cab" />
        <Payload SourceFile="build\releases\cab11.cab" />
        <Payload SourceFile="build\releases\cab12.cab" />
        <Payload SourceFile="build\releases\cab13.cab" />
        <Payload SourceFile="build\releases\cab14.cab" />
        <Payload SourceFile="build\releases\cab15.cab" />
      </MsiPackage>
    </Chain>
  </Bundle>
</Wix>

with the following error, after running candle input.wxs then light -ext WixBalExtension input.wixobj:

light.exe : error LGHT0306 : An error (E_FAIL) was returned while finalizing a CAB file. This most commonly happens when creating a CAB file with more than 65535 files in it. Either reduce the number of files in your installation package or split your installation package's files into more than one CAB file using the Media element.

Have I done something wrong?

[bevanweiss]

You've tried to embed 15 cab files within the Bundle... but have you authored 15 cab files? I think the error might be a little misleading, I think it's just saying that it was looking for a cab file to add to the chain, but it didn't exist.. and if you've authored everything right, then it's most likely that one of the cab files wasn't correctly created (due to having too many files for the cab indexing).

In your MSI Wxs having it set to NOT embed the cab files, and use the MediaTemplate to split them into sub 2GB cabinets. Then in your bundle, have the MSI package, and the payloads as the number of CAB files created from the split into 2GB cabinets (which I'd expect would be ~2 given you mentioned it was only just over the 2GB total sizing). Do you have a large number of files also (i.e. >32767)? That's another Windows Installer default limitation.

Another stackoverflow link for reference https://stackoverflow.com/questions/28443460/how-can-i-avoid-lght0306-when-adding-a-very-large-file-to-a-wix-burn-bootstrappe

So there's a few limitations that you might run into:

1. Per CAB size-limit of 2GB
2. Embedded MSI (i.e. MSI engine + embedded CAB) total size limit of 2GB
3. Total MSI file count 32767 (without schema modifications)
4. EXE bundle total size 4GB

If you author your original MSI (<32767 files) with non-embedded CABs, each limited to <2GB, and then embed these into the Bundle with a total size < 4GB then you should be fine.

Orca on the original MSI should help to identify any issue with file counts or CAB sizes.

The way that a few other vendors handle this is to just ship a self-extracting zip file (i.e. this is how Siemens do it). Their non-installer EXE then unzips everything into a temp directory, and then launches the bundle proper where all the CABs/MSIs are now separately available on disk. I believe the likes of Visual Studio uses a custom bundle which downloads the various CABs/MSIs as required.

[david-sitsky]

The 15 CAB files were automatically created from WiX using <MediaTemplate EmbedCab="no"/>.

I can see running procmon, that all 15 CABs are read as well as the MSI initially. Then each of the CAB files are read again, and data is being written to a single temporary file. It seems once the 2GB offset of the temporary file is hit, then a failure within WiX occurs. I've attached two screenshots from procmon showing it as it works through the CABs, before the failure at cab14.cab, where it hit an offset of 2GB. Could it be there is a bug in WiX in handling file offsets with this temporary file?

[robmen]

IIRC, WiX v4 added support for multiple attached Containers. You may need to split your MSI so it does not require >2GB of content to fit in a Container (as Containers are still CABs).

Note: very large executables (especially when signed) can experience significant launch delays on Windows. I generally do not recommend creating multi-GB .exe files.

[david-sitsky]

Indeed - I think I have reached the point where I will just create a ZIP file containing the MSI and its CAB companions.

[bevanweiss]

Or you could manually author the containers such that each is <2GB https://github.com/wixtoolset/issues/issues/6521

[david-sitsky]

@bevanweiss - given the example file I posted in https://github.com/wixtoolset/issues/issues/8601#issuecomment-2198896948, how would you change this into say 4 containers (each container holding 3 or 4 cab files) using WiX 3? Thanks for the help.

[bevanweiss]

@bevanweiss - given the example file I posted in #8601 (comment), how would you change this into say 4 containers (each container holding 3 or 4 cab files) using WiX 3? Thanks for the help.

I think you should look to move away from the no longer (openly) maintained WiX v3.

It looks like I was wrong about a separate container being able to just hold the payload CABs. It appears that a Container can (currently, as per doco) only hold non-Payload items, so if you had your installer split into individual install packages (MSI/EXE etc), then you could put them each into a separate container, but if you're just trying to add CAB files belonging to an install package.. not supported. https://wixtoolset.org/docs/schema/wxs/packagegroup/

I think @nirbar did develop some alternative ideas around this, but I don't think they've all made their way into WiX.

For new functionality you'd really want to be using a more modern version of WiX.. even if it might not include all of @nirbar 's stuff, it still has more capabilities in most areas than the now legacy v3.

[barnson]

MSIs and CABs are limited to ~2GB and EXEs are limited to ~4GB (and perform poorly at gigabyte sizes).

[latenitefilms]

MSIs and CABs are limited to ~2GB and EXEs are limited to ~4GB (and perform poorly at gigabyte sizes).

Does this mean that upgrading to WiX v4 or v5 wouldn't actually fix the issue if you have a 5GB application?

[nirbar]

The limits on cab and msi sizes are intrinsic, so WiX v4/5 still have the same limits. With msi, you can have any size of application with external cabs With burn, you're limited to 4GB executable size, and can have any number of detached containers

You can use my WiX extension to automatically assign your burn payloads to containers: nuget https://www.nuget.org/packages/PanelSwWixExtension4/ For WiX v4, use PanelSwWixExtension4 version 3.x For WiX v5, use PanelSwWixExtension4 version 5.x

    <PanelSW:ContainerTemplate DefaultType="attached" />

You can use my custom WiX to use 7z compression instead of cabs for containers. It is a drop-in replacement for all nuget references- replace all nugets "WixToolset.XXX" with "PanelSwWix4.XXX": nuget https://www.nuget.org/packages/PanelSwWix4.sdk For WiX v4, use PanelSwWix4.sdk version 5.x For WiX v5, use PanelSwWix4.sdk version 6.x And use PanelSwWixExtension4 with the "-psw-wix" version postfix

    <PanelSW:ContainerTemplate DefaultType="attached" Compression="SevenZip" NameTemplate="$(TargetName).7z"/>

This will create an exe with size up to 4GB, and - if needed - an additional .7z file with unlimited size