Windows vcruntime140.dll error

[ryancinsight]

Pyoxidizer 11.0-pre Windows 10

When redistributing standalone, one of the issues that comes up for users is the vcruntime140.dll:

This is only an issue if the user never installed visual studio or vc_redist however it comes with python in the python/install folder. I will show the results of each below being run on microsoft sandbox.

Therefore, can we add the option to move the vcruntime140.dll over to the distribution as well?

Error on sandbox:

However, copying dll from python folder allows it to work:

[wkschwartz]

This issue is particularly concerning since, if you're running PyOxidizer on Windows, then you necessarily have MSVC installed, so you won't notice that your binary needs an extra file that's not in the install folder.

I'm not sure if standalone_static Python distributions already provide this, but even better than copying vcruntime140.dll would be statically linking the CRT (cl.exe's /MT switch). The problem with standalone_static distributions is that they don't support dynamic linking (DLLs or PYDs) at all. What I'd really like is to link the CRT (and Python, if possible) statically while still supporting dynamic runtime linking to third-party extension modules.

It's possible that python-build-standalone would be a better place for this discussion.

[ryancinsight]

@wkschwartz, probably should have mentioned this before, but this also seems to be a problem with pyoxidizer built with msvc as well due to using the zstd crate. I’m rebuilding pyoxidizer from source with the x86_64-pc-windows-gnu toolchain instead to overcome this since Rust requires using nightly with +crt-static -zunstable to rebuild with runtime included for msvc. I’m also reading there is known instability when going the nightly route.

unfortunately, Python still didn’t see the vcruntime statically linked to pyoxidizer when using nightly either. I noticed in tugger some initial comments about checking for vcruntime and downloading if not found but if pyoxidizer isn’t linked for or bypassed with gnu toolchain than that won’t be much help.

[wkschwartz]

I didn't see anything in there or on zstd's docs about requiring nightly Rust. What issue were you having with x86_64-pc-windows-msvc trying to use +crt-static? Is this related to the PyOxidizer docs claim that Nightly Rust Required on Windows, which I haven't been able to reproduce or verify (I've only been using stable Rust and have had no issues)?

[ryancinsight]

I was receiving errors about zstd being dynamically linked and pyoxidizer failing to recompile at zstd with +crt-static rust flag on windows, if I remember correctly it didn’t support some process. I then used -Zunstable as well, which -Z can only be used with the nightly toolchain, and that allowed zstd to be compiled with +crt-static so that’s the only reason I used nightly. The gnu toolchain worked flawlessly with same flags as linux though.

Nevermind Again, Here is Error with just using +crt-static and not nightly -Zunstable-options: = note: "C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\bin\amd64\link.exe" "/NOLOGO" "/NXCOMPAT" "/LIBPATH:C:\Users\RyanC\.rustup\toolchains\stable-x86_64-pc-windows-msvc\lib\rustlib\x86_64-pc-windows-msvc\lib" "D:\PyOxidizer\target\x86_64-pc-windows-msvc\release\deps\pyoxidizer.pyoxidizer.1mhbkjtm-cgu.6.rcgu.o" "/OUT:D:\PyOxidizer\target\x86_64-pc-windows-msvc\release\deps\pyoxidizer.exe" "/OPT:REF,ICF" "/DEBUG" "/NATVIS:C:\Users\RyanC\.rustup\toolchains\stable-x86_64-pc-windows-msvc\lib\rustlib\etc\intrinsic.natvis" "/NATVIS:C:\Users\RyanC\.rustup\toolchains\stable-x86_64-pc-windows-msvc\lib\rustlib\etc\liballoc.natvis" "/NATVIS:C:\Users\RyanC\.rustup\toolchains\stable-x86_64-pc-windows-msvc\lib\rustlib\etc\libcore.natvis" "/NATVIS:C:\Users\RyanC\.rustup\toolchains\stable-x86_64-pc-windows-msvc\lib\rustlib\etc\libstd.natvis" "/LIBPATH:D:\PyOxidizer\target\x86_64-pc-windows-msvc\release\deps" "/LIBPATH:D:\PyOxidizer\target\release\deps" "/LIBPATH:C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\atlmfc\lib\amd64" "/LIBPATH:D:\PyOxidizer\target\x86_64-pc-windows-msvc\release\build\bzip2-sys-d0685c354ed0bb7c\out\lib" "/LIBPATH:D:\PyOxidizer\target\x86_64-pc-windows-msvc\release\build\ring-8ef846b2cc777550\out" "/LIBPATH:C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\atlmfc\lib\amd64" "/LIBPATH:D:\PyOxidizer\target\x86_64-pc-windows-msvc\release\build\zstd-sys-410d14341ca59f75\out" "/LIBPATH:C:\Users\RyanC\.rustup\toolchains\stable-x86_64-pc-windows-msvc\lib\rustlib\x86_64-pc-windows-msvc\lib" "C:\Users\RyanC\AppData\Local\Temp\rustc9SEXNl\libzstd_sys-ff478c3bd3b416c7.rlib" "C:\Users\RyanC\AppData\Local\Temp\rustc9SEXNl\libring-181aebcb37a15011.rlib" "C:\Users\RyanC\AppData\Local\Temp\rustc9SEXNl\libbzip2_sys-aa9547397d061437.rlib" "C:\Users\RyanC\.rustup\toolchains\stable-x86_64-pc-windows-msvc\lib\rustlib\x86_64-pc-windows-msvc\lib\libcompiler_builtins-d9775021cc0867d6.rlib" "advapi32.lib" "ole32.lib" "oleaut32.lib" "advapi32.lib" "ws2_32.lib" "kernel32.lib" "bcrypt.lib" "advapi32.lib" "cfgmgr32.lib" "credui.lib" "fwpuclnt.lib" "gdi32.lib" "kernel32.lib" "ktmw32.lib" "msimg32.lib" "ntdll.lib" "ole32.lib" "opengl32.lib" "secur32.lib" "shell32.lib" "synchronization.lib" "user32.lib" "winspool.lib" "ws2_32.lib" "advapi32.lib" "ws2_32.lib" "userenv.lib" "libcmt.lib" = note: LINK : warning LNK4098: defaultlib 'MSVCRT' conflicts with use of other libs; use /NODEFAULTLIB:library libzstd_sys-ff478c3bd3b416c7.rlib(zstd_v07.o) : warning LNK4049: locally defined symbol free imported libzstd_sys-ff478c3bd3b416c7.rlib(fse_decompress.o) : warning LNK4049: locally defined symbol free imported libbzip2_sys-aa9547397d061437.rlib(bzlib.o) : warning LNK4049: locally defined symbol free imported libzstd_sys-ff478c3bd3b416c7.rlib(zstd_v03.o) : warning LNK4049: locally defined symbol free imported libzstd_sys-ff478c3bd3b416c7.rlib(zstd_v04.o) : warning LNK4049: locally defined symbol free imported libzstd_sys-ff478c3bd3b416c7.rlib(zstd_v05.o) : warning LNK4049: locally defined symbol free imported libzstd_sys-ff478c3bd3b416c7.rlib(zstd_v06.o) : warning LNK4049: locally defined symbol free imported libzstd_sys-ff478c3bd3b416c7.rlib(zstd_common.o) : warning LNK4217: locally defined symbol free imported in function ZSTD_customFree libzstd_sys-ff478c3bd3b416c7.rlib(xxhash.o) : warning LNK4049: locally defined symbol free imported libzstd_sys-ff478c3bd3b416c7.rlib(zstd_v01.o) : warning LNK4049: locally defined symbol free imported libzstd_sys-ff478c3bd3b416c7.rlib(zstd_v02.o) : warning LNK4049: locally defined symbol free imported libzstd_sys-ff478c3bd3b416c7.rlib(zstd_v07.o) : warning LNK4049: locally defined symbol malloc imported libzstd_sys-ff478c3bd3b416c7.rlib(fse_decompress.o) : warning LNK4049: locally defined symbol malloc imported libbzip2_sys-aa9547397d061437.rlib(bzlib.o) : warning LNK4049: locally defined symbol malloc imported libzstd_sys-ff478c3bd3b416c7.rlib(zstd_v03.o) : warning LNK4049: locally defined symbol malloc imported libzstd_sys-ff478c3bd3b416c7.rlib(zstd_v04.o) : warning LNK4049: locally defined symbol malloc imported libzstd_sys-ff478c3bd3b416c7.rlib(zstd_v05.o) : warning LNK4049: locally defined symbol malloc imported libzstd_sys-ff478c3bd3b416c7.rlib(zstd_v06.o) : warning LNK4049: locally defined symbol malloc imported libzstd_sys-ff478c3bd3b416c7.rlib(zstd_common.o) : warning LNK4217: locally defined symbol malloc imported in function ZSTD_customMalloc libzstd_sys-ff478c3bd3b416c7.rlib(xxhash.o) : warning LNK4049: locally defined symbol malloc imported libzstd_sys-ff478c3bd3b416c7.rlib(zstd_v01.o) : warning LNK4049: locally defined symbol malloc imported libzstd_sys-ff478c3bd3b416c7.rlib(zstd_v02.o) : warning LNK4049: locally defined symbol malloc imported libzstd_sys-ff478c3bd3b416c7.rlib(zstd_v04.o) : warning LNK4049: locally defined symbol memmove imported libzstd_sys-ff478c3bd3b416c7.rlib(zstd_v05.o) : warning LNK4049: locally defined symbol memmove imported libzstd_sys-ff478c3bd3b416c7.rlib(zstd_v06.o) : warning LNK4049: locally defined symbol memmove imported libzstd_sys-ff478c3bd3b416c7.rlib(zstd_v07.o) : warning LNK4049: locally defined symbol memmove imported libzstd_sys-ff478c3bd3b416c7.rlib(zstd_decompress_block.o) : warning LNK4217: locally defined symbol memmove imported in function ZSTD_decompressSequencesLong_body libzstd_sys-ff478c3bd3b416c7.rlib(zstd_v01.o) : warning LNK4049: locally defined symbol memmove imported libzstd_sys-ff478c3bd3b416c7.rlib(zstd_v02.o) : warning LNK4049: locally defined symbol memmove imported libzstd_sys-ff478c3bd3b416c7.rlib(zstd_v03.o) : warning LNK4049: locally defined symbol memmove imported libzstd_sys-ff478c3bd3b416c7.rlib(zstd_common.o) : error LNK2019: unresolved external symbol __imp_calloc referenced in function ZSTD_customCalloc D:\PyOxidizer\target\x86_64-pc-windows-msvc\release\deps\pyoxidizer.exe : fatal error LNK1120: 1 unresolved externals

[wkschwartz]

I've reproduced the original report on my own app. For others' convenience: here's the link to Windows Sandbox

[wkschwartz]

Another solution to consider—and I don't even know enough about this to know if it's really relevant here—is isolated applications and side-by-side components:

Isolated applications and side-by-side assembly sharing can be used to develop applications that safely share operating system assemblies. Developers can use this technology to correct DLL versioning conflicts caused by an incompatible version of a shared assembly.

At least part of the reason I'm trying to use PyOxidizer is to isolate my app from users' settings/environment. Ideally this would include isolating from whatever python3.dll, python38.dll, and vcruntime140.dll the user has.

I'll also note that Microsoft's DLL security advice recommends using manifests to list the specific DLL files that are permitted for an executable. (The URLs suggest that manifests and isolated applications are the same thing... I'm no expert here. Just gathering links in case someone knows more than I do.)

[ryancinsight]

Once I realized that the pyoxidizer we build and the app we want to produce are separate I added a config file to C:\users\*name\.cargo containing, you can ignore linker, which is the final location rust will look for global configurations: [target.x86_64-pc-windows-msvc] linker = "rust-lld" rustflags = ["-C","target-feature=+crt-static"]

The actual app doesn't use zstd so this compiles without errors and need for nightly.

So this will help at least run the app, however to figure out what dependencies I will need I am using either the rust crate copydeps or windows dumpbin /dependents on whatever pyd is failing to import. For example I find some that actually require vcruntime140_1.dll as well.

I think the best bet will be some form of automated dependency walker that copies dlls over next to the exe.

To keep everything contained together and allow easy updates I am using a modified version of warp-packer that uses zstd to store and export everything to a hidden folder in appdata: you can see it here https://github.com/ryancinsight/warp-packer

[wkschwartz]

So this will help at least run the app, however to figure out what dependencies I will need I am using either the rust crate copydeps or windows dumpbin /dependents on whatever pyd is failing to import. For example I find some that actually require vcruntime140_1.dll as well.

Do you mean that placing a .cargo file containing

[target.x86_64-pc-windows-msvc]
linker = "rust-lld"
rustflags = ["-C","target-feature=+crt-static"]

next to pyoxidizer.bzl will cause the C runtime to be linked statically while still allowing for the import of .pyd extension modules? Further, do you mean that the remaining dependencies your app has on vcruntime140_1.dll come from some of the .pyds, or from your .exe?

---

In the meantime I'd like to be able to write a pyoxidizer.bzl that makes a decent guess about where to find vcruntime140.dll. The CRT docs say that

The vcruntime library contains Visual C++ CRT implementation-specific code... is specific to the version of the compiler used.

The file name just has the version number 14.0. It's unclear to me if there are different versions of the file with the same name. Any clues about how to find the right one?

Also thanks for the tip about dumpbin /dependents. See #359.

[ryancinsight]

When you install Rust it should create a global .cargo folder on your c drive, in there you can place the config file that will be used if not designated by project, which you need as pyoxidizer runs cargo build in temp folders so .config folder near .bzl won't help currently:

After this pyoxidizer will execute without needed c runtime, however python will still need it's own so I would consider them separate. Depending on what packages you use you may also need to add other dll's. For my module I needed (deleting ones present on every windows system) dumpbin /dependents _simple.pyd so it can be pyds or exe that have dependencies, but sandbox will say failed to load dll for module if this is the situation: python38.dll RPCRT4.dll (openmp) MSVCP140.dll VCOMP140.DLL VCRUNTIME140.dll VCRUNTIME140_1.dll

The vcruntime140 and vcruntime140_1 come with python though and can be obtained from the python install folder. The rest can be obtained from: C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\redist\x64 though technically you can just install vc_redist.x64.exe and use those instead. That looks like the goal in windows tugger files but it would need to run without it present to start download in first place.

[wkschwartz]

Thanks, I now see that you meant a file named config in my .cargo directory. You can accomplish the same thing locally to a specific project with

> set RUSTFLAGS=-C target-feature=+crt-static
> pyoxidizer build

I then confirmed that vcruntime140.dll does not show up in my .exe's dependents but does show up in python38.dll.

On the one hand it seems like indygreg/python-build-standalone#37, fixed in May 2020, should have solved this problem for standalone_static-flavor distributions (which I think I can't use because I need to be able to import .pyd extension modules), but then I just tried to compile one, and, well, #360.

[wkschwartz]

copydeps is a cool tip, but it wants to copy into my install directory many more DLLs than I need as gauged by what's the minimum set of DLLs needed to get my PyOxidizer-built app working in Windows Sandbox.

copydeps output

I chose `--exedir --search-dir $SYSTEMROOT/system32 --search-dir $WINDIR` based on [Windows's DLL search order](https://docs.microsoft.com/en-us/windows/win32/dlls/dynamic-link-library-search-order#standard-search-order-for-desktop-applications) and ` --ignore 'api-ms-win-(core|crt|eventing|security)(-\w+){1,2}-l[^.]+\.dll'` because those DLLs appear to be somehow built into the Windows-supplied [universal C runtime](https://docs.microsoft.com/en-us/cpp/c-runtime-library/crt-library-features) (which used to be part of `msvcrt.dll`). ```console $ copydeps build/x86_64-pc-windows-msvc/release/install/.exe \ > --exedir --search-dir $SYSTEMROOT/system32 --search-dir $WINDIR \ > --ignore 'api-ms-win-(core|crt|eventing|security)(-\w+){1,2}-l[^.]+\.dll' \ > --no-clobber --dry-run "ADVAPI32.dll": (ignored) "KERNEL32.dll": (ignored) "KERNELBASE.dll": C:\WINDOWS/system32\KernelBase.dll "SHLWAPI.dll": C:\WINDOWS/system32\shlwapi.dll "VCRUNTIME140.dll": C:\WINDOWS/system32\vcruntime140.dll "VERSION.dll": (ignored) "WS2_32.dll": (ignored) "api-ms-win-core-apiquery-l1-1-0.dll": (ignored) "api-ms-win-core-apiquery-l1-1-1.dll": (ignored) "api-ms-win-core-debug-l1-1-0.dll": (ignored) "api-ms-win-core-delayload-l1-1-0.dll": (ignored) "api-ms-win-core-delayload-l1-1-1.dll": (ignored) "api-ms-win-core-errorhandling-l1-1-0.dll": (ignored) "api-ms-win-core-file-l1-1-0.dll": (ignored) "api-ms-win-core-handle-l1-1-0.dll": (ignored) "api-ms-win-core-heap-l1-1-0.dll": (ignored) "api-ms-win-core-heap-l2-1-0.dll": (ignored) "api-ms-win-core-heap-obsolete-l1-1-0.dll": (ignored) "api-ms-win-core-io-l1-1-0.dll": (ignored) "api-ms-win-core-kernel32-legacy-l1-1-0.dll": (ignored) "api-ms-win-core-libraryloader-l1-2-0.dll": (ignored) "api-ms-win-core-libraryloader-l1-2-1.dll": (ignored) "api-ms-win-core-localization-l1-2-0.dll": (ignored) "api-ms-win-core-localization-obsolete-l1-2-0.dll": (ignored) "api-ms-win-core-path-l1-1-0.dll": (ignored) "api-ms-win-core-privateprofile-l1-1-0.dll": (ignored) "api-ms-win-core-processenvironment-l1-1-0.dll": (ignored) "api-ms-win-core-processthreads-l1-1-0.dll": (ignored) "api-ms-win-core-profile-l1-1-0.dll": (ignored) "api-ms-win-core-registry-l1-1-0.dll": (ignored) "api-ms-win-core-registryuserspecific-l1-1-0.dll": (ignored) "api-ms-win-core-rtlsupport-l1-1-0.dll": (ignored) "api-ms-win-core-shlwapi-legacy-l1-1-0.dll": (ignored) "api-ms-win-core-shlwapi-obsolete-l1-1-0.dll": (ignored) "api-ms-win-core-sidebyside-l1-1-0.dll": (ignored) "api-ms-win-core-string-l1-1-0.dll": (ignored) "api-ms-win-core-string-l2-1-0.dll": (ignored) "api-ms-win-core-string-l2-1-1.dll": (ignored) "api-ms-win-core-string-obsolete-l1-1-0.dll": (ignored) "api-ms-win-core-stringansi-l1-1-0.dll": (ignored) "api-ms-win-core-synch-l1-1-0.dll": (ignored) "api-ms-win-core-synch-l1-2-0.dll": (ignored) "api-ms-win-core-sysinfo-l1-1-0.dll": (ignored) "api-ms-win-core-threadpool-legacy-l1-1-0.dll": (ignored) "api-ms-win-core-url-l1-1-0.dll": (ignored) "api-ms-win-core-util-l1-1-0.dll": (ignored) "api-ms-win-core-version-l1-1-0.dll": (ignored) "api-ms-win-core-versionansi-l1-1-0.dll": (ignored) "api-ms-win-crt-conio-l1-1-0.dll": (ignored) "api-ms-win-crt-convert-l1-1-0.dll": (ignored) "api-ms-win-crt-environment-l1-1-0.dll": (ignored) "api-ms-win-crt-filesystem-l1-1-0.dll": (ignored) "api-ms-win-crt-heap-l1-1-0.dll": (ignored) "api-ms-win-crt-locale-l1-1-0.dll": (ignored) "api-ms-win-crt-math-l1-1-0.dll": (ignored) "api-ms-win-crt-process-l1-1-0.dll": (ignored) "api-ms-win-crt-runtime-l1-1-0.dll": (ignored) "api-ms-win-crt-stdio-l1-1-0.dll": (ignored) "api-ms-win-crt-string-l1-1-0.dll": (ignored) "api-ms-win-crt-time-l1-1-0.dll": (ignored) "api-ms-win-crt-utility-l1-1-0.dll": (ignored) "api-ms-win-eventing-provider-l1-1-0.dll": (ignored) "api-ms-win-security-base-l1-1-0.dll": (ignored) "msvcrt.dll": (ignored) "ntdll.dll": C:\WINDOWS/system32\ntdll.dll "python38.dll": .\build\x86_64-pc-windows-msvc\release\install\python38.dll ```

[ryancinsight]

All of the ones that say ignored should not be copied over. However, the only ones it looks like your exe will need are msvcrt.dll and vcruntime...this tells me it didn’t statically link to the c runtime. None of the others standout to me but if you know where they are you can copy them next to executable on sandbox to see if they fix your issue. Try to run your app from command line as well so that you can see where it crashes, it won’t start at all if it’s the exe and it will crash on module if it is one of your packages, I.e. mine are usually on the pyds now so I’ve had to check dependents of those pyds and copy those dll’s over next to the exe as well.

[ryancinsight]

That would be the goal of dependency walker at least, glob all exe, dll, and pyd files in package, check dependencies of each and add unique to a list. Filter the dependencies that are universal or call dependencies in the same respective folders. Then, find and copy whatever is left over next to the main executable.

[ryancinsight]

@wkschwartz I had a chance to start working on the dependency walker, there really isn't any I can find that analyze more than the exe. I don't have filtering applied beyond removing duplicates https://github.com/ryancinsight/depwalker

However, it will list the exe,pyd, and dll's found in the install folder and quote the imports from that dll unless again something else already imported it. i.e.

BBB.exe "python38.dll" "KERNEL32.dll" "ADVAPI32.dll" "MSVCP140.dll" "bcrypt.dll" "VCRUNTIME140.dll" "api-ms-win-crt-stdio-l1-1-0.dll" "api-ms-win-crt-conio-l1-1-0.dll" "api-ms-win-crt-runtime-l1-1-0.dll" "api-ms-win-crt-utility-l1-1-0.dll" "api-ms-win-crt-heap-l1-1-0.dll" "api-ms-win-crt-string-l1-1-0.dll" "api-ms-win-crt-convert-l1-1-0.dll" "api-ms-win-crt-math-l1-1-0.dll" "api-ms-win-crt-locale-l1-1-0.dll"

and for what I needed with pyd files: _SimpleITK.cp38-win_amd64.pyd "RPCRT4.dll" "WSOCK32.dll" "VCOMP140.DLL"

Now their should be a way to stage this to look at exe and pyd files used to remove python3.dll if no package uses it.

Edit: Dressed it up a little more: ("python38.dll", ".\install\python38.dll") ("WS2_32.dll", "Not in folder") ("ADVAPI32.dll", "Not in folder") ("KERNEL32.dll", "Not in folder") ("api-ms-win-crt-runtime-l1-1-0.dll", "Not in folder") ("api-ms-win-crt-environment-l1-1-0.dll", "Not in folder") ("VERSION.dll", "Not in folder") ("python3.dll", ".\install\python3.dll") ("api-ms-win-crt-locale-l1-1-0.dll", "Not in folder") ("api-ms-win-crt-string-l1-1-0.dll", "Not in folder") ("api-ms-win-crt-time-l1-1-0.dll", "Not in folder") ("api-ms-win-crt-utility-l1-1-0.dll", "Not in folder") ("api-ms-win-crt-convert-l1-1-0.dll", "Not in folder") ("SHLWAPI.dll", "Not in folder") ("api-ms-win-crt-heap-l1-1-0.dll", "Not in folder") ("api-ms-win-crt-process-l1-1-0.dll", "Not in folder") ("api-ms-win-crt-filesystem-l1-1-0.dll", "Not in folder") ("api-ms-win-crt-stdio-l1-1-0.dll", "Not in folder") ("api-ms-win-crt-math-l1-1-0.dll", "Not in folder") ("new.exe", ".\install\new.exe") ("VCRUNTIME140.dll", "Not in folder") ("api-ms-win-crt-conio-l1-1-0.dll", "Not in folder")

[ryancinsight]

now corrected to require importation of python3.dll to be listed #359 ("api-ms-win-crt-heap-l1-1-0.dll", "Not in folder") ("api-ms-win-crt-string-l1-1-0.dll", "Not in folder") ("api-ms-win-crt-filesystem-l1-1-0.dll", "Not in folder") ("api-ms-win-crt-convert-l1-1-0.dll", "Not in folder") ("api-ms-win-crt-utility-l1-1-0.dll", "Not in folder") ("WS2_32.dll", "Not in folder") ("api-ms-win-crt-runtime-l1-1-0.dll", "Not in folder") ("api-ms-win-crt-environment-l1-1-0.dll", "Not in folder") ("api-ms-win-crt-stdio-l1-1-0.dll", "Not in folder") ("ADVAPI32.dll", "Not in folder") ("api-ms-win-crt-locale-l1-1-0.dll", "Not in folder") ("VCRUNTIME140.dll", "Not in folder") ("new.exe", ".\install\new.exe") ("VERSION.dll", "Not in folder") ("api-ms-win-crt-math-l1-1-0.dll", "Not in folder") ("api-ms-win-crt-time-l1-1-0.dll", "Not in folder") ("KERNEL32.dll", "Not in folder") ("python38.dll", ".\install\python38.dll") ("api-ms-win-crt-conio-l1-1-0.dll", "Not in folder") ("api-ms-win-crt-process-l1-1-0.dll", "Not in folder") ("SHLWAPI.dll", "Not in folder")

[indygreg]

Over in the commits referenced in #360, I changed the build configuration for standalone_static distributions to use the static CRT. This was done to fix a dynamic/static CRT mismatch between the Python distribution and the Rust-built binary. However, built executables still depend on vcruntime140.dll. Why this is, I'm unsure. Perhaps we need to explicitly link libvcruntime.lib?

For standalone_dynamic, we absolutely need to have a dependency on vcruntime140.dll. Since this is a required dependency not present on freshly installed Windows machines, PyOxidizer should manage this dependency for you.

The msi/exe installer support in main has some support for ensuring the vcruntime140.dll file is installed as part of the application. If you produce an .exe installer for your application it will automatically bundle and run the vc_redist<arch>.exe installer as part of running your app's installer. This is the Microsoft-preferred method for installing vcruntime140.dll, as the DLL gets installed at the system level.

For people not using installers (including local builds), the vcruntime140.dll file can be copied to the same directory as your exe and everything should just work. PyOxidizer does not currently do this. But it should.

The UCRT files are in a similar boat as vcruntime140.dll. However, I believe they are available in some Windows base installs, so there is no explicit dependency to manage in some environments. However, we support down to Windows 8 and the UCRT isn't installed by default there. So like the Visual C++ Redistributable Runtime, we should bundle the UCRT installer in our app installers and copy files alongside the binary outside of managed installation contexts (like local builds).

Regarding other dependencies (like ws2_32.dll, kernel32.dll, and even msvcrt.dll), these are Windows global libraries and should be universally present in any Windows environment. I don't believe we need to worry about them. I only think we need to worry about the vcruntime140.dll VC++ Runtime and api-ms-*.dll UCRT files.

[ryancinsight]

Do the installers access the internet to download vc_redist? Otherwise, I may try use include_bytes to store in my zpack installer for offline systems.

Then, filter the universal dll's. It would be nice to provide a warning of potentially missing dll's, I've had issues with TBB dll's,openmp dll,pytorch had some dll's related to cuda I believe as some examples, which I now catch with dependency walker.

[indygreg]

The WiX .exe installers will bundle the vc_redist installer and then run this file locally during install. The generation of the installer may access the Internet to download the vc_redist installers. But I think that should be fine.

Regarding missing DLLs, that's a separate feature request from this issue. It is in the realm of possibility and I intend to ship that feature someday. There is already some code for DLL dependency scanning in the tugger-binary-analysis crate in the repo. It just needs to get hooked up to various build processes.

[ryancinsight]

Sounds good, I’ll keep working on the zstandard packer with dll copier for now but let me know if there is anyway I can help further. 👍

[wkschwartz]

[W]e support down to Windows 8 and the UCRT isn't installed by default there.... I only think we need to worry about the vcruntime140.dll VC++ Runtime and api-ms-*.dll UCRT files.

I would strongly argue for making the inclusion of the api-ms-*.dlls off by default, and only turn it on with an explicit option in pyoxidizer.bzl. For Windows 10, distributing the api-ms-*.dlls only bloats the distribution because Windows 10 always uses the system copy of the UCRT—even if the bundled copy is newer. Now, Python 3.8 is compatible with Windows 7 and up, and Python 3.9 and 3.10 with Windows 8.1 and up (see PEP 11). Windows 7 is EOL and not supported by PyOxidizer anyway. Just 1.3% of Steam gamers on Windows used Windows 8 or 8.1 as of last month. 0.49% of Passmark's PerformanceTest users running Windows on x86 or x64 hardware were using Windows 8.1 as of today. This suggests that the market for PyOxidized apps that can even use a bundled set of api-ms-*.dlls is just .5-1.5% of the total supported Windows market, and that will decline quickly over time.

Side note

Those surveys also indicate that rather less than 1% of Windows users on still on 32-bit hardware.

Now, I'm aware the api-ms-*.dlls are fairly small (I'm on my Mac right now, but IIRC they're on the order of hundreds of kilobytes). But part of the reason I'm using PyOxidizer in the first place is because I've found that both my junior developers and my end users perceive excess complexity when software comes bundled with tons of unfamiliar looking files. This also raises the burden of debugging (*scrolling through dozens of api-ms-*.dlls* is this DLL that's missing or damaged relevant to this bug? Ok, what about this one?).

---

For people not using installers (including local builds), the vcruntime140.dll file can be copied to the same directory as your exe and everything should just work.

I'm here for the non-installer features. In particular, I would strongly urge you to put the find-the-DLL features in plain vanilla PyOxidizer and not depend on the installer crates.

---

Here's a sketch of how I'm currently finding vcruntime140.dll. I'm using pywin32's win32 package because my code already needs it, but GetModuleFileNameW is also available from Python's undocumented, private _winapi module, or, with some ceremony, from ctypes.windll.kernel32. Using GetModuleFileNameW together with loading a DLL via ctypes allows us to use Windows's real search algorithm. In contrast, ctypes.util.find_library just searches $PATH.

import ctypes
import shutil

from win32.win32api import GetModuleFileNameW  # pywin32

def copy_vcruntime140dll(dst: Union[str, bytes, os.PathLike]) -> None:
    """Copy to `dst` the instance of `vcruntime140.dll` that Python actually loads from Windows."""
    src = GetModuleFileNameW(ctypes.windll.vcruntime140._handle)
    shutil.copy(src, dst)

If writing this code from Rust, you'd probably declare something like the following rather than bringing in a whole new crate.

type DWORD = u32;
type HMODULE = *mut std::ffi::c_void;
type LPWSTR = *mut u16;
extern "stdcall" {
    fn GetModuleFileNameW(HMODULE hModule, LPWSTR lpFilename, DWORD nSize) -> DWORD;
}

(Windows data type reference)

[indygreg]

I just pushed functionality for installing vcruntime140[_1].dll (vcruntime140_1.dll is present in modern versions of the redistributable) automatically when building exes.

Details are documented at https://pyoxidizer.readthedocs.io/en/latest/config_type_python_executable.html#pythonexecutable-windows-runtime-dlls-mode and https://pyoxidizer.readthedocs.io/en/latest/packaging_binary_compatibility.html#packaging-windows-portability.

There is still room for improvement. We don't yet handle the UCRT install (even though it might be a small slice of the population). And we don't attempt to locate the DLLs outside of a Visual Studio installation. But it is a start.

[ryancinsight]

@indygreg and @wkschwartz Hello all, I wanted to run some tests before suggesting this but I do not believe leaving the python packages as is with dll's inside is the way to continue.

First, my program had 10 copies of msvcrt.dll from various packages and the vcruntime that was needed for gui was actually in the package folder so the packages are not fully seeing the dll files.

Second, this got me curious about scipy and numpy .lib folders which only have dll files, I'm able to delete them completely by copying the dlls next to the actual exe, so for these specific cases it looks like moving the dll files next to exe may allow us to place numpy and scipy in memory.

This makes me believe all dll files should be moved next to the exe with duplicates removed and I tested in sandbox to show that numpy .libs can now be deleted and still work properly. Now this makes exe folder noisy but if goblin can save to memory and load at run this can be hidden.

[indygreg]

The matter of where to copy non-extension DLLs is a strictly separate matter from vcruntime140.dll.

We're currently installing them in the paths present in wheels because that seems safest, as that is how the package is built and it may have code telling it where to load DLLs from.

If we put DLLs next to the exe, LoadLibrary() and similar should automatically find it. So things may just work. However, that may break packages. I'm not opposed to it being user-controlled behavior, however.

Could you give a concrete example (with file layouts) of the error you are running into? I'd like to understand what packages are leading to a missing DLL error at run-time.

[ryancinsight]

The package that started this issue was wxpython, which is a gui for python originally written in c++. It would not run without VCruntime140. When you mentioned that most wheels include the dll's they use I realized I never checked and low-and-behold vcruntime140 was in lib/wx/ right next to init and pyd files so it wasn't being picked up for whole package and it seems moving next to exe allows use for wx as well but at that point I don't want duplicates. Wxpython also stores about 11 dll files in the wx package folder as well.

The other package I had issues with was SimpleITK, which was requiring vcruntime140_1.dll so I added that to my list of checks. and I was able to store that package in memory after adding dll, though newest version doesn't require this dll anymore. What I started to realize at this point was that if my package had dll files than I can't store in memory, Wxpython, numpy, pandas, scipy. Scipy is also in .libs and pandas is in lib/pandas/_libs/window. I was also compiling myself so I had tbb dll files, openmp, mkl dlls that weren't being found but this was a swigged package so basically the structure was lib/SimpleITK and it just contained a .pyd and init file.

The third package was Pytorch, which recently started requiring cudnn64_7.dll, this was also not stored but needed to be found in my local nvidia program files.

furthermore, with my dependency walker, my first iteration counted the number of duplicates and I found it annoying that matplotlib, numpy, pandas, scipy, pytorch all had variations of certain copies of vcruntime140, msvcrt, concrt140,msvcp140, etc... In all honesty removing duplicates only saved me 3-5 Mb overall.

Now if .libs is truly a requirement than I shouldn't be able to delete these folders either but I'm able to delete for numpy and scipy anyway and they still work properly with dll files next to exe. There are still other issues like file when trying to put in memory but for other packages this may be helpful to put in memory.

for wxpython, I found this interesting: Placing in memory = no dll installing files to D:\rustbuilds\BBB.\build\x86_64-pc-windows-msvc\release\install Traceback (most recent call last): File "runpy", line 194, in _run_module_as_main File "runpy", line 87, in _run_code File "MRANALYSIS3", line 2, in File "wx", line 17, in File "wx.core", line 12, in ImportError: DLL load failed while importing _core: The specified module could not be found. error: cargo run failed

Place in memory but move dll files next to exe=dll load works but now no file name suggesting we can make a potential placeholder instead and load next to exe (base) D:\rustbuilds\BBB> "D:\rustbuilds\BBB\build\x86_64-pc-windows-msvc\release\install\BBB.exe" Traceback (most recent call last): File "runpy", line 194, in _run_module_as_main File "runpy", line 87, in _run_code File "MRANALYSIS3", line 2, in File "wx", line 17, in File "wx.core", line 319, in NameError: name 'file' is not defined

for numpy, this appears to be the reason why this is fine, it is just adding the folder as a dll directory: ``extra_dll_dir = os.path.join(os.path.dirname(file), '.libs')

if sys.platform == 'win32' and os.path.isdir(extra_dll_dir): if sys.version_info >= (3, 8): os.add_dll_directory(extra_dll_dir) else: os.environ.setdefault('PATH', '') os.environ['PATH'] += os.pathsep + extra_dll_dir``

[ryancinsight]

VTK is another interesting one with file structure that has issues in Pyoxidizer as well: First is run with in-memory for VTK, it looks for a dll and gives a static error if it can't find it, however, I copied the dll files next to exe and now it has issues with the file structure as vtk tries to allow you to import sub-packages separately:

for example: vtk has folder wx with own init but wx is not actually a module in vtk main init

[ryancinsight]

@indygreg This is probably my best example, I started to get worries about dll files in lib and realized ctypes uses libffi-7.dll so I copied this to sandbox with windows shared resources enabled, I then added libffi-7.dll and reimport/tried to delete to show that it was being used.

so it seems using in_memory_shared_library_loading may be broken on windows and mask the fact you don't use those modules often or you have copies of dll elsewhere, libffi is actually on my system so I didn't realize why some of my pyd modules would fail on someone else's computer when using shared_library_loading. In that regard I prefer filtering the modules not being imported at all rather than hiding.