adang1345 / delvewheel

Self-contained Python wheels for Windows
MIT License
116 stars 12 forks source link

Bundling Third Party DLL with executable #53

Closed WillAyd closed 1 month ago

WillAyd commented 1 month ago

Thank you for delvewheel - this library rocks!

I have a project called pantab that bundles both a dll and an executable from a third party vendor. The dll and executable are closed source, but the vendor has noted that there are stricted requirements to how those two objects are placed relative to one another.

The vendor issues their own pip package (tableauhyperapi), and unfortunately the ABI between the version that I vendor and what they provide in their own package is not guaranteed to be stable. On Unix systems, I am using RPATH to ensure that my application always refers to the bundled library, but I cannot get that same behavior working on Windows.

Without knowing a lot about Windows, I am under the impression that delvewheel does something to ensure the bundled copy is loaded instead of any other library. Due to the strict requirements on how the dll and exe need to be laid out relative to one another, I have a CMake script that installs those files in the right location, and with delvewheel I then run:

python -m delvewheel repair -v --ignore-in-wheel -w {dest_dir} {wheel}

When trying this in CI, the repair step "works" but importing the library throws an access violation.

Is my way of handling this incorrect? Does the --ignore-in-wheel option prevent delvewheel from fixing up load paths so that my application uses the bundled library? If delvewheel needs to be responsible for copying in the .dll, can it move the associated .exe as well?

adang1345 commented 1 month ago

Try adding the argument --add-path C:/Windows/System32. I took a look at https://github.com/innobi/pantab/actions/runs/10422256854/job/28866337713?pr=303, and I noticed that the version of msvcp140.dll that is bundled is coming from C:\hostedtoolcache\windows\Java_Temurin-Hotspot_jdk\8.0.422-5\x64\bin\msvcp140.dll, which is likely too old. Another recent report of an access violation (https://github.com/adang1345/delvewheel/issues/52) was found to be due to a bundled version of msvcp140.dll that was too old.

WillAyd commented 1 month ago

@adang1345 I've been struggling with this issue for weeks and you solved it in the matter of minutes. Amazing!

So I take it delvewheel still does modify how the linker resolves to the bundled library, even with the --ignore-in-wheel option?

adang1345 commented 1 month ago

delvewheel does some tricks to force the bundled copies of DLLs to be loaded instead of some other library on the system. The --ignore-in-wheel option causes delvewheel to exclude certain DLLs from this mechanism: the DLLs that are already in the wheel and their direct dependencies.

WillAyd commented 1 month ago

Thanks for that explanation. I'm guessing I shouldn't be installing the vendor library with CMake then, and instead should have delvewheel copy that (alongside the executable).

Happy to close this for now though - seems like the more immediate issue I was facing had to do with the msvcp140.dll. Thanks again!