search

microsoft / vcpkg

C++ Library Manager for Windows, Linux, and MacOS
MIT License
23.27k stars 6.42k forks source link

Windeployqt from vcpkg does not deploy all required libraries #21044

Open Shatur opened 3 years ago

Shatur commented 3 years ago

Describe the bug Windeployqt from vcpkg (located at tools\qt5\bin) does not deploy the following required libraries:

Environment

To Reproduce Steps to reproduce the behavior:

  1. ./vcpkg install qt5-tools
  2. Build any Qt application with vcpkg
  3. Run windeployqt with the path to your application

Expected behavior Usually all libraries are copied to the application folder when using windeploqt from the standard installation.

JackBoosY commented 3 years ago

cc @Neumann-A

LilyWangLL commented 2 years ago

Ping @Neumann-A, could you please take a look this issue? Thanks in advance.

Shatur commented 2 years ago

Here is a minimal example to reproduce: TestQt.zip

  1. Compile it as usual (vcpkg.json is included, dependencies will be installed automatically).
  2. Run the executable from the build folder (all dlls are copied automatically using vcpkg), it should show empty window.
  3. Copy the executable to any other place and run <you build folder>\vcpkg_installed\x64-windows\tools\qt5\bin\windeployqt.exe --no-opengl-sw --no-angle <path to TestQt.exe>
  4. Try to run the executable, it will say that the mentioned dlls from the issue description are missing.

It's important to have windeployqt.exe working because it usually used to deploy binaries that depends on Qt. Even @Neumann-A suggested to use CMake + windeployqt for Qt deployment.

JackBoosY commented 2 years ago

Ping @Neumann-A

Neumann-A commented 2 years ago

Looks like an upstream issue for me? The real question is why are those missing?

fseegraeber commented 2 years ago

I'm having the same problem. Interestingly as @Shatur writes vcpkg manages to copy all required dlls during the build (Not sure how this works, qtdeploy.ps1?). For normal development this is great, but for deployment this is not quite enough.

The main issue with this is that there is no (good?) way to make cmake install work for a Qt app with vcpkg on Windows. You can copy the other runtime dependencies with $, but for Qt you need to run windeployqt.exe which does not work with vcpkg.

So for deployment you either

Is there a connection to #14608? E.g. the official Qt5 library has the abovementioned dlls statically linked in, while the vcpkg one does not, so not directly a windeployqt problem? In that case I would expect CMakes $<TARGET_RUNTIME_DLLS:target> to pick them up though right? Perhaps the IMPORTED_LOCATION is missing for these?

JackBoosY commented 2 years ago

It seems like an upstream bug and we can't reproduce this, so I'm going to close this issue. If more information comes up, or this issue appears again, please reopen.

Thanks.

Shatur commented 2 years ago

we can't reproduce this,

But I provided minimal reproducible example in https://github.com/microsoft/vcpkg/issues/21044#issuecomment-1024424989. Does the problem not reproduce on your machine?

JackBoosY commented 2 years ago

I'm sorry for your doubts, I didn't make it clear: Closed this issue prefers that we've been waiting for an upstream fix for a long time.

Shatur commented 2 years ago

Closed this issue prefers that we've been waiting for an upstream fix for a long time.

@JackBoosY I afraid that the issue not in the upstream. windeployqt installed as usual doesn't have this issue. It behaves like this only with vcpkg installation.

JackBoosY commented 2 years ago

I will have an investigation later.

JackBoosY commented 2 years ago

@Shatur Sorry for late, I built your sample project successful but I'm not familar with qt coding. Which parameter should I pass into windeployqt,exe?

Shatur commented 2 years ago

@JackBoosY copy the compiled application to a clean folder, run windeployqt --no-compiler-runtime --no-angle <path to the application> - the mentioned DLLs from the first message should be missing.

JackBoosY commented 2 years ago

@Shatur Such like:

C:\Users\usr\CMakeBuilds\bd6fc262-4128-8c3e-9d7a-de6165a29aa7\build\x86-Debug\vcpkg_installed\x86-windows\tools\qt5\bin>windeployqt.exe C:\Users\usr\CMakeBuilds\bd6fc262-4128-8c3e-9d7a-de6165a29aa7\build\x86-Debug\tst --no-compiler-runtime --no-angle
Unable to find dependent libraries of C:\Users\usr\CMakeBuilds\bd6fc262-4128-8c3e-9d7a-de6165a29aa7\build\x86-Debug\vcpkg_installed\x86-windows\bin\Qt5Widgetsd.dll :Cannot open 'C:/Users/usr/CMakeBuilds/bd6fc262-4128-8c3e-9d7a-de6165a29aa7/build/x86-Debug/vcpkg_installed/x86-windows/bin/Qt5Widgetsd.dll': The system cannot find the file specified.

Right?

I can run this command successfully on x86-release:

C:\Users\vusr\CMakeBuilds\bd6fc262-4128-8c3e-9d7a-de6165a29aa7\build\x86-Release\test\TestQt.exe 32 bit, release executable
Adding Qt5Svg for qsvgicon.dll
Direct dependencies: Qt5Core Qt5Widgets
All dependencies   : Qt5Core Qt5Gui Qt5Widgets
To be deployed     : Qt5Core Qt5Gui Qt5Svg Qt5Widgets
Updating icuin71.dll.
Updating icuuc71.dll.
Updating icudt71.dll.
Updating Qt5Core.dll.
Updating Qt5Gui.dll.
Updating Qt5Svg.dll.
Updating Qt5Widgets.dll.
Creating directory C:/Users/vusr/CMakeBuilds/bd6fc262-4128-8c3e-9d7a-de6165a29aa7/build/x86-Release/test/iconengines.
Updating qsvgicon.dll.
Creating directory C:/Users/vusr/CMakeBuilds/bd6fc262-4128-8c3e-9d7a-de6165a29aa7/build/x86-Release/test/imageformats.
Updating qgif.dll.
Updating qicns.dll.
Updating qico.dll.
Updating qjp2.dll.
Updating qjpeg.dll.
Updating qsvg.dll.
Updating qtga.dll.
Updating qtiff.dll.
Updating qwbmp.dll.
Updating qwebp.dll.
Creating directory C:/Users/vusr/CMakeBuilds/bd6fc262-4128-8c3e-9d7a-de6165a29aa7/build/x86-Release/test/platforms.
Updating qwindows.dll.
Creating directory C:/Users/vusr/CMakeBuilds/bd6fc262-4128-8c3e-9d7a-de6165a29aa7/build/x86-Release/test/styles.
Updating qwindowsvistastyle.dll.
Creating C:\Users\vusr\CMakeBuilds\bd6fc262-4128-8c3e-9d7a-de6165a29aa7\build\x86-Release\test\translations...
Warning: Could not find any translations in C:\Users\vusr\CMakeBuilds\bd6fc262-4128-8c3e-9d7a-de6165a29aa7\build\x86-Release\vcpkg_installed\x86-windows\share\qt5\translations (developer build?)

Since we only install the release tool, I think it's reasonable.

Shatur commented 2 years ago

Right?

This is correct.

I can run this command successfully on x86-release:

Yes, but the result is wrong. As I mentioned in the first message the following libraries won't be copied:

    pcre2-16.dll
    zstd.dll
    harfbuzz.dll
    freetype.dll
    bz2.dll
    brotlidec.dll
    brotlicommon.dll

If you try to run the application it will crash. This does not happen if you are using windeployqt from the official installation.

JackBoosY commented 2 years ago

@Shatur Are you sure that the official windeployqt can copy those libraries?

Shatur commented 2 years ago

@JackBoosY yes, I checked it before opening this issue.

JackBoosY commented 2 years ago

I think it maybe related to the installed tool path but I copied all the libraries from bin to tools/qt5/bin couldn't fix this issue. I don't check the logic code in the tool source.

Shatur commented 2 years ago

Hm, Qt could be installed in any folder. Maybe the issue in how windeploqt it built?

Neumann-A commented 2 years ago

windeployqt pulls the paths from qmake and qmake should pull the paths from qt.conf in the same folder as qmake.

Shatur commented 2 years ago

@JackBoosY so qt.conf could be adjusted to fix the issue?

JackBoosY commented 2 years ago

According to https://doc.qt.io/qt-6/qt-conf.html, I added Binraries=bin and Libraries=lib or changed LibraryExecutables to bin didn't fix this issue.

julianxhokaxhiu commented 1 year ago

As a workaround to this issue I found out that using this snippet allows you to copy over the missing DLLs which windeployqt is NOT copying over:

if (X_VCPKG_APPLOCAL_DEPS_INSTALL)
    install(TARGETS ${MY_APP_TARGET} RUNTIME DESTINATION .)
else()
    install(TARGETS ${MY_APP_TARGET}
        RUNTIME_DEPENDENCIES
            PRE_EXCLUDE_REGEXES "api-ms-" "ext-ms-"
            POST_EXCLUDE_REGEXES ".*system32/.*\\.dll"
            DIRECTORIES ${QT_BIN_DIR}
        RUNTIME DESTINATION .)
endif()

Question is: can't we link this list of DLLs statically into Qt while building it? Also why those are not required when building with the official Qt online installer framework?

Neumann-A commented 1 year ago

Also why those are not required when building with the official Qt online installer framework?

Probably because Qt vendors those dependencies statically on its own. (or disables the features requiring them.)

julianxhokaxhiu commented 1 year ago

Probably because Qt vendors those dependencies statically on its own. (or disables the features requiring them.)

Can we involve someone from Qt to help us on fixing this on our own side in vcpkg? Would be best if we could get an aligned behavior.

jobor commented 1 year ago

Traditionally, Qt bundles a bunch of 3rdparty dependencies (libpng, zlib, libjpeg, ...) to aid esp. Windows users where we don't have "system libraries". With package managers like vcpkg and conan available the situation has changed a bit, and we're looking into getting rid of the bundled libraries in the future. See https://bugreports.qt.io/browse/QTBUG-73760

When using the Qt from vcpkg, the bundled libraries are not used but the ones that are provided by vcpkg. Qt handles those as "system libraries" and windeployqt does not deploy them.

What can you do to cope with this?

  1. Recent enough vcpkg versions come with a facility that can install vcpkg dependencies. Set X_VCPKG_APPLOCAL_DEPS_INSTALL=ON for your CMake project to enable it.

  2. Alternatively, you could create an overlay triplet that builds all libraries but Qt statically.

The example under "Per-port customization" at https://learn.microsoft.com/en-us/vcpkg/users/triplets does exactly this for Qt5.

Shatur commented 1 year ago

Recent enough vcpkg versions come with a facility that can install vcpkg dependencies. Set X_VCPKG_APPLOCAL_DEPS_INSTALL=ON for your CMake project to enable it.

Not enough, X_VCPKG_APPLOCAL_DEPS_INSTALL won't install libraries that windeployqt install.

Neumann-A commented 1 year ago

Do you use https://www.qt.io/blog/cmake-deployment-api ?

Shatur commented 1 year ago

Interesting, no, I didn't know about it! I executed windeployqt manually: https://github.com/crow-translate/crow-translate/blob/652a04ff5c5440f59db03ece04e07bb6fce25807/CMakeLists.txt#L256-L258 And then copy missing libs manually: https://github.com/crow-translate/crow-translate/blob/652a04ff5c5440f59db03ece04e07bb6fce25807/CMakeLists.txt#L260-L263

Shatur commented 1 year ago

I also provided a minimal example to reproduce the issue in https://github.com/microsoft/vcpkg/issues/21044#issuecomment-1024424989.

JackBoosY commented 1 year ago

Do you use https://www.qt.io/blog/cmake-deployment-api ?

Maybe we can integrate it into port qt.

Neumann-A commented 1 year ago

Maybe we can integrate it into port qt.

No..... deployment needs to be done from the consumer side not the package manager side. Otherwise the build scripts will depend on the package manager which is not vcpkg goal as far as I know.

I mean we could add a message with the info to the qt6 deployment api.

julianxhokaxhiu commented 1 year ago

Overall the Qt deployment process is not really well documented in vcpkg so I really vote for that. Even having a simple CMakeLists.txt example that we can start from that would be nice, if possible.

I agree though that this shouldn't be part of the port itself but left to the end user, otherwise is going to be on vcpkg to maintain that script forever.

as-shura commented 1 year ago

The real problem I have encountered is with the --debug option which is not working or calculated correctly. The following error shows that windeployqt is looking for the debug dlls inside of the release vcpkg installation folder.

C:\Users\usr\CMakeBuilds\bd6fc262-4128-8c3e-9d7a-de6165a29aa7\build\x86-Debug\vcpkg_installed\x86-windows\tools\qt5\bin>windeployqt.exe C:\Users\usr\CMakeBuilds\bd6fc262-4128-8c3e-9d7a-de6165a29aa7\build\x86-Debug\tst --no-compiler-runtime --no-angle
Unable to find dependent libraries of C:\Users\usr\CMakeBuilds\bd6fc262-4128-8c3e-9d7a-de6165a29aa7\build\x86-Debug\vcpkg_installed\x86-windows\bin\Qt5Widgetsd.dll :Cannot open 'C:/Users/usr/CMakeBuilds/bd6fc262-4128-8c3e-9d7a-de6165a29aa7/build/x86-Debug/vcpkg_installed/x86-windows/bin/Qt5Widgetsd.dll': The system cannot find the file specified.

Either we should allow setting the debug and release folder of the installation folder like --debug-runtime-folder and --release-runtime-folder option directly within windeployqt

The correct path looking for the dlls should be

'C:/Users/usr/CMakeBuilds/bd6fc262-4128-8c3e-9d7a-de6165a29aa7/build/x86-Debug/vcpkg_installed/x86-windows/debug/bin/Qt5Widgetsd.dll'
github-actions[bot] commented 1 year ago

This is an automated message. Per our repo policy, stale issues get closed if there has been no activity in the past 28 days. The issue will be automatically closed in 14 days. If you wish to keep this issue open, please add a new comment.

Shatur commented 1 year ago

Still relevant.

github-actions[bot] commented 1 year ago

This is an automated message. Per our repo policy, stale issues get closed if there has been no activity in the past 28 days. The issue will be automatically closed in 14 days. If you wish to keep this issue open, please add a new comment.

Shatur commented 1 year ago

Still relevant.

github-actions[bot] commented 1 year ago

This is an automated message. Per our repo policy, stale issues get closed if there has been no activity in the past 28 days. The issue will be automatically closed in 14 days. If you wish to keep this issue open, please add a new comment.

github-actions[bot] commented 1 year ago

This is an automated message. Per our repo policy, stale issues get closed if there has been no activity in the past 28 days. The issue will be automatically closed in 14 days. If you wish to keep this issue open, please add a new comment.

teamyayc commented 2 months ago

We are also encountering this issue. Which we addressed by manually copying over the missing dlls from vcpkg bin dir. But, in fact, one more: When deploying our webengine5 based application (https://github.com/yayc-stream/yayc), the resulting deployment crashes after startup with a QtWebEngineCore error (https://paste.debian.net/hidden/588db314). I checked and all the listed webengine dependencies from the doc page "Deploying Qt WebEngine Applications" are present. Yet, using windeployqt from the qt 5.15.2 release coming from the Qt online installer produces a working release, while even after fixing all the dependencies, the deployment from vcpkg's windeployqt fails.

Note that running the application from within QtCreator with vcpkg Qt version works as expected.