search

jtv / libpqxx

The official C++ client API for PostgreSQL.
http://pqxx.org/libpqxx/
BSD 3-Clause "New" or "Revised" License
1.03k stars 240 forks source link

Executing runner.exe or unit_runner.exe on W10 fails due to DLL mismatch error #341

Closed yscontrol closed 4 years ago

yscontrol commented 4 years ago

--Question: Attempt to run .exe built on ununtu 16.04/xenial with mingw64 failed with error: The procedure entry point ''' ZNK4pqxx11streamfrom13extract_valuelDnEERKNSt7_cxx1112basicstringlcSt11char_traitslcESalcEEERTRyRS7 could not be located in the dynamic library c:\Users\me\Downloads\POSTGRESQL\install\mingw64\lib\unit_runner.exe ''' All .dll files including libpq and libpqxx are located in the same folder as the executables. I had to manually copy several gcc .dlls from mingw64 installation C:\mingw64\mingw64\x86_64-w64-mingw32\lib mingw libwinpthread-1.dll, libstdc++-6.dll libgcc_s_seh-1.dll to this folder to

set PATH=C:\mingw64\mingw64\bin;%PATH%

On windows target - W10 version 1809, mingw64 version was 4.8.1 so gcc .dll are older than ones the exe was built on https://sourceforge.net/projects/mingw-w64/files/Toolchains%20targetting%20Win32/Personal%20Builds/mingw-builds/

on build host The build is done with later version using x86_64-w64-mingw32-gcc (GCC) 5.3.1 20160211 for windows 64 bit

I am going to try mingw 5.3.0

Home / Toolchains targetting Win32 / Personal Builds / mingw-builds / 5.3.0 https://sourceforge.net/projects/mingw-w64/files/Toolchains%20targetting%20Win32/Personal%20Builds/mingw-builds/5.3.0/

However I thought to ask this question bc I see the error seems to implicate libpqxx extract_value , char_traits

tt4g commented 4 years ago

Applications built with MinGW will not run on Windows as is. This is because MinGW links its own library that provides gcc compatible instructions in order to be able to build a library for gcc on Windows.

In order to run an application built by MinGW, you must either add the MinGW library path to the Windows environment variable or copy the MinGW libraries to the same folder as the executable file.

See also: c++ - MinGW .exe requires a few gcc dll's regardless of the code? - Stack Overflow

yscontrol commented 4 years ago

Yes, of course, I added to the system path, path to mingw64 bin folder C:\mingw64\mingw64\bin, but this would not be affecting .dll path serach , right? After this I got missing DLL errors and copied all of these DLL files libstdc++-6.dll libgcc_s_seh-1.dll libwinpthread-1.dll to the folder where I run .exe ( see description below). However as I understand this error it is occurs when 2nd copy but different version of dll found in the path. But the error message indicates (exported) extract_value libpqxx method.

On Tue, Jun 23, 2020 at 9:51 PM tt4g notifications@github.com wrote:

Applications built with MinGW will not run on Windows as is. This is because MinGW links its own library that provides gcc compatible instructions in order to be able to build a library for gcc on Windows.

In order to run an application built by MinGW, you must either add the MinGW library path to the Windows environment variable or copy the MinGW libraries to the same folder as the executable file.

See also: c++ - MinGW .exe requires a few gcc dll's regardless of the code? - Stack Overflow https://stackoverflow.com/questions/18138635/mingw-exe-requires-a-few-gcc-dlls-regardless-of-the-code

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/jtv/libpqxx/issues/341#issuecomment-648550145, or unsubscribe https://github.com/notifications/unsubscribe-auth/AE4UHIRVURRRRRWC3HSS6CLRYFS2BANCNFSM4OGCXYGQ .

tt4g commented 4 years ago

Did you check to see if the OS can find the DLL in your path? Environment variables set at the command prompt are only temporarily recorded. If you update the path from the Windows System Settings, the new environment variables will not be loaded unless you restart the command prompt.

Did you check with the where libwinpthread-1.dll command to see if the path is set?

tt4g commented 4 years ago

But the error message indicates (exported) extract_value libpqxx method.

This is because we use std::char_traits in libpqxx.

The std::char_traits is specified in the C++ standard specification. Therefore, if the version of the C++ standard is changed, the internal implementation may be changed.

To accommodate changes in the STL internal implementation, the compiler changes the linked DLLs according to the C++ version. In order to avoid problems when DLLs with the same symbol are placed in a single folder, the DLLs for each C++ version may be stored in different folders.

It looks like only the standard DLLs for MinGW are present in C:\mingw64\mingw64\bin that you added to the path, and the DLLs for C++11 were placed in C:\mingw64\mingw64\x86_64-w64-mingw32\lib.

jtv commented 4 years ago

When cross-compiling, I I would work with static libraries as much as possible. It avoids these DLL problems.

yscontrol commented 4 years ago

I gave it quick try by adding static target in libpqxxvm/test/CMakeLists.txt but got same error when running runner_static. add_executable(runner_static ${TEST_SOURCES}) target_link_libraries(runner_static PUBLIC pqxx_static) add_test(NAME runner_static WORKING_DIRECTORY ${CMAKE_RUNTIME_OUTPUT_DIRECTORY} COMMAND runner_static)

jtv commented 4 years ago

I can't tell for sure if it's equivalent, but I was talking about building, and using, only static libraries.

Ideally, even the compiler runtime. I seem to remember that that was a little DLL but there was an obscure option to link a static version instead. Going on years-old memories here though, and unrelated to libpqxx.

yscontrol commented 4 years ago

The libpqxx project does not provide option to build runner statically so with the added lines both version runner.exe and runner_static.exe are being built independently. However more changes might be needed for the static build. My ultimate goal is still to build proper dll of my library which has dependency on libpq/libpqxx because I am providing library for consumers who would develop their app. I still want to build static library as a interim step.

tt4g commented 4 years ago

@yscontrol You have to link the system libraries static as well. Set CMAKE_EXE_LINKER_FLAGS or set the target_link_options() option to statically link the system libraries.

Example:

target_link_options(target_link_options PRIVATE -static -static-libgcc -static-libstdc++)

See also: c++ - CMake: Linking statically against libgcc and libstdc++ into a shared library - Stack Overflow

jtv commented 4 years ago

There's no option to build a static test runner, but if you build only a static library, then you will get a static test runner.

Is there anything stopping you from building your own DLL which links to a static libpqxx? You'll need position-independent code, but that's normally just a matter of passing the right compiler option.

yscontrol commented 4 years ago
  1. Hi @tt4g as you indicated the error is caused by "use std::char_traits in libpqxx" ,.."It looks like only the standard DLLs for MinGW are present in C:\mingw64\mingw64\bin that you added to the path, and the DLLs for C++11 were placed in C:\mingw64\mingw64\x86_64-w64-mingw32\lib." so to resolve this I could find which .dll has this symbol and move conflicting .dll in separate folder so the proper one picked first. Which one should be used for std::char_traits ? Then we might see that some other objects missing due to new order?
yscontrol commented 4 years ago

Hi @jtv 1. The libpqxx.a static library is being built ok(not sure if compiler options are correct). So by "only a static library" do you mean that somehow building libpqxx.dll somehow affects libpqxx.a ? Or that independent targets libpqxx_static & libpqxx_shared in cmake one after another somehow different than commenting out one and build another and doing it twice - once with SKIP_PQXX_SHARED and once with SKIP_PQXX_STATIC ? What would be the mechanism of such interaction. In cmake targets binding is done only via dependencies& target properties. Maybe you meant that building statically would require different toolchain to set different compiler options? Then this could be done using CMAKE own var COMPILE_DEFINITIONS setting it differently for each target and passing this to toolchain. So what are the right " the right compiler option" for libpqxx_static. Do you think setting -fPIC / set( CMAKE_CXX_FLAGS "-fPIC ${CMAKE_CXX_FLAGS}" ) / set_property(TARGET mylib_shared PROPERTY POSITION_INDEPENDENT_CODE TRUE) / cmake target_compile_definitions() is enough or libpqxx_static has to have compiler linker extra flags like below Why was not they set in the 6.4.6 libpqxx_shared build ? '''

set(CMAKE_CXX_STANDARD_LIBRARIES "-static-libgcc -static-libstdc++ -lwsock32 -lws2_32 ${CMAKE_CXX_STANDARD_LIBRARIES}")

set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -Wl,-Bstatic,--whole-archive -lwinpthread -Wl,--no-whole-archive")

set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -static")

''' Since I am using mingw-std-threads header onli lib, I thought I don't need winpthread, agree? Also mingw64 W10 installation set provides 2 options for file downloads threads : win32 and pthreads and I picked pthreads (hopefully it match what I am building in ubuntu). 2, yes, I had negative experience mixing static and shared libs, but assuming I could keep using same version of libpqxx(libpq, mingw-std-threads(headers only)) for long time, willing to try this approach as do have libpqxx.a built but .lib is not built. I am trying to add support for building static runner.exe/ unit_runner.exe just for testing purpose. What I need to add right compiler option as you said (not sure yet which)

  1. @tt4g "Did you check to see if the OS can find the DLL in your path? - I set system path to begin with C:\mingw64\mingw64\bin;C:\mingw64\mingw64\x86_64-w64-mingw32\bin;C:\mingw64\mingw64\x86_64-w64-mingw32\lib;, then opened new cmd window and checked echo %PATH% activated. However this is just a system path so how could I tell path used to search through .dll-s? "Did you check with the where libwinpthread-1.dll command to see if the path is set?" - do I need this? I thought I dont since I am building with mingw-std-threads header only lib. What affect would it have on finding " std::char_traits"?
jtv commented 4 years ago

Sidenote: Windows looks for DLLs in your command path. So the PATH variable defines where the system tries to find your DLLs.

jtv commented 4 years ago

@yscontrol you're piling on a lot of questions at once, so I can't fully parse and address them all. I find them a bit hard to read and I'm quite busy at the moment, my apologies.

My point about building only the static library is that you don't need to build the DLL at all. Building only the static library does not affect how the static library gets built, but it does affect how the test runner is built. If you build only the static library, then the test runner will have no choice: it will have to use the static library. And that solves the DLL problem.

If you want to know how to build a static library using MinGW that you can use inside a DLL, that's a generic problem, so I suggest you search the internet for that. With gcc on other platforms I do think -fPIC is the main thing, but who knows — it might be a little different on Windows.

tt4g commented 4 years ago

Which one should be used for std::char_traits ? Then we might see that some other objects missing due to new order?

I don't know. There are rules for the system libraries that the compiler links to, but not many people know all of them. I think the only way to determine these problems is based on the error messages.

yscontrol commented 4 years ago

At this point both shared (defined PQXX_SHARED) or static (PQXX_SHARED not defined) versions of generate message of missing std::char_traits. The differences in the code is like the following: compiler-internal.hxx

'''

ifdef _WIN32

ifdef PQXX_SHARED

undef PQXX_LIBEXPORT

define PQXX_LIBEXPORT __declspec(dllexport)

define PQXX_PRIVATE __declspec()

endif // PQXX_SHARED

''' compiler-public.hxx '''

ifdef _WIN32

/* For now, export DLL symbols if _DLL is defined. This is done automatically

''' So I wonder if libpqxx code was ever built PQXX_SHARED not defined and then where else code follows PQXX_SHARED even when it is not defined to not allow static linking.

When building shared runner.exe (with PQXX_SHARED ) what should be done to export std::char_traits or make it found.

and looking at libpqxx.dll there are some undefined like ''' U _ZNK4pqxx5field2toINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEEENSt9enable_ifIXoontsrSt10is_pointerIT_E5valuesrSt7is_sameISAPKcE5valueEbE4typeERSA ''' So maybe this problem caused by building flags to ignore not-defined?

tt4g commented 4 years ago

I don't think the DLL providing the std::char_traits is loaded. You should link the static system libraries to the test_runner.

yscontrol commented 4 years ago

I "grepped" through all libs finding char_traits and copied them to the folder with runner.exe libstdc++-6.dll libstdc++.dll.a libstdc++.a libstdc++.dll.a libstdprov.a (This is aside verifying mingw64 path is ath the head of System and User %PATH%) Still having this error. Note when windows could not find dependent library I had different error implicitly indicating not found lib. So now I am trying to add libstdc+ statically in cmake with

add_executable(runner_static ${TEST_SOURCES}) target_link_libraries(runner_static PUBLIC pqxx_static) add_compile_options(-static -static-libstdc++ )

However I just want to ask you about another warning I am getting

''' [ 74%] Building CXX object test/CMakeFiles/runner_static.dir/test87.cxx.obj /home/cnh/workspace/framework/libpqxxvm/test/test87.cxx:19:0: warning: "NOMINMAX" redefined

define NOMINMAX

^ In file included from /usr/lib/gcc/x86_64-w64-mingw32/5.3-posix/include/c++/x86_64-w64-mingw32/bits/c++config.h:482:0, from /usr/lib/gcc/x86_64-w64-mingw32/5.3-posix/include/c++/cctype:41, from /home/cnh/workspace/framework/libpqxxvm/test/test87.cxx:1: /usr/lib/gcc/x86_64-w64-mingw32/5.3-posix/include/c++/x86_64-w64-mingw32/bits/os_defines.h:45:0: note: this is the location of the previous definition

define NOMINMAX 1

^

'''

tt4g commented 4 years ago

NOMINMAX is to avoid the definition of the min and max macros defined in Windows.h. The value of this macro is not usually referenced and can be ignored.