jtv / libpqxx

The official C++ client API for PostgreSQL.
http://pqxx.org/libpqxx/
BSD 3-Clause "New" or "Revised" License
1.03k stars 240 forks source link

problem of installing Windows #306

Closed chenqingguo closed 4 years ago

chenqingguo commented 4 years ago

I install x64 libpq then I build libpqxx using cmake and visual studio 2017. I got this error MSB4126 Debug |x64 is invalid. I need to build x64 libpqxx. How to do? Thanks,chenqingguo.

jtv commented 4 years ago

Folks, there's way more to read here than I can keep up with right now. If PostgreSQL_ROOT seems to be a problem for some reason, I can just document PostgreSQL_LIBRARY, PostgreSQL_INCLUDE_DIR, and PostgreSQL_TYPE_INCLUDE_DIR as the way to override the results of the search.

But that doesn't solve the problem of where to install. On a Unix-like system I trust my build system to figure out the standard locations — usually /usr/local/. But what's the standard on Windows? And if you want a different location, how do you indicate it?

GordonLElliott commented 4 years ago

I think this is more simple. I don't understand how you set policies etc. But using PostgreSQL_ROOT is a capability of the newest, and I have CMake 3.17.1. The help said:

C:\Users\Bravo.BRAVO7>cmake --help-policy CMP0074
CMP0074
-------

``find_package()`` uses ``<PackageName>_ROOT`` variables.

In CMake 3.12 and above the ``find_package(<PackageName>)`` command now
searches prefixes specified by the ``<PackageName>_ROOT`` CMake
variable and the ``<PackageName>_ROOT`` environment variable.
Package roots are maintained as a stack so nested calls to all ``find_*``
commands inside find modules and config packages also search the roots as
prefixes.  This policy provides compatibility with projects that have not been
updated to avoid using ``<PackageName>_ROOT`` variables for other purposes.

The ``OLD`` behavior for this policy is to ignore ``<PackageName>_ROOT``
variables.  The ``NEW`` behavior for this policy is to use
``<PackageName>_ROOT`` variables.

This policy was introduced in CMake version 3.12.  CMake version
3.17.1 warns when the policy is not set and uses ``OLD`` behavior.
Use the ``cmake_policy()`` command to set it to ``OLD`` or ``NEW``
explicitly.

.. note::
  The ``OLD`` behavior of a policy is
  ``deprecated by definition``
  and may be removed in a future version of CMake.

I just don't understand well enough where these are set--somewhere specific to your settings for PQXX, since my CMake is most up to date.

jtv commented 4 years ago

Well, if it's being ignored because it's too new then at least I know it's worth keeping an eye on!

GordonLElliott commented 4 years ago

I changed this and it worked: cmake_minimum_required(VERSION 3.12) in the top level .txt file. I don't understand it yet--but there is a way to specifically enable that enve though your minimum is lower. It is that minimum of 3.7 that is turning the policy off now.

Still don't understand, but says: Use the cmake_policy() command to set it to OLD or NEW explicitly. So if I figure out, we are to set CMP0074 policy to "NEW" if you don't update the minimum required to at least 3.12.

KayEss commented 4 years ago

In case it helps, here's how another project manages CMake versions and policy. You can do the checks and turn them to the settings you need even whilst supporting older versions. Clearly you need to be careful that the rest of the files conform to only what the lower version can do.

https://github.com/noloader/cryptopp-cmake/blob/master/CMakeLists.txt#L32

GordonLElliott commented 4 years ago

Rather than update your minimum CMake, this works as change in top level CMakeLists.txt:

cmake_minimum_required(VERSION 3.7)
cmake_policy(SET CMP0074 NEW)

I didn't let it go all the way through yet, but used -DPostgreSQL_ROOT=.... in the command line and it then found the PostgreSQL version---so much simpler than 3 variables!

If I understand, all this does is enable the added command format of xxx_ROOT variable. If user has an older system in which this does not work--won't matter at all unless using the DPostgreSQL_ROOT feature which then requires user to update to newer CMake.

If in the future you set cmake_minimum_required(VERSION 3.12) or higher, then you no longer need the explicit policy exemption. (Sounds like someone from insurance industry worked on this...)

Regarding KayEss's point, this "policy" change should not affect any CODING in the setup, it just enables a new feature that someone might use by CMake command line during installation.

YOU GUYS should make this change in your distribution.

GordonLElliott commented 4 years ago

OK, assuming you guys put that policy line into the distribution, here is an extremely simple batch way to install on windows systems, even with multiple database or compiler systems. You can also just type in or copy to a command window (but get every character correct).

REM simplified batch or command line installation for CMake of pqxx:
cmake -S libpqxx-master -B build_intermediate -G "Visual Studio 16 2019" -A x64 -DPostgreSQL_ROOT="C:\Program Files\PostgreSQL\12" -DCMAKE_INSTALL_PREFIX="C:\pqxx_7_install"

REM this line requires administrator priveleges for Program Files installation (established above)
cmake --build build_intermediate --target INSTALL --config Release

pause

Make a directory for the installation. Place the libpqxx-master directory (the one with files just inside, not another layer) into that directory. Then place a MYMAKE.bat file or whatever with text shown above. Edit that as needed to adjust Visual Studio version, PostgreSQL version, or final install location. Delete entirely the following entries if not needed:

Change "Release" to "Debug" if you want a debug version of the library. (The install location will not have the associated .pdb file but as long as you don't recompile the source without re-installing the library it will work.)

To activate the installation, double click on the MYMAKE.bat file (or as you named it).

Now, if only single PostgreSQL is installed, only single compiler is available or want to use the latest, and you want to install into Program Files directory (and will give administrator privileges when running install step), and only want the Debug version of the library, then:

(DEBUGGED VERSION: )

cmake -S libpqxx-master -B build_intermediate -A x64 -DCMAKE_INSTALL_PREFIX="C:\Program Files/libpqxx"
REM this line requires administrator priveleges for Program Files installation:
cmake --build build_intermediate --target INSTALL 

pause

This is all needed for simplest installation, arrange files as bove.

To run as administrator, in the Windows task bar there is a "Type here to search" box. Type command. The "Command Prompt App" will show. To the right somewhere is "run as administrator".

Then CD to location of the batch files. (This can be done by copying the location and past into command prompt after typing CD .) Then type name of the batch file.

(I tried many ways to run as administrator and no other way worked. Ways that didn't work included right clicking and running as administrator--which jumped to wrong location to run the file, Logging in as Administrator and simply running file either clicking or in command prompt that was not started with admin privileges simply didn't have privileges.)

Another method is to remove (or ignore the failed) --build line, and open libpqxx.sln in Visual Studio--that having been opened as administrator. Then specifically right click and build the INSTALL project, which has been pre-loaded with location.

A key feature of the above methods is that the user does not have to create the intermediate directory structures and there are no change directory steps.

To answer jtv's question:

But what's the standard on Windows? And if you want a different location, how do you indicate it?

The standard on Windows is to do what PostgreSQL did, put in Program Files under application's name. The default method has the right idea--just puts it in "Program Files (x86)/libpqxx " which is specifically for Win32 programs and is very much the wrong place. The examples here show also how to select another location.

GordonLElliott commented 4 years ago

But that doesn't solve the problem of where to install. On a Unix-like system I trust my build system to figure out the standard locations — usually /usr/local/. But what's the standard on Windows? And if you want a different location, how do you indicate it?

See the just prior post. But please add line to the top level CMake file:

cmake_minimum_required(VERSION 3.7)
cmake_policy(SET CMP0074 NEW)

The (second) line allows the -DPostgreSQL_ROOT=... to work, the problem was on your end. (The low minimum version causes the flag to be set inoperative.)

(I'm getting the feeling you guys don't actually run the Windows 10/Visual Studio we are talking about. If you make the minor changes needed, document a method based upon what we have learned, I will be glad to "act dumb" and follow the directions (and new download) to test the installation, both for VS 2017 and VS 2019. Also note that to get the C++ 17, one has to be using VS 2017 minimum, so these are the only choices today. Just remember I have the tougher case of multiple installs of both database and compiler and will need the more specific flags to get to work completely correctly.)

This is clearly a "programmer's" installation system, not a user system. Normal "install" projects come up as windows and have a place to override the location--either presenting the default or giving a check box to allow override then enter. Windows installers vary widely because they are written by different groups.

I cannot see how to pre-populate the CMake GUI in a meaningful way. Is there a way to set that up with relative values from a stored file? (such located relative to the stored file?) One does not know where the user will store the installation files on a windows install, that is user's choice when downloading an application. An standardization is then carried out normally in some sort of install application that installs the final application.

Another related issue is that on Windows the 32 bit applications are placed in \Program Files (x86), while 64 bit applications go to \Program Files. (In older 32 bit operating system they all went to \Program Files, so this was a change of installation method when the operating system was upgraded.) Since the "platform" (a.k.a. "architecture) is known (or picked by CMake), the system should be able to figure out where to put that. But I don't understand where that is controlled. At present it is picking incorrectly.

By the way, making a 32 bit version of pqxx on Windows is very difficult. To do so one must have a 32 bit version of PostgreSQL, which means limiting to version 10.xx, and that must be installed (whether or not a 64 bit version is also installed). I set up the proper flags to do so, including pointing at the 32 bit PostgreSQL, and it did build a library and a set of tests that could be run. However I could not use the IDE to run the tests as normal as they were loaded incorrectly. I was able to run (or at least start) the tests separately, didn't get around to setting up conmnection to database for 32 bit version. The main point is that anyone using PostgreSQL 12 will be using exclusively 64 bit version--so the libraries should be going to \Program Files directory (without '(x86)'). And that would be "the standard on Windows."

I think I mentioned the difficulty in getting permissions to install in the \Program Files directories (either one) because that step must run with administrator privileges. Yet using the Microsoft tools to trigger that step in the INSTALL project. (That project then simultaneously does make-like activity to guarantee up to date-ness before triggering the final install in this setup.) But Microsoft worked very hard to get Visual Studio out of the business of having administrative rights, because people could embed viruses in the compiling of libraries wherein the compiler itself would asist in the bad works. Likewise in batch files, tend to be special rules for triggering batch files that install. So the "normal" way projects are installed with permissions is that GUI based installer first acquires the administrative permissions and then controlls the last steps with permission--that installer itself having been vetted by anti-virus software for the most part. Here we are allowing most recently compiled (and un-vetted) code immediate access to administrative areas, as an inherent part of the process. Once again very much a programmer's installation not at all like a user's install.

tt4g commented 4 years ago

@jtv

https://github.com/jtv/libpqxx/issues/306#issuecomment-620689899:

Well, if it's being ignored because it's too new then at least I know it's worth keeping an eye on!

I had forgotten that <PackageName> _ROOT is a feature added in CMake 3.12.0: https://cmake.org/cmake/help/v3.17/policy/CMP0074.html

When CMake adds incompatible features, it may add Policy along with it. It is affected by the CMake policy cmake_minumum_required(). Since pqxx specifies cmake_minumum_required(VERSION 3.7), OLD seems to be the default Policy: https://cmake.org/cmake/help/latest/command/cmake_minimum_required.html#policy-settings If specify cmake_minumum_required(VERSION 3.12), the Policy CMP0074 will automatically become NEW but on some modern platforms (such as CentOS 7, Ubuntu 19.10 ...) CMake version is less than 3.12.

https://github.com/jtv/libpqxx/issues/306#issuecomment-620673796:

Folks, there's way more to read here than I can keep up with right now. If PostgreSQL_ROOT seems to be a problem for some reason, I can just document PostgreSQL_LIBRARY, PostgreSQL_INCLUDE_DIR, and PostgreSQL_TYPE_INCLUDE_DIR as the way to override the results of the search.

Yes, PostgreSQL_ROOT can be considered as a PostgreSQL_LIBRARY, PostgreSQL_INCLUDE_DIR, PostgreSQL_TYPE_INCLUDE_DIR shorthand. Since these three variables have been around for a long time, I believe they will work with many versions of CMake.

tt4g commented 4 years ago

For the pqxx installation location, refer to the CMAKE_INSTALL_PREFIX document for the default installation location: https://cmake.org/cmake/help/latest/variable/CMAKE_INSTALL_PREFIX.html#variable:CMAKE_INSTALL_PREFIX The default for Windows is C:/Program Files/${PROJECT_NAME}. This follows the default installation location convention used by many Windows installers.

The disadvantage of default install location C:/Program Files/${PROJECT_NAME} is that it often limits who can write to C:/Program Files. The installation will fail if the CMake user does not have write permissions.

The project can change the default installation location, as described in CMAKE_INSTALL_PREFIX_INITIALIZED_TO_DEFAULT: https://cmake.org/cmake/help/latest/variable/CMAKE_INSTALL_PREFIX_INITIALIZED_TO_DEFAULT.html

# NOTE Must be done before `project()`.
if(CMAKE_INSTALL_PREFIX_INITIALIZED_TO_DEFAULT)
  set(CMAKE_INSTALL_PREFIX "/my/default" CACHE PATH "..." FORCE)
endif()

However, if the project changes the install location, it will be confusing as users behave unexpectedly when they expect the default install location. I believe it is best for the user to specify the installation location by setting CMAKE_INSTALL_PREFIX on the command line (this is similar to autotools ./configure --prefix=/usr/local.).

GordonLElliott commented 4 years ago

The default for Windows is C:/Program Files/${PROJECT_NAME}. This follows the default installation location convention used by many Windows installers.

I agree completely that this is the "correct" thing to do. (And just work until the permissions difficulties are overcome and procedures are able to be explained simply.)

The problem is that somewhere in this system that default is accidentally set to C:\Program Files (x86)... Possibly because some subsystem still thinks the project is a 32 bit rather than 64 bit application. (In other words some part knows that it is to build an x64 program rather than a Win32 program, but the part that selects the install location is not tracking that decision.) I assure you that the x64 compiles are being installed in C:\Program Files (x86)... when no CMAKE_INSTALL_PREFIX is specified, in our case.

tt4g commented 4 years ago

The problem is that somewhere in this system that default is accidentally set to C:\Program Files (x86)... Possibly because some subsystem still thinks the project is a 32 bit rather than 64 bit application. (In other words some part knows that it is to build an x64 program rather than a Win32 program, but the part that selects the install location is not tracking that decision.) I assure you that the x64 compiles are being installed in C:\Program Files (x86)... when no CMAKE_INSTALL_PREFIX is specified, in our case.

For now, this is an intentional CMake specification. See this CMake issue: https://gitlab.kitware.com/cmake/cmake/issues/18312

As mentioned in the issue above, the default installation location of Windows may change in the future. I don't want to do that because changing the install location in the CMakeLists.txt in pqxx may break the CMake default install location in the future.

And CMake set default CMAKE_INSTALL_PREFIX from bellow code:

https://github.com/Kitware/CMake/blob/615129f3ebd308abeaaee7f5f0689e7fc4616c28/Modules/CMakeGenericSystem.cmake#L168-L177

https://github.com/Kitware/CMake/blob/9afd5f0d3296194694fb05232a4da17b059b77d8/Modules/CMakeGenericSystem.cmake#L75

You can see that it is inferring CMAKE_INSTALL_PREFIX from the variables that specify CMake generator such as CMAKE_GENERATOR and CMAKE_GENERATOR_PLATFORM, and the environment variables. You should not change these variables in pqxx CMakeLists.txt, as changing them will impact your build tree.

GordonLElliott commented 4 years ago

You all have a lot to consider. I have been having all the problems mentioned by others in other projects, so this is somewhat structural to CMake not just something that happened to me for the first time.

Observations:

CMake is a programmer's method of creating an installation dataset. I have never understood Linux installation except to know that many people install Linux programs by directly compiling (and such compliling causes the program to be "installed" after that). So we even have a basic viewpoint difference between the meaning between Linux users and Windows users. (Apple is sort of a third case, tends to have a final install like Windows as well.)

So on Linux, it should do as you all expect (and I am not sure I even know what that is).

On Windows we don't expect our compiling tools to try to install. We have a completely separate installation process, motivated by several considerations:

Here the "product" being installed is actually a programmer's tool. While that programmer's tool is indeed very much a parallel to a user's tool (namely the PostgreSQL installation itself), general PostgreSQL users don't necessarily install pqxx programming tools, while pqxx programmers necessarily install user's PostgreSQL. The view that pqxx install should look like the PostgreSQL install is perhaps mistaken.

I am going to suggest NOW (and probably completely contradicting any of my prior suggestions) that for Windows installation the final (and possibly priviledge related) installation step should be completely separated from CMake.

This would be consistent with the very distinct views, where Linux users might consider compiling as part of an installation process, Windows users think of compiling as preparation of the package for install and not an installation process.

Now before making my NEW recommendation, consider from way above another point:

The CMake -S, -B flag method keeps the steps of creating the output "packaging" (as I shall call it) separate from the source code and also separate from the resulting "packaging". It furthermoer automatically creates the output directory of that "packaging", which otherwise is an additional user step. (The older CMake would only operate from views of change directory to one of those locations, this new methodology allows getting out of all that changing directories.)

Recommendation:

View the entire CMake process as CREATING an installable product, not a process OF installation, when done on Windows. This is done by documentation for the most part, as a separate documentation of how to compile and install for Windows from how to do so for Linux etc.

The above CMake -S, -B flag method allows inputs and outputs of the input source, and output intermediate compilation datasets to be done as two directories separate from the command process and directory in which user operates. Namely the source may be a subdirectory libpqxx-master), the intermediate a second subdirectory (possibly "intermediate-build"). Then the prepared installation can be a third subdirectory (possibly "installation_product" not a good name but you get idea).

By (on windows) making that "INSTALL" project directory local to the process, we avoid all the security issues of making a CMake, or a Visual Studio instance doing secure steps requiring privileges.

People have complained that they don't like "batch" scripts, but in Window's world batch scripts have been used for compiling and even installation steps forever.

So with the point in mind that the entire CMake process would just make the "installation product" and not do the install, we put that installation product in a third subdirectory. (AND I will highlight that in the Visual Studio solution there is also a PACKAGE task that builds exactly that installation product in a gzip form.)

Getting all the options right is quite a bit of typing, and batch files are an excellent method (on Windows) of preparing all that detail then executing after preparation is done.

So recommendation is something like I have been saying, use batch files -- but now all the versions are just simpler or more complex versions of the input flag settings and in all cases prepare the installation product in a third subdirectory from the batch and compile process (the three subdirectories being libpqxx-master, intermediate-build, and installation-product).

Then we do a completely separate step of placing that into the "\Program Files" directory (and I intentionally used the Windows "\" there). WE have a separate debate how to get it there but stop discussing how CMake might interact with that. CMake is broken in this regard anyway (SPECIFIC to Windows) and is conceptually the wrong process in the manner Windows' users expect. Users that want it a different place just move that in a different way from this last step involving privileges. AND the gzip package of the PACKAGE project is simply a third option for that same output to be installed later.

Second Recommendation:

I highly recommend adding the

cmake_policy(SET CMP0074 NEW)

line to the file. It does not break anything! It only enables something that is not present in older CMake's. DO NOT make use of the new capability inside of your CMake files -- this is strictly so the user can use the PostgreSQL_ROOT method. And why not have users upgrade Windows to latest CMake -- they pretty much need to do that to even use Visual Studio 2017 or 2019, which is a requirement to get the C++ 17 anyway. This will simplify the task in the prior recommendation of setting up for the build when the user has complex setup with multiple database installs etc.

tt4g commented 4 years ago

My opinion remains the same. I believe there is no need to significantly change the pqxx's CMakeLists.txt. The famous test library googletest also has CMakeLists.txt, but CMAKE_INSTALL_PREFIX has to be set manually by the user. It may seem awkward to use, but this is CMake's style.

I'm not against adding a batch file. But I don't want to maintain it because I have been struggling. The batch file on Windows recognizes only CRLF as a line feed character, but the line feed code is changed to LF in the git repository. It was awkward because it depends on .gitattributes and git config whether the newline character is restored when git checkout.

Also, the character encoding that Windows expects in a batch file must be the same as the console. The character encoding of the console is transitively determined from the system locale. Due to this convention, I often have experience with batch files not working.

GordonLElliott commented 4 years ago

.... but CMAKE_INSTALL_PREFIX has to be set manually by the user.

With or without batch files, I agree that the CMAKE_INSTALL_PREFIX is the way to operate. (Users will still need to control where the "install-product" goes.) The intention above is that you don't ever try to fix CMake's attempt to install in a \Program Files directory requiring privileges. Rather the user (however done)--ON Windows--would consider the install output as the data they need to put where they want it. Remember it comes in two forms: INSTALL generates a subdirectory, and PACKAGE generates the same output in a gzip form. And if one is to standardize into the correct Program Files location, use a separate method including manual copy (in which the operating system asks permission at the time rather than failing and requiring user to re-think their process).

Also I understand the logic of enabling policy CMP0074 only for PostgreSQL_ROOT. But if there has been no use of the form _ROOT in the present system, the logic required seems a bit complicated. I tried enable at the top and had no conflicts on Windows, but of course that did not test a large body of CMake scripting. I searched the string _ROOT and it was not located anywhere. If it enabled a user to specify a _ROOT for another one of the package searches, is that a bad thing?

tt4g commented 4 years ago

use a separate method including manual copy (in which the operating system asks permission at the time rather than failing and requiring user to re-think their process).

CMake cannot retry after an installation failure. The Makefile is free to execute commands so it may be able to handle installation failures. But CMake only provides a way to perform the installation. CMake's install function does not provide a way to interactively change the install location or install to a different path after a failure.

If it enabled a user to specify a _ROOT for another one of the package searches, is that a bad thing?

It is a rule that is close to etiquette that scopes changes in CMake policies. CMake has add_subdirectory() command which allows users to include the pqxx project in their project. Global policy changes may affect other projects. To avoid this, policy changes should have scope.

You can find many similar codes by searching for "if(POLICY"" in the GitHub code search.

GordonLElliott commented 4 years ago

use a separate method including manual copy (in which the operating system asks permission at the time rather than failing and requiring user to re-think their process).

CMake cannot retry after an installation failure. .....

I wasn't very clear, I meant from the user's view any attempt using CMake with the privileges restricted location in Windows (like "\Program Files") becomes a miserable experience. (Even making batch files do this on Windows is miserable--in the latest versions of Windows. Used to be one just logged in a admin and did anything. But restrictions are placed because Windows has so much problem with viruses and external attack.)

So I ment a user procedure external from the CMake process, and have CMake "INSTALL" produce a locally placed result. (Which of course would be done with the CMAKE_INSTALL_PREFIX on the user's part,) I'm suggesting a documentaton of recommended methodology for windows. Jtv asked what the standard is on Windows, and of course that cannot be directly achieved using CMake. (And only indirectly with jumping through many hoops.)

As to CMake rules, I see a lot of comments I have not read, but after looking at the logic and finally understanding it better I think that perfectly solves our issues here. Just use the various CMake command line flags to achieve the user intent and leave CMake with these last changes which are in progress.

So we still need to write up a methodology for Windows users. I am going to do an internal method just for the development group I am working with, that will have batch files -- but everyone is using Windows and everyone is using the exact same version of Postgres and VS 2019.

For you the most standard location for the final install files to wind up is in C:\Program Files\libpqxx (which cannot be done directdly using CMake), so I was suggesting just use the Windows file explorer to manually copy from the CMake output, copy and paste into a libpqxx directory in C:\Program Files\ after manually making that libpqxx directory there. The point is that using Windows file explorer won't "fail" but rather demands of the user the proper credentials--then goes on to work without further interruption. It can be explained in documentation without further programming of any kind.

I don't think this is a really satisfactory answer for chenqingguo, who wants a smoother less-user involved process. But without writing a special GUI based install app just for windows, I don't think we are going to get that.

PS: I'm moving on to integrating pqxx and wxWidgits--we have a backlog of many programs to update for my client. Though I had a lot to learn here, but I think we resolved with a good plan and you guys have done some excellent work. The areas just changed in pqxx version 7 are going to be very useful in my next tasks--so tremendous thank you all and from me and my client and group as well.

PPS: Just started reading Jeroen's master's thesis--very interesting--hi jtv.

GordonLElliott commented 4 years ago

New problem

Now have an easy way to specify the PostgreSQL, have problem--I can't build a 32 bit app.

I'm not sure I care, but if somebody does, we should investigate.

Here are my CMake commands:

cmake  --verbose -S libpqxx-master -B "build-intermediate-32" -A Win32 -DCMAKE_INSTALL_PREFIX="libpqxx-install-32" -DPostgreSQL_ROOT="C:\Program Files (x86)\PostgreSQL\10"

cmake --build "build-intermediate-32" --verbose --target INSTALL

It definitely finds the correct PostgreSQL:

cmake  --verbose -S libpqxx-master -B "build-intermediate-32" -A Win32 -DCMAKE_INSTALL_PREFIX="libpqxx-install-32" -DPostgreSQL_ROOT="C:\Program Files (x86)\PostgreSQL\10"
-- Building for: Visual Studio 16 2019
-- The CXX compiler identification is MSVC 19.25.28614.0
-- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Professional/VC/Tools/MSVC/14.25.28610/bin/Hostx64/x86/cl.exe
-- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Professional/VC/Tools/MSVC/14.25.28610/bin/Hostx64/x86/cl.exe - works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found PostgreSQL: C:/Program Files (x86)/PostgreSQL/10/lib/libpq.lib (found version "10.12")
-- Looking for poll
-- Looking for poll - not found
-- Looking for PQencryptPasswordConn
-- Looking for PQencryptPasswordConn - not found
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Generating config.h
-- Generating config.h - done
-- Configuring done
-- Generating done

Response to attempting to use runner or unit_runner is: "The application was unable to start correctly (0xc000007b)."

Same response if I built my employee database program, connecting to the library made, and of course the 32 bit Postgresql.

I'm out of time on this one for the moment since I don't need 32 bit, sometime later I can download Lucasg/Dependencies and try to debug it.

NOTE: This problem is not unique to using the PostgreSQL_ROOT, it also occurs if use the three flags to specify PostgreSQL, and in either VS 2019 or VS2017.

GordonLElliott commented 4 years ago

Windows!!! What a complicated mess.

While the 64 bit ran, the 32 bit appears to have problem finding the Postgres .DLL library.

But the mess is generated by Visual Sudio because creating a default application it scrambles the location of its own execution files when having both 32 and 64 bit versions compile.

I suppose it will be simpler if one creates an application strictly one architecture, I'll try that later. Old version of Dependencies program showed that missing the libpq.dll is probably the problem. Older programs at run time would call out missing DLL's, so this version of tools (not yours but Microsoft) is even less helpful. I'm not sure if one can link the Postgresql library without DLL, that is not done that way on the CMake runner and unit_runner so it fails there.

I'll have to get back to this--just threw in the 32 bit test in my set of example programs for my development testing which were mostly 64 bit. SO don't assueme that a strictly 32 bit system can't be built if there are no mixed installations. And once again I need to do some paying work and 32 bit wasn't really important since upgrading all the apps to 64 now.

Do recognize there may be questions and problems in the future -- 64 bit works more correctly on first try with PostgreSQL 12 only and people should be pointed that direction.

tt4g commented 4 years ago

Sounds like a famous Windows DLL problem (DLL Hell).

Windows uses System32 and SysWOW64 when the 32-bit DLL and 64-bit DLL have the same name. When there are multiple DLLs with the same name but different architectures, it is necessary to devise to load the correct DLL.

See also: windows - 64-bit and 32-bit unmanaged dlls on the same system - best practices? - Stack Overflow

GordonLElliott commented 4 years ago

Not "like".... it is.

And you know that "SysWOW64" is for 32 bit DLL's etc! And System32 is for 64 bit? (Isn't that wonderfully easy to understand.)

But PostgreSQL doesn't install in either just their local. I think might work (not tried yet) to copy the respective--just of 32 and 64 bit--to the respective (weirdly named) places manually. I suppose that always the latest 64 bit DLL is the one to use, and always the latest 32 bit DLL is the one to use. So I could test and write up a recommended standard--and I am sure somewhere else the whole issue has been discussed extensively.

The main issue is the cryptic nature of failure. One cannot even step first line of code with debugger, and message is totally cryptic. As I mentioned, older systems used to complain that specific DLL was missing, and one could address that one. Here don't know if DLL bad (or which one if several) or if was method of linking libraries failed.

OK--tested runner (complete success) with copying libpq locally--presume the other ideas will work.

Interestingly enough, I caused my own "locality" issue (like the issue in CMake scripts) by setting my path to include the 64 bit PostgreSQL lib directory--which worked great when only running 64 bit. But probably directed the 64 bit DLL into the 32 bit version, and also explains why I had unreasonably good success with 64 bit working just as built. Having removed my non-locality I'll have to deal with all the DLL hell in a consistent manner.

I'll write something up...

HAVING THOUGHT: I think one should copy the required (and latest available) psql.dll to the execution directory of the application. AND furthermore do so when a user of pqxx builds their own app and has it installed on some 3rd party machine. Remember that the final install machine may not have any PostgreSQL installed at all! (It may strictly use ethernet or internet data sources.)

tt4g commented 4 years ago

Well, I wrote System32 and SysWOW64 as an example for DLL Hell (I previously encountered the issue of which folder the ODBC library was loaded from).

I think one should copy the required (and latest available) psql.dll to the execution directory of the application. AND furthermore do so when a user of pqxx builds their own app and has it installed on some 3rd party machine. Remember that the final install machine may not have any PostgreSQL installed at all! (It may strictly use ethernet or internet data sources.)

This idea is the simplest and most effective DLL Hell workaround, and I recommend it.

GordonLElliott commented 4 years ago

DLL Hell Report

OK, turns out that the Debug version is completely cryptic, but the Release version actually tells what DLL is missing.

Also turns out that either manually or by installation, versions may provide a path to the PostgreSQL bin directory. Which works out if one only has one installation. (That set so one can use psql and other utilities in batch or command windows with out a lot of typing.)

With all the paths to PostgreSQL removed, DLL's are needed, and we think the preferred method in that case is to place DLL's in the directory with the EXE. If one compiles both Release and Debug versions of the CMage generated pqxx library, the test executable directories include: \test\Release, \test\Debug, \test\unit\Release, and \test\unit\Debug. Then of course any application the user is developing which links to pqxx.lib.

To remove or modify the path, on my computer I wound up with so many path variables that the system setting command did not work, nor could I use command line tools to set in temporary basis. So I downloaded a free (donation requested proprietary) tool called Rapid Environment Editor, which worked very well (even more easily than system commands).

The x64 versions required the following DLL's copied from the latest version 12 PostgreSQL: libcrypto-1_1-x64.dll libiconv-2.dll libintl-8.dll libpq.dll libssl-1_1-x64.dll

The 32 bit versions built with Cmake -A Win32 of course can only use the latest version 10 of PostgreSQL in (x86) form. (Only version 10 of PostgreSQL has a 32 bit compile.) There the following DLL's were required: libiconv-2.dll libintl-8.dll libpq.dll

These libraries are found in the respective PostgreSQL\xx\bin directory. The \lib directory contains some of the DLL's but is not consisten with the requirements for runtime.

Once again, by compiling and executing the "Release" version, for some reason more information is provided at run time and one can see what DLL's are missing. This may not work if a path is provided such that the wrong DLL is loaded, however. But the CMake has now been tested to build both 64 and 32 bit libraries which successfully run all tests.

Someone building an application for Windows (32 or 64 bit) needs to know these DLL's and supply them along with their application for istallation of their application--especially in the case where the final target machine is not itself running the version of the PostgreSQL server that would have those DLLs. The application builder should probably create commands in the install utility for that application that copies the DLL's required by the specific libpq version compiled against into the application execution forlder, or have a strategy for handling the DLL issue.

jtv commented 4 years ago

@GordonLElliott with apologies, I am not able to keep up with the notes! I'm a bit stressed out with some other problems at the moment (you know, world in crisis...) and I feel terrible for how hard @tt4g has to work to keep up as well. He's a very helpful contributor and I don't want to wear him out. :-)

So I beg you, try to limit it to the most important parts, make them as easy as possible to read, maybe split things into separate tickets. It'll help me a lot.

Anyway, I'm adding the policy change to the CMakeLists.txt. I'm assuming I only need to do it in the top-level one, right? I already had the minimum CMake set to 3.7.

For compiling 32-bit x86 code, what I'd do on other platforms is pass either a specific compiler, or compiler & linker options to tell the compiler to generate different code. It's very flexible, but I have no idea whether that'll solve it with your compiler.

Oh, and... If you're reading my masters thesis (very flattering!) did you spot the joke? I'm still surprised by how many people did.

GordonLElliott commented 4 years ago

I have more or less completed a Windows batch file that installs libpqxx for Visual Studio.

The way it works is documented in the file. But basically you put the libpqxx-master directory with all the sorce and CMake just within into a directory with this batch file. Get the batch file into that directory by using a programming editor (thus correcting CR/LF) and changing parameters in the top as appropriate in the documented configuration secction. Double click and off it goes....

REM Build and install pqxx library using CMake and source code files.
:: ************************************************************************
:: ***  This batch file builds and then installs the pqxx library       ***
:: ***  (libpqxx), for Microsoft Visual Studio on Windows.              ***
:: ***  It should work for any version of libpqxx after 7.0.0, but      ***
:: ***  some features depend upon version 7.0.7 or later.               ***
:: ***                                                                  ***
:: ***  Usage:                                                          ***
:: ***  Use a programming editor (Visual Studio, Notepad++, etc.) to    ***
:: ***  copy this file to a directory from which the install will       ***
:: ***  occur. This should be a permanent directory if you intend to    ***
:: ***  build and use the Debug configuration, because debugging of     ***
:: ***  source code using the copy made here will step into the copy    ***
:: ***  of the source code in this directory. Example:                  ***
:: ***  Documents\libpqxx                                               ***
:: ***                                                                  ***
:: ***  This directory should also contain the libpqxx-master source    ***
:: ***  file directory. (Not the top level but the libpqxx-master which ***
:: ***  has source files immediately thereunder.) If not already there, ***
:: ***  then make it so. Rename with -<version> convention if used.     ***
:: ***                                                                  ***
:: ***  Using the programming editor, make any changes to the           ***
:: ***  configuration section of this file to tune to your              ***
:: ***  requirements. Each flag is documented and alternative values    ***
:: ***  are provided with REM comment or are uncommented as default.    ***
:: ***  Since GitHub has a tendency to provide files with Unix style    ***
:: ***  linefeeds (instead of the anachronistic windows CR/LF),         ***
:: ***  using your programming editor to both copy and edit this file   ***
:: ***  can also translate to the proper line terminations for a batch  ***
:: ***  file on Windows.                                                ***
:: ***                                                                  ***
:: ***  No command line arguments should be supplied to this batch file ***
:: ***  by the user. This file takes command line arguments from an     ***
:: ***  internally generated recursive call to itself with arguments:   ***
:: ***          CMAKE_INSTALL or COPY_INSTALL                           ***
:: ***  which will cause the file to skip from variable configuration   ***
:: ***  directly to the respective installation section--and presumes   ***
:: ***  that the (recursively called) file has administrative           ***
:: ***  privileges as needed to do the final installation step, such    ***
:: ***  as placing files in the C:\Program Files\ directory.            ***
:: ***                                                                  ***
:: ***  If you select both BUILD_DEBUG and BUILD_RELEASE below, then    ***
:: ***  the CMake process will be circumvented to copy the Debug        ***
:: ***  version of the pqxx.lib to a new Debug subdirectory of the      ***
:: ***  installation's lib directory. If you do not wish this, then    ***
:: ***  select only one of BUILD_DEBUG or BUILD_RELEASE.                ***
:: ***  An option allows the addition of the pqxx.psd file to that      ***
:: ***  Debug directory if desired to allow debug linking while the     ***
:: ***  source files are not available or are out of sync.              ***
:: ************************************************************************
@echo off
setlocal

:: Set this to 1 to show verbose results in CMake commands and allow
:: the batch file to echo its steps, to debug. Set to 0 for less output,
:: which will be much more readable.
SET VERBOSE=0

:: leave these lines in... VERBOSE block.... ----------------------------->>
if not defined VERBOSE ( SET VERBOSE=0 )
if %VERBOSE% neq 0 ( @echo on )
:: <<----------------------------------------  .... End of VERBOSE block. ::

:: Set this to pause at the end of batch file runs. This is needed if
:: you click on the batch file to run it. (This is the easiest way!)
:: But at the end the command window will close before you get a chance
:: to read the success or failure. This causes a pause to hit key before
:: end of the batch file. Default is on (1) if not defined here, off (0).
SET PAUSE_AT_END=1

:: Set the version number of the libpqxx being installed. This will be
:: used in a suffix in intermediate directories created in the install
:: process. (These intermediate directories are normally kept not deleted.)
:: Optionally the libpqxx-master directory used as input may have a suffix
:: of this version number, as libpqxx-master-<version>, so that multiple
:: versions may be maintained in this directory. Check SOURCE_DIRECTORY
:: setting if you do not name the source directory with version, but this
:: still applies to intermediate file directories.
SET PQXX_VERSION=7.0.7

:: The SUFFIX may be used to distinguish builds and libraries...
:: Recommend to match the suffix to the configuration below. This will be
:: used in intermediate file locations. Use the leading '_' if you want
:: the suffix to be set off with that character each place used.
:: Here we are assuming a 64 bit build, and no need to distinguish
:: otherwise. Set blank for no suffix (and no underscore).
REM SET SUFFIX=_x64
SET SUFFIX=
REM SET SUFFIX=_test

:: Working backward, define a final installation directory for the library.
:: This is the location you expect to use for your library settings in 
:: applications that you create. This directory can be in a privileged 
:: location, such as C:\Program Files. Use %programfiles% to let the
:: operating system variable supply the correct location for Program Files,
:: which is the location of 64 bit programs on a 64 bit Windows operating 
:: system.

:: NOTE this directory must exist, or an attempt will be made to create it
:: without administrator permissions. If that fails, the batch fails.
:: Comment the first and enable the second line to only "install" in the build
:: directory with version and suffix identifiers.
SET FINAL_INSTALL_DIRECTORY=%programfiles%
REM SET FINAL_INSTALL_DIRECTORY=FINAL_INSTALL_%PQXX_VERSION%%SUFFIX%

:: Subdirectory of the intermediate and final install directories into which 
:: this project is installed. This subdirectory will be created if necessary. 
:: If not defined, this defaults to 'libpqxx'.
SET INSTALL_SUBDIRECTORY=libpqxx

:: USE_ADMIN_CMAKE_FINAL_INSTALL:
:: Set this to use the CMake system to do the final install.
:: In this instance, that final install will always be run with
:: administrator privileges, and will request authorization before
:: recursively running this batch file to finalize the install.
:: Set to 0 or 1. Default 0. To install a local non-privileged directory,
:: just use INTERMEDIATE_INSTALL_DIRECTORY only (below), do not define 
:: FINAL_INSTALL_DIRECTORY, and turn USE_ADMIN_CMAKE_FINAL_INSTALL off (0).
::
:: Advantages:
::      No failure to trigger administrator login, no testing of
::      target location. Creates nested subdirectories if not
::      present. Uses CMake to set up, then uses MSBUILD function of
::      Visual Studio to implement.
:: Disadvantages:
::      NOTE: Must be set up in the early construction step, which establishes
::      the target location when building the Visual Studio solution.
::      Running Visual Studio compile and build facilities as 
::      administrator--this could be a security risk--creating and
::      running programs!
::      NOTE that if any build parameter has changed, the entire build
::      activity will be triggered here (running as administrator).
:: Alternative:
::      Set to 0, and use FINAL_INSTALL_DIRECTORY above to copy to the
::      program files or other final directory, using robocopy
::      (as administrator).
::
:: Default 0 -- do not use admin with CMake. 1 to use.
::      
REM SET USE_ADMIN_CMAKE_FINAL_INSTALL=1

:: Place for intermediate copy of the output library and include files.
:: Here we have used a subdirectory relative to the top build directory.
:: Use Windows conventions to set an absolute directory if desired,
:: HOWEVER do not attempt to directly install in  privileged directory
:: such as \Program Files, as this will fail with few options for remedy.
:: Copy to the privileged directory can be accomplished by setting
:: FINAL_INSTALL_DIRECTORY above.
::
:: Ignored if USE_CMAKE_ADMIN_FINAL_INSTALL=1, wherein this is set
:: to FINAL_INSTALL_DIRECTORY (if defined).
SET INTERMEDIATE_INSTALL_DIRECTORY=install_%PQXX_VERSION%%SUFFIX%

:: Relative directory for CMake build files, a Visual Studio solution file.
:: This intermediate directory will be created if missing, and will contain
:: the intermediate compiling steps, including a Visual Studio solution file
:: filled by multiple projects that build the library, test the library,
:: and construct an "INSTALL" directory or "PACKAGE" gzip of the library.
:: Various CMake files will also be kept there, allowing a true "make"
:: file character if the process is run again, only applying to changes.
SET BUILD_DIRECTORY=build__%PQXX_VERSION%%SUFFIX%

:: Source input directory. This has the source and CMake files immediately
:: in this directory. Typically relative to directory in which this
:: batch file is run. If the actual source files start 2 levels deep, then
:: edit to reflect the nested structure. Use (and rename to) the <version>
:: form to keep multiple versions in the same directory.
:: Recommend placing the libpqxx-master directory with source files
:: directly under in the same directory as the batch file so follwing
:: forms are correct.
SET SOURCE_DIRECTORY=libpqxx-master
REM SET SOURCE_DIRECTORY=libpqxx-master-%PQXX_VERSION%

:: Set Generator suitable for CMake -- specific to Visual Studio.
:: Note that Visual Studio 15 2017 is the first to support the C++ 17
:: required by pqxx version 7, so earlier generations are not included.
:: https://cmake.org/cmake/help/latest/generator/Visual%20Studio%2016%202019.html
:: https://cmake.org/cmake/help/latest/generator/Visual%20Studio%2015%202017.html
::
:: Leave undefined to let CMake choose the generator (generally the latest)
:: version on your machine, presuming your CMake itself is up to date.

REM SET GENERATOR=Visual Studio 16 2019
REM SET GENERATOR=Visual Studio 15 2017

:: Platform (architecture) used for target of compilation (32/64 bit etc.),
:: must be exactly Win32 or x64. If undefined, CMake will choose its default,
:: which generally turns out to be WRONG, (Murphy's law) so select your 
:: desired result here.
SET PLATFORM=x64
REM SET PLATFORM=Win32

:: The build computer must have at least one installation of PostgreSQL.
:: If the build computer has multiple installations of PostgreSQL, then
:: CMake will someimes find the wrong copy, and it needs to be specified
:: here. For PSQLROOT to work, the minimum release version of pqxx is 7.0.7.
:: Using the (x86) library means you have to set PLATFORM=Win32 above.
:: Leave undefined if you only have one installation of PostgreSQL and
:: CMake will find it. (Of course match the PLATFORM to that installation)
REM SET PSQLROOT=C:\Program Files\PostgreSQL\12
REM SET PSQLROOT=C:\Program Files\PostgreSQL\11
REM SET PSQLROOT=C:\Program Files\PostgreSQL\10
REM SET PSQLROOT=C:\Program Files (x86)\PostgreSQL\10

:: Define PSQLOC to establish the directory for PostgreSQL "the hard way."
:: This triggers three defines: PostgreSQL_LIBRARY,
:: PostgreSQL_INCLUDE_DIR, and PostgreSQL_TYPE_INCLUDE_DIR, so that
:: CMake searching for PostgreSQL is essentially overruled. Only a couple
:: of typical choices are shown:
REM SET PSQLOC=C:\Program Files\PostgreSQL\12
REM SET PSQLOC=C:\Program Files (x86)\PostgreSQL\10

:: Set cores to number of CPU cores on your machine, or 1 to disable. Set 8 
:: cores only shaves from 5' 20" to 4' compile time on an older computer.
REM SET CORES=8

:: Set this variable if you have a 64 bit operating system. It will shorten
:: the compile time. Otherwise set value to 0. Value must be 1 or 0.
:: The 0 setting lets CMake use default setting for host tool selection,
:: while 1 requests that CMake utilize 64 bit tools. (Does not determine
:: compiled output architecture.)
REM SET USE_64_BIT_HOST=1
REM SET USE_64_BIT_HOST=0

:: CHARSET must be exactly "Unicode" or "MultiByte" or else will not work 
:: correctly. NOTE this must be used in each case of a --build, even
:: if re-selecting the the --target project. This is used with the
:: direct to tool flag:   -- /property:CharacterSet=%CHARSET%
:: for command line CMake --build runs.

:: Default is MultiByte. But for example linking with wxWidgets, one may
:: need to use a Unicode compiled library.

REM SET CHARSET=MultiByte
REM SET CHARSET=Unicode
:: Note that the VS IDE solution is set for MultiByte, regardless of this 
:: setting. If you use Unicode and intend to compile using the IDE, you must 
:: change pqxx, runner, and unit_runner projects to Unicode character set,
:: from the property pages of those projects. Building the test applications
:: will recompile the library itself--if this is not done.

:: Select building of Release and Debug compiles. Set at least ONE!
:: If only one flag is set, then only the respective Release or Debug files
:: are written to the INSTALL directory. These flags must be set 1 or 0.
:: Setting both Debug and Release flags causes a special method of copying
:: the Debug library into a "Debug" subdirectory of the final install location.
:: Use 1 to enable compiling of that configuration, or else 0 to disable.
:: Building only Release means you don't have to track the source code with
:: the libraray, but disables any ability to step into the source code during
:: debugging of the applications built with pqxx.lib. MUST SET AT LEAST ONE!
:: I am not sure, but I think one might have to match the app compile to
:: the library configuration.
SET BUILD_DEBUG=1
SET BUILD_RELEASE=1

:: Turn on to have the pqxx.pdb file installed with the Debug library, 
:: only applies if both BUILD_DEBUG=1 and BUILD_RELEASE=1 is set above,
:: so that the debug library is in a special Debug subdirectory of lib. 
:: Some users may find this useful. Disabled by default. Set to 1 to use.
REM SET INSTALL_PDB=1

:: ************************************************************************
:: ************************************************************************
:: ************************************************************************
:: ***                      Operations Section                          ***
:: *** This section implements the build using the configuration.       ***
:: ************************************************************************
:: ************************************************************************
:: ************************************************************************

:: Note that a technique with command line arguments and a special auxiliary
:: method acquiring admin privileges is used for the last step. ANY command
:: line argument at all presumes that these privileges have been acquired,
:: and the special recursive call will not recur (avoiding infinite looping).
:: The acquiring of privileges causes a re-run of this batch file with the
:: privileges. The Configuration section above is run each pass to give the 
:: configuration variables their values in the local task.

echo **********************************************************************
echo * STEP 0: Flag processing, adjusts or configures variables           *
echo **********************************************************************

:: Flow control--check if called recursively (with argument) for 
:: final install:
if NOT "%~1"=="" (
    :: Got here through recursive call from original batch file.
    :: Since called with system login, wrong directory, fix
    :: by using the path of the batch file name iteslf:
    :: Use the batch file path to select the path, which would have
    :: been lost during the recursive call as administrator.
    :: https://ss64.com/nt/pushd.html re pushd "%~dp0"
    pushd "%~dp0"
    :: Continue with argument re-initialization...
    REM (This line seems to be required)
)

:: Flags that (may) appear in all CMake commands...

if not defined VERBOSE (
    SET VERBOSE=0
)
if %VERBOSE% neq 0 (
    @echo on
)
:: For use in CMake
if %VERBOSE% neq 0 (
    set VERBOSE_FLG=--verbose
) else (
    set VERBOSE_FLG=
)

if not defined PAUSE_AT_END (
    set PAUSE_AT_END=1
)

if not defined CORES (
    set CORES=0
)
if %CORES% neq 0 (
    set CORES_FLG=-j %CORES%
) else (
    set CORES_FLG=
)

:: Recursive call parameters (as administrator) requires an argument,
:: even if accidentally not provided below. Or else h**l breaks loose!
set params=DUMMY

:: Flags that appear on the initial construction CMake command...

:: Cmake host=x86 or host=x64, selct 32 or 64 bit tools (not compile target).
:: Note that this flag is used in the construction of the VS solution, and
:: will have its effect later in the build activity.
if not defined USE_64_BIT_HOST (
    set USE_64_BIT_HOST=0
)
:: Either choose 64 bit tools, or leave to default of CMake.
if %USE_64_BIT_HOST% neq 0 (
    set HOST_FLG=-T "host=x64"
) else (
    set HOST_FLG=
)

if defined GENERATOR (
    set GENERATOR_FLG=-G "%GENERATOR%"
) else (
    set GENERATOR_FLG=
)

:: Here choosing 32 or 64 bit target compile.
if defined PLATFORM (
    set PLATFORM_FLG=-A "%PLATFORM%"
) else (
    set PLATFORM_FLG=
)

if defined PSQLROOT (
    set PSQLROOT_FLG=-DPostgreSQL_ROOT="%PSQLROOT%"
) else (
    set PSQLROOT_FLG=
)

if defined PSQLOC (
    :: Selecting input library "the hard way", all these flags need to be set:
    set PSQLOC_FLGS=-DPostgreSQL_LIBRARY="%PSQLOC%\lib\libpq.lib" -DPostgreSQL_INCLUDE_DIR="%PSQLOC%\include" -DPostgreSQL_TYPE_INCLUDE_DIR="%PSQLOC%\include"
)else (
    set PSQLOC_FLGS=
)

:: Flags that appear on the build or install CMake command...

if not defined BUILD_DEBUG (
    set BUILD_DEBUG=0
)
if not defined BUILD_RELEASE (
    set BUILD_RELEASE=0
)

if not defined INSTALL_SUBDIRECTORY (
    set INSTALL_SUBDIRECTORY=libpqxx
)
if "%INSTALL_SUBDIRECTORY%" equ "" (
    echo INSTALL_SUBDIRECTORY must be defined an not empty string.
    echo for example to place in \Program Files directory, the install
    echo must be placed in a subdirectory as in C:\Program Files\libpqxx .
    set errorlevel=1
    goto error:
)

if not defined USE_ADMIN_CMAKE_FINAL_INSTALL (
    set USE_ADMIN_CMAKE_FINAL_INSTALL=0
)
if %USE_ADMIN_CMAKE_FINAL_INSTALL% neq 0 (
    :: If we are using CMake install process target the final install,
    :: and FINAL_INSTALL_DIRECTORY was defined, that is expected to
    :: establish the output directory.
    if defined FINAL_INSTALL_DIRECTORY (
        set INTERMEDIATE_INSTALL_DIRECTORY=%FINAL_INSTALL_DIRECTORY%
    )
)

:: Regarding usage of and (%BUILD_RELEASE% equ 0) NOTE AND is for binary 
:: variables not logic. Construct BUILD_DEBUG==0 and BUILD_RELEASE==0.
echo Build flags: BUILD_DEBUG=%BUILD_DEBUG%, BUILD_RELEASE=%BUILD_RELEASE%
if %BUILD_DEBUG% equ 0 (
    if %BUILD_RELEASE% equ 0 (
        echo Both BUILD_DEBUG and BUILD_RELEASE cannot be left undefined or 0.
        set errorlevel=1
        goto :error
    )
)

:: Note this is a "pass through" flag direct to toolset.
if defined CHARSET (
    set CHARSET_FLG=/property:CharacterSet=%CHARSET%
) else (
    set CHARSET_FLG=
)

:: NOTE that this '--' flag on CMake command line makes all following flags
:: passed to tool. Enable if any of those flags are non-empty.
:: Amend the if condition if more "pass through" flags are defined.
if defined CHARSET (
    set PASS_OPTIONS_FLG=--
) else (
    set PASS_OPTIONS_FLG=
)

if not defined INSTALL_PDB (
        set INSTALL_PDB=0
)

echo **********************************************************************
echo * STEP 0.5: Batch file control switching.                            *
echo **********************************************************************

:: ***  Logic flow control of this batch file:                          ***
:: ***  This batch file may be called recursively with an argument      ***
:: ***  which selects the final install operation. DO NOT continue      ***
:: ***  with normal operations in any case if any argument is present.  ***
:: ***  This recursive call will occur with administrative privileges.  ***

if NOT "%~1"=="" (
    echo CALLING RECURSIVELY with argument: %~1
)

if "%~1"=="CMAKE_INSTALL" goto :cmake_install_only
if "%~1"=="COPY_INSTALL" goto :copy_install_only
if NOT "%~1"=="" (
    echo The recursive command line arguments must be one of:
    echo CMAKE_INSTALL or COPY_INSTALL.
    set errorlevel=1
    goto :error
)

::   * Build occurs in three steps. (Once variables are initialized.)      *
echo **********************************************************************
echo * STEP 1: Creation of Visual Studio solution and build files.        *
echo **********************************************************************

:: ***  Actual  start of processing !                                   ***

if "%GENERATOR%"=="" (set PRT_G=[NONE]) else (set PRT_G=%GENERATOR%)
if "%CHARSET%"=="" (set PRT_C=[NONE]) else (set PRT_C=%CHARSET%)
echo Start Generator: %PRT_G%, char %PRT_C% build, cores: %CORES% >> build_time_log.txt
echo Start build: %time% >> build_time_log.txt

:: This batch file can be started 'cold' with no settings, but must be in the
:: property directory relative to the directories specified.

:: This step assembles a Visual Studio solution with several projects.
:: It does not 'build' those projects yet, only the project files are created.
:: However several settings are established, including the location of
:: the "INSTALL" directory which will be used much later...

:: This cmake constructs the Visual Studio solution, without building projects:
cmake %VERBOSE_FLG% -S %SOURCE_DIRECTORY% -B "%BUILD_DIRECTORY%" %GENERATOR_FLG% %PLATFORM_FLG% %HOST_FLG% %PSQLROOT_FLG% %PSQLOC_FLGS%  -DCMAKE_INSTALL_PREFIX="%INTERMEDIATE_INSTALL_DIRECTORY%\%INSTALL_SUBDIRECTORY%"

if %errorlevel% neq 0 goto :error
:: Note that the IDE opens as Debug configuration no matter what.
:: Note that the IDE projects are configured with MultiByte character set,
:: no matter which setting was used for build operations by this batch file.
:: However the project will be Win32 or x64 according to the initial 
:: construction.

echo **********************************************************************
echo * STEP 2: Build of Visual Studio solution build files.               *
echo **********************************************************************

:: ************************************************************************
:: *** Note this will not include the "INSTALL" or "PACKAGE"            ***
:: *** steps that create the final install output. This step is ***
:: *** strictly not required, but is used to simplify the 3rd step      ***
:: *** such that all programs are pre-built.                            ***
:: ************************************************************************

:: Usage: cmake --build <dir> [options] [-- [native-options]]
:: Options:
::   <dir>          = Project binary directory to be built.
::   --parallel [<jobs>], -j [<jobs>]
::                  = Build in parallel using the given number of jobs.
::                    If <jobs> is omitted the native build tool's
::                    default number is used.
::                    The CMAKE_BUILD_PARALLEL_LEVEL environment variable
::                    specifies a default parallel level when this option
::                    is not given.
::   --target <tgt>..., -t <tgt>...
::                  = Build <tgt> instead of default targets.
::   --config <cfg> = For multi-configuration tools, choose <cfg>.
::   --clean-first  = Build target 'clean' first, then build.
::                    (To clean only, use --target 'clean'.)
::   --verbose, -v  = Enable verbose output - if supported - including
::                    the build commands to be executed.
::   --             = Pass remaining options to the native tool.
::     (On pass options):
::     https://docs.microsoft.com/en-us/visualstudio/msbuild/msbuild-command-line-reference?view=vs-2019       

::     NOTE that -j parameter more than 1 allows for faster compiling.
::     Set no higher than number of cores on your computer.
::     Note that addition of -- /p:CharacterSet=Unicode
::     does change from _MBCS to UNICODE, which can be observed in the
::     CL command line used (as shown by --verbose) in compile,
::     but does not change the MS solution file in the IDE.

::     Keeps prior work for the most part (like a makefile), can
::     specify --clean-first to contradict that.

::     Although the build and "INSTALL" tasks can be combined, here
::     we prefer to pre-build all projects that will be used.
::     This is the long compile time that may take several minutes....
::     (Note at least one will be defined.)

if %BUILD_RELEASE% neq 0 (
    cmake --build %BUILD_DIRECTORY% %VERBOSE_FLG% --config Release %CORES_FLG% %PASS_OPTIONS_FLG% %CHARSET_FLG%
if %errorlevel% neq 0 goto :error
)

if %BUILD_DEBUG% neq 0 (
    cmake --build %BUILD_DIRECTORY% %VERBOSE_FLG% --config Debug   %CORES_FLG% %PASS_OPTIONS_FLG% %CHARSET_FLG%
if %errorlevel% neq 0 goto :error
)

echo **********************************************************************
echo *          STEP 3: Creation of local copy of install library files.  *
echo *          Exception is use with administrative privileges.          *
echo *          Note this assembles library, include, and help files.     *
echo **********************************************************************
if %USE_ADMIN_CMAKE_FINAL_INSTALL% neq 0 (
    :: Call an external process to trigger this batch file, run as
    :: administrator. Passes argument CMAKE_INSTALL which causes
    :: flow to only go to :cmake_install_only in recursively called
    :: batch file.
    set params=CMAKE_INSTALL
    goto :CallAsAdministrator
)

:cmake_install_only

:: NOTE can also get here by recursive call with an argument, used
:: strictly for processing with administrative privileges.

::     ********************************************************************
::     **** Below this point, requires administrator privileges,       ****
::     **** if that is required by the INTERMEDIATE_INSTALL_DIRECTORY. ****
::     ********************************************************************

::     Install one configuration in the normal manner. This does the
::     heavy lifting of include and doc directories as well as lib with 
::     configured library, also in the normal CMake manner. Always performed!
::     That said, the Release copy is used for this primary install unless
::     only Debug is defined, in which case Debug is used instead.
::     The CONFIGURATION will be Release if either both or just Release
::     is defined. (Setting neither is not allowed.)
if %BUILD_RELEASE% equ 0 (
    set CONFIGURATION=Debug
) else (
    set CONFIGURATION=Release
)

:: Note after %PASS_OPTIONS_FLG% any options go directly to tool, not CMake.
cmake --build %BUILD_DIRECTORY% %VERBOSE_FLG% --target INSTALL --config %CONFIGURATION% %PASS_OPTIONS_FLG% %CHARSET_FLG%
if %errorlevel% neq 0 goto :error

::      Next step optional, if using Debug configuration. Circumvent CMake!
::     Here copy the Debug directory   (Return 0 up to date, 1 copied OK)
set COMB_DEB_REL_FLAGS=1
if %BUILD_DEBUG% equ 0 (set COMB_DEB_REL_FLAGS=0)
if %BUILD_RELEASE% equ 0 (set COMB_DEB_REL_FLAGS=0)
:: COMB_DEB_REL_FLAGS is set only if both debug and release selected.
if %INSTALL_PDB% neq 0 (
    set INST_DEB_FILES=*.*
) else (
    set INST_DEB_FILES=*.lib
)

if %COMB_DEB_REL_FLAGS% neq 0 (
robocopy "%BUILD_DIRECTORY%\src\Debug" "%INTERMEDIATE_INSTALL_DIRECTORY%\%INSTALL_SUBDIRECTORY%\lib\Debug" /MIR /E /R:0 /DCOPY:T /Z %INST_DEB_FILES%
if %errorlevel% gtr 1 goto :error
)

:: Batch logic: If got here by recursive call with arguments, this
:: was all that was run, now pause and exit.
if NOT "%~1"=="" goto :exiting

echo **********************************************************************
echo *          STEP 4: Copy to final install library files.              *
echo *          Note this is what you link applications against.          *
echo **********************************************************************

::     ********************************************************************
::     **** Below this point, requires administrator privileges,       ****
::     **** if that is required by the FINAL_INSTALL_DIRECTORY.        ****
::     ********************************************************************

if not defined FINAL_INSTALL_DIRECTORY goto :exiting

:: First check if the directory (including subdirectory) exists.
:: If it does not exist, create it. (This will not occur in program files 
:: directory.) Then after creating directory, will test if privileges to write.
set HAS_FINAL_DIRECTORY=0
if exist "%FINAL_INSTALL_DIRECTORY%\" (
    echo final install directory exists: %FINAL_INSTALL_DIRECTORY%
) else (
    echo final does not exist
    set HAS_FINAL_DIRECTORY=1
    md "%FINAL_INSTALL_DIRECTORY%"
    echo Created: %FINAL_INSTALL_DIRECTORY%
    echo with errorlevel: %errorlevel%
    :: if %errorlevel% neq 0 goto :error
    :: NOTE does not get to exit with error, and DID create directory!
)

if not exist "%FINAL_INSTALL_DIRECTORY%\" (
    echo After creating final directory: %FINAL_INSTALL_DIRECTORY%
    echo the directory does not exist.
    set errorlevel=1
    goto :error
)

:: Test that we have privileges to write to the output folder:
:: Method taken from:
:: https://stackoverflow.com/questions/7272850/best-way-to-check-if-directory-is-writable-in-bat-script
copy /Y NUL "%FINAL_INSTALL_DIRECTORY%\.__writable" > NUL 2<&1 && set INSTALL_WRITEOK=1
IF DEFINED INSTALL_WRITEOK ( 
    rem ---- we have write access ----
    echo we have write access !!!!!!!!!!!!!!!!!!!
    erase "%FINAL_INSTALL_DIRECTORY%\.__writable"
    echo did erase command of .__writable
    goto :gotPrivileges
) else (
    rem ---- we don't ----
    echo we do not have access ******************
    set params=COPY_INSTALL
    goto :CallAsAdministrator
)

:gotPrivileges

:copy_install_only

:: Had...    include lib share    didn't copy files in them...

if defined FINAL_INSTALL_DIRECTORY (
    :: Got here with final install or because we think we are ready:
    :: And furthermore have a defined final install directory:
    robocopy "%INTERMEDIATE_INSTALL_DIRECTORY%\%INSTALL_SUBDIRECTORY%" /MIR /R:0 "%FINAL_INSTALL_DIRECTORY%\%INSTALL_SUBDIRECTORY%"
    echo The robocopy error: %errorlevel%
    if %errorlevel% gtr 1 goto :error
    echo
    echo Examine above for errors...
    echo Robocopy does not report errors and this batch file
    echo may not report them.
)
goto :exiting

::   *********************************************************************
:CallAsAdministrator
echo *********************************************************************
echo *                Trigger recursive call as administrator            *
echo *********************************************************************

REM 4 lines "%temp%\getadmin.vbs"
    :: Here we run special program to get privileges, and
    :: recursively call this same batch file with an argument:
    echo Set UAC = CreateObject^("Shell.Application"^) > "%temp%\getadmin.vbs"
    :: set params= %*
    :: set params=COPY_INSTALL
    echo UAC.ShellExecute "cmd.exe", "/c ""%~s0"" %params:"=""%", "", "runas", 1 >> "%temp%\getadmin.vbs"

    echo CALLing final installation with privileges....
    "%temp%\getadmin.vbs"
    REM erase "%temp%\getadmin.vbs"
    goto :exiting

::   **********************************************************************
::                         Successful exit.
:exiting
echo **********************************************************************
echo *                 Error and exit processing                          *
echo **********************************************************************
:: You can double click this file in the top level directory for the project,
:: and it pauses so the command window does not go away before you see the 
:: results. The build_time_log.txt file shows start and end times of the 
:: batch runs. Turn off PAUSE_AT_END (0) to disable pause.
echo End   build: %time% *** %~1>> build_time_log.txt
if %PAUSE_AT_END% neq 0 pause
goto :EOF

:error
echo Failed with error #%errorlevel%, at %time% *** %~1>> build_time_log.txt
echo Failed with error #%errorlevel%.
if %PAUSE_AT_END% neq 0 pause
exit /b %errorlevel%
jtv commented 4 years ago

That's massive! I'm guessing this could help a lot of people, but maintaining it is going to be difficult as circumstances change. Maybe you could put this up in your own repo? Something like "easy libpqxx builds in Visual Studio" perhaps.

By the way, one small thing I noticed scanning through the script: "psql" is the postgres command-line client. The client library ls called "libpq".

GordonLElliott commented 4 years ago

Thanks, Jtv.

I think that the CMake command line flags will be long-lived, and supposedly the techniques are good all the way back to Windows 7. (I think the batch file will work on 7, have not tested.) However there are about 2-3 places where I use the specific locations that CMake chooses to put the intermediate files (specific to Windows verson also). Still fixing minor issues, mostly cosmetic. (Like forgot to delete the temporary file used to detect if folder was writable, was commented out in copy above.) I could capture all those CMake-specific locations into their own variables (say relative to the CMake project locations which is established in the batch file), and put them just after the user configuration section, to make easier to change if CMake changes. This is only useful with CMake 3.13 and later (because that is necessary to even use minimum Visual Studio 2017).

I could do that (repository).

The other problem is the DLL H**L problem. I could also offer a (PostgreSQL) version specific method of gathering the PostgreSQL DLL's into a specified project location and a post-build batch file to go with that to automatically copy to user's project's EXE directory. Something like that is even needed to go with that install so that the build directory's runner and unit_runner tests will function. (How much time would it take to learn how to get CMake to do that?? CMake already knows where PosgreSQL is... Especially after pointing here, it knows the exact version for the user.)

I was using PSQL as prefix for PostgreSQL in several variables, for example batch variable PSQLROOT referring to PostgreSQL_ROOT setting. I'll have to think about that naming convention. (We use psql all the time, would want to avoid confusion.)

And as to why I did it: I use practially every flag, had to figure most of it out anyway. Just adding the automated administrator switch...... (Lost some hair over that...) My CMake was choosing some of the locations incorrectly--probably based on older CMake upgrade leaving old data. Then I use Unicode character set, and had multiple PostgreSQL installs.

By the way, after the first corrected run on most of the locations--further CMake runs (scratch, no intermediates) without any flags set chosses the correct locations in almost all cases.

jtv commented 4 years ago

I'm not sure I understand that last paragraph correctly: are you saying that CMake remembers the correct location settings after its first run?

There doesn't seem to be a single good standard for abbreviating PostgreSQL. :( There are several: postgres, pg, pq. (Hence the name "libpqxx," which was built as a replacement for the older "libpq++".)

GordonLElliott commented 4 years ago

I don't know the answer on saved data, I will tell the history.

SIMPLIFIED result: With some install history, the minimal CMake of pqxx finds the wrong settings. However latest everything -- NO storage of last PostgreSQL version.

First each "run" case described is delete the directory with CMake output, and even occasionally copy the libpqxx-master in again so any files accidentally left there were no longer.

Old CMake like 3.7??? Anyway too old for VS 2019.

Have PostgreSQL Versions: 10 (64 bit) 10 (32 bit) Install PostgreSQL Version 12 (64 bit) (Might be out of order here) Install VS 2019 Install VS 2017 (Note still old CMake, I don't realize that is an issue) Download pqxx V 7 Try: Result CMake on pqxx: Uses Visual Studio 2015 (yes 20 "15") Finds PostgreSQL V 10 in 32 bit (x86) directory.

Upgrade to latest CMake V 3.17.1 CMake on pqxx: Don't remember exactly, but chooses approximately: Visual Studio 2017 PostgreSQL V 10 (x86) (the 32 bit version) May have attempted made a 32 bit app. I think was using the 32 bit tooset (but this is weird as the 64 bit toolset is in (x86) directory) Failed to link (so one of the above is wrong, as that combinatin would link, but you get the idea several settings wrong)

Uninstalled all but PostgreSQL 12 (64 bit) Experiment... Reinstall PostgreSQL 10 (32 bit) So now there is a 32 bit V 10 to choose from....

Now ran my batch above selecting 64 bit, PostgreSQL 12 (64 bit), which succeeds.

Now run this simple batch file (new directory, clean libpqxx-master from download from github:

REM simplified batch or command line test of CMake of pqxx:
cmake -S libpqxx-master -B build_intermediate -A x64 -DCMAKE_INSTALL_PREFIX="libpqxx"

echo Errorlevel %errorlevel%

cmake --build build_intermediate --target INSTALL

echo Errorlevel %errorlevel%
pause

Result succeeds, finds VS 2019, Finds PostgreSQL Version 12 (64 bit),

SOMEWHERE in the computer's state is memory such that the result changed (past tense). However I can't say that CMake is storing that -- could be the install order changed so the search is different. Furthremore I now cannot reproduce failures (see next).

I DID a test: Simple batch for Win32 referencing PostgreSQL V 10 (x86), then generic batch (above) no specification. In this case the generic found the correct V 12 ..(64 bit).. version -- in this case no storage affecting default of latest version.

GordonLElliott commented 4 years ago

Here is an almost minimal explanation for building and installing pqxx using CMake on Windows for Visual Studio 2017 or 2019, which will work for almost everyone:

Copy the libpqxx-master directory into a "build" directory in your documents or development area.

Copy these lines into a "build.bat" file in that build directory, using a good programming editor which will fix any problem with line ends: (Now tested!)

@echo off
cmake -S libpqxx-master -B build_intermediate -DCMAKE_INSTALL_PREFIX="libpqxx"
if %errorlevel%==0 (
  cmake --build build_intermediate --target INSTALL
)
echo Errorlevel: %errorlevel%
echo Please copy libpqxx directory to %programfiles%
echo Asks permission to copy. Point your application build to those files.
pause
exit /B %errorlevel%

Then click (or double click) on the build.bat file. Watch for any errors, but this should work for most users. (Then my complex solution should solve most problems for people with more complex situations. Be sure you have the latest CMake installed for Windows if having any difficulty. Be sure that the source files are just under the libpqxx-master directory.)

NOTE:

NOTE -A option is important. When this option is not specified, even if you execute CMake on 64bit OS, the generated project file is for 32bit OS.

I am not showing that this is a normal problem. Yes I had all kinds of problems, and when one then many problems requiring more extensive fix. But when things seem "normal" it doesn't even need the -A flag. Even ran as 64 bit without flag, immediately after a Win32 build was run.

tt4g commented 4 years ago

Old CMake like 3.7??? Anyway too old for VS 2019.

Upgrade to latest CMake V 3.17.1

CMake 3.7 was released in 2016. In 2016 PostgreSQL only existed 9.x, so CMake 3.7 may not be able to detect it because it doesn't know PostgreSQL version 10 and above.

See FindPostgreSQL.cmake: https://github.com/Kitware/CMake/blob/v3.7.0/Modules/FindPostgreSQL.cmake#L81-L94

SOMEWHERE in the computer's state is memory such that the result changed (past tense). However I can't say that CMake is storing that -- could be the install order changed so the search is different. Furthremore I now cannot reproduce failures (see next).

CMake has cache file. If change build settings, must clear cache file CMakeCache.txt in build directory.

GordonLElliott commented 4 years ago

CMake has cache file. If change build settings, must clear cache file CMakeCache.txt in build directory.

That is a local directory of the build process?

I switched from using the forms in which CMake was run from one of the process directories to the -S -B form, and things did get better. But I don't remember when. But I did completely delete everything and start with new inputs and outputs -- but don't remember how many problems I was still having.

I think that the -S -B form of the construction step is important (and should be recommended) because it keeps from getting confused. The input is libpqxx-master, the output is one (or two) additional subdirectories that are not inside of libpqxx-master. Thus the input stays clean. And the output state (including "cache"???) is stored in those new output directories. Clean them away (which is safe--keeping any batch files if used and keeping libpqxx-master input uncontaminated) and then can safely start over, hopefully without saving state.

MY troubles were multi-dimensional including lack of experience... So I'm thinking that a "clean" methodology will work for almost everyone using minimal methods (until the customer has a complex situation).

I am still trying to get at a minimal explanation and setup method that works for almost everyone. Whether or not a batch file, I think the two lines:

cmake -S libpqxx-master -B build_intermediate -DCMAKE_INSTALL_PREFIX="libpqxx"

then followed by

cmake --build build_intermediate --target INSTALL

These are a "clean" methodology without changing directories (which leads to getting mixed up). No administrative priviledges are required by these steps (so far). The final output is appropriately named 'libpqxx' directory with only the files needed for the application builder (of course until stepping into a debug operation).

Generally speaking there are two methods of getting those two lines executed by the customer:

  1. Put files into directory and open command window and CD to location... Copy paste, or type the commands. OR,
  2. Put batch file with the commands next to the input directory. Click the batch file (or CD to it in command window as before and type name).

NORMALL I would not quote forward in a thread, but I wanted to document what tt4g said (in case I wind up writing something) which is the terminology for this:

By the way, this build method is called CMake out-of-source build. Building in the source directory is called CMake in-source build.

tt4g commented 4 years ago

CMake has cache file. If change build settings, must clear cache file CMakeCache.txt in build directory.

That is a local directory of the build process?

Local directory that is specified by -B.

tt4g commented 4 years ago

I think that the -S -B form of the construction step is important

By the way, this build method is called CMake out-of-source build. Building in the source directory is called CMake in-source build.

GordonLElliott commented 4 years ago

Possible documentation change for Windows build: [Edit 5/8-more compatible with CMake documentation: use "binary" for intermediate] [Eidt 5/9-refer to top-level directory with source, binary, and install subdirectories--more consistent with common terminology]

On Microsoft Windows

There used to be custom Visual C++ project files and Makefiles in libpqxx, but these are no longer supported.

Instead, use the CMake build.

One problem specific to Windows is that apparently it doesn't let you free memory in a DLL that was allocated in the main program or in another DLL, or vice versa. This can cause trouble when setting your own notice handlers to process error or warning output. Recommended practice is to build libpqxx as a static library, not a DLL.

CMake is available for several compilers for Windows, including Visual Studio. [Insert relevant comments other compilers] Only Visual Studio 2017 and later support the C++ 17 standard required by libpqxx 7.

Cmake Build for Visual Studio

An easy way to build a static libpqxx library on Windows for Visual Studio is an "out-of-source build" using CMake, in which source and build files are subdirectories of a top-level directory. (Check that your version of CMake is up to date.) In this method you put the input source files subdirectory (libpqxx-master) into the top-level directory. Then in this top-level directory execute the following two commands:

cmake -S libpqxx-master -B binary -DCMAKE_INSTALL_PREFIX="libpqxx"

This constructs a Visual Studio solution with multiple projects and CMake build information, in the 'binary' subdirectory, but does not build those projects. The CMAKE_INSTALL_PREFIX parameter establishes the final install directory in advance as a third subdirectory 'libpqxx', of the build directory. This cannot be located in the protected Program Files directory without running CMake as administrator. Leaving off the CMAKE_INSTALL_PREFIX parameter usually causes CMake to default to C:\Program Files (x86)\libpqxx in the next step. This is often the wrong install subdirectory, requires administrative privileges or will fail, and those privileges are not granted just by logging in as admin.

Then

cmake --build binary --target INSTALL

will build the projects, including copying the final installation files into the libpqxx subdirectory of your build directory, as previously established. That libpqxx subdirectory can be manually copied into the C:\Program Files directory if desired, to give it a standardized location that may be expected on Windows. A manual copy wil ask for administrative rights.

A version of PostgreSQL must be installed on the build computer, at least the libraries must be installed. See the notes on additional flags below if the first cmake command cannot locate the correct PostgreSQL library.

One way to automate this build process is to copy the following lines and paste into a good programming editor, which will fix any Windows new line problems. Then save as build.bat batch script file, into your top-level directory. Click on the batch file to automate the build process:

REM Automated build of libpqxx placing install files in libpqxx subdirectory.
cmake -S libpqxx-master -B binary -DCMAKE_INSTALL_PREFIX="libpqxx"
if %errorlevel%==0  cmake --build binary --target INSTALL
pause

Any changes can be made by clicking the batch build file again and only necessary changes will be compiled, as is normal for a "make" process.

Flags that can be added on the first cmake line include: -A x64 or -A Win32 These will select the target architecture as 64 or 32 bit. -G "Visual Studio 15 2017" may be used to select the 2017 version of Visual Studio, or explicitly select -G "Visual Studio 16 2019" (default if installed). The architecture and generator parameter values must be exactly as shown. Use -DPostgreSQL_ROOT="C:\Program Files\PostgreSQL\12" to select the version of PostgreSQL at the specified location. To compile for Win32, the "C:\Program Files (x86)\PostgreSQL\10" version of libpq will probably be required from the x86 directory (and include -A Win32).

The second (build) cmake command can also have: --config Release to switch the install from a Debug to a Release configuration.

A slightly more complex sequence of commands will build (completely separate) Debug and Release installations:

REM Automated build of libpqxx, both Debug and Release
cmake -S libpqxx-master -B binary
if %errorlevel%==0  cmake --build binary --config Debug
if %errorlevel%==0  cmake --build binary --config Release
if %errorlevel%==0  cmake --install binary --config Debug --prefix "libpqxx/Debug"
if %errorlevel%==0  cmake --install binary --config Release --prefix "libpqxx/Release"
pause
GordonLElliott commented 4 years ago

I've discovered another improvement to make programming easier on Windows with Visual Studio. This is called a "property sheet". Rather than having to make entries for the libraries and directories needed, the "property sheet" has those pre-loaded one time.

The file here should be saved as libpqxx.props, in the base directory where the lib, include, etc subdirectories are placed for the installation:

<?xml version="1.0" encoding="utf-8"?>
<!--
    This is a property sheet to be included in MSVS projects of the applications
    using libpqxx. Use "View|Property Manager" and choose "Add Existing
    Property Sheet..." from the context menu to add it from the IDE.
  -->
<Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
  <ImportGroup Label="PropertySheets" />
  <PropertyGroup Label="UserMacros" />
  <PropertyGroup />
  <ItemDefinitionGroup>
    <ClCompile>
      <AdditionalIncludeDirectories>$(MSBuildThisFileDirectory)include;%(AdditionalIncludeDirectories)</AdditionalIncludeDirectories>
    </ClCompile>
    <Link>
      <AdditionalLibraryDirectories>$(MSBuildThisFileDirectory)lib;%(AdditionalLibraryDirectories)</AdditionalLibraryDirectories>
      <AdditionalDependencies>pqxx.lib;C:\Program Files\PostgreSQL\12\lib\libpq.lib;wsock32.lib;ws2_32.lib;%(AdditionalDependencies)</AdditionalDependencies>
    </Link>
  </ItemDefinitionGroup>
  <ItemGroup />
</Project>

Now notice that the cross-reference to the PostgreSQL library is hard coded, in my case here to version 12. To auto-generate this file, I am sure that a CMake procedure can be developed, I will look into that in the future. For now one can hand edit.

As to the location of the libpqxx files themselves, the loading of the property sheet from a specific location allows the "MSBuildThisFileDirectory" macro to be filled, so this file does not need to be adjusted for any of the other locations as long as the relative subdirectory structure is maintained.

To use, the Visual Studio user select the "Property Manager" to view the project. The various configurations show up including combinations of X64 and X86 and Debug and Release. Use "Add Existing Property Sheet..." by right clicking over the respective configuration, and select the file as saved above.

Now still have to declare compiler standard ISO C++ 17!

Also note that one has to match the library build configuration (Debug, Release) to the project configuration. (I have that batch file that keeps separate libraries, working on integrating that....) For now also could keep separate complete Debug and Release install directories (see above post), and then place the property sheet in each root directory for respective configuration--and insert the respective property sheet from the respective configuration in VS.

This is much easier than having to set up all those settings each time a new project is started.

jtv commented 4 years ago

Great! You've done so much work on this, I think it's definitely worth a repository of its own, where you can work freely. I could link to it from the README.

GordonLElliott commented 4 years ago

Great! You've done so much work on this, I think it's definitely worth a repository of its own, where you can work freely. I could link to it from the README.

Great, Thanks! I have started a junk repository with (incomplete) versions just for getting used to all that. And working on a new version of the batch file. And the property sheet can also do a "Post Build" step that copies a list of DLL's, (which need to go with the property sheet location) so that it becomes simple to build a new app. (Batch to compile pqxx, property sheet for new app, and gather proper list of DLL's from PostgreSQL. MIGHT even copy the PostgreSQL library there, and that fixes the problem of knowing the location within the property sheet. Anyway you see several things to do before ready for prime time. The new version of the pqxx compile batch file uses CMake for virtually every step rather than "robocopy" so it will be more consistent with all operating systems and not dependent upon updates except for CMake itself.)

Thanks again

GordonLElliott commented 4 years ago

Progress (or un-progress) on batch file to build and "install" libpqxx

I have a batch file which does several things very successfully -- and then a most spectacular failure!

The successes:

  1. Configure, Build, and organize "installation" files for libpqxx, with both Debug and Release copy.
  2. Allow configuration of various parameters by commented variables, including Multibyte vs Unicode character set, and selecting compilers and platform within the Visual Studio offerings.
  3. Easy to edit to specify the final "installation" location.
  4. Gather DLLs from PostgreSQL that are needed by the libpq Postgres library. Selected by Postgres version (though have not yet researched the list for each version--in progress--among 10,11,12--tested and working with version 12).
  5. Property Page (as it is called) to make creating a new project in Visual Studio that uses libpqxx super easy--just insert property page from the install location at it will automatically connect all include, library files as well as (optionally) copy the DLL files required by Postgres.
  6. I know how to automatically trigger request to become administrator to install library, property pages, and documents in "Program Files" directory, to be comparable to the PostgreSQL installation--but see below.

Now the spectacular failure:

I was using a copy tool from Windows called 'robocopy' to copy the files. There was an option to make target directory look just like the source directory--including deleting items not intended.

I used that method, but made a simple mistake, and it was triggered as administrator. Apparently I tried to make my entire "Program Files" directory look like the install of the library--empasis on entire. Having some trouble getting the admin level part working, I finally got it to start--and noticed the files being listed didn't look like mine. Turns out it had started making my entire program files directory look like the library (only) and started with deleting the A's. Luckily they were all Adobe files on a subscription and I restored them with relative ease.

But this points out the difficulty of having an "installation" that occurs in the privileged file directories. (The same sort of mistake could wipe out program source, other areas, without security issues--a tool is a tool and needs to be used correctly and used incorrectly can be dangerous--a fact of life.)

So not quite ready, my next approach will be a script that can be run at admin level, but display the script for the user before starting so user can examine its simplicity, then simplify the script that is run to use only something called "xcopy" on Windows, with target of a simply copy of the install to a location in the Program Files directory. Removal of files will become a responsibility of user by using file explorer to delete the entries--not a complete installation method with uninstall.

GordonLElliott commented 4 years ago

Easy PQXX Build for Windows Visual Studio

My long awaited batch build system is now online:

https://github.com/GordonLElliott/Easy-PQXX-Build-for-Windows-Visual-Studio

See the repository for documentation. Major features:

  1. Automated batch file to build and install in the selected location (typically C:\Program Files\libpqxx), asking for administrator privileges if needed to copy the files.
  2. Configure for Unicode or MultiByte character sets. (For example Unicode character set may be needed to interface wxWidgets in the same application.)
  3. Keep named installations for different configurations.
  4. One library installation with both Debug and Release compiles that automatically switch in Visual Studio.
  5. Property Sheets so your Visual Studio application (with both Debug and Release compiles) can be instantly configured.
  6. Gather DLLs from PostgreSQL library into your application automatically.

Additionally various configuration solving selections can be made, for example if you have more than one PostgreSQL library or want to compile for more than one version of Visual Studio.

I repaired the possible danger issue by switching to XCopy which is more bug free and has fewer options to cause problems. It basically just copies the directory to the target in the Program FIles subdirectory for libpqxx.

JTV, don't reference it yet, but if you know some users who would be willing to try it out in Windows first, I would appreciate that. Needs work on docu,mentation before ready for prime time.

GordonLElliott commented 4 years ago

@jtv said:

That's massive! I'm guessing this could help a lot of people, but maintaining it is going to be difficult as circumstances change. Maybe you could put this up in your own repo? Something like "easy libpqxx builds in Visual Studio" perhaps.

and

Great! You've done so much work on this, I think it's definitely worth a repository of its own, where you can work freely. I could link to it from the README.

Jtv, reading the license for libpqxx, it appears I might need permission from you to name and promote my batch file for installing PQXX .

Thanks Gordon

GordonLElliott commented 4 years ago

Jtv, I went over the documentation in BUILDING-cmake.md, specifically trying on Windows. There really are several problems, much better to use CMake for each of the 3 steps. In BUILDING-cmake.md:

If you just wawnt to get it built and installed quickly, run cmake from the root of the libpqxx source tree. This configures your build.

The rest depends on your system. To compile:

On Windows with Visual Studio, try: msbuild libpqxx.sln On Unix-like systems, including Linux and macOS, try: make Your system may use some other command.

Then

How to compile depends on your CMake "generator":

With Unix Makefiles, run make With Ninja, run ninja With Visual Studio, run msbuild libpqxx.sln ...or whatever works in your situation.

Missing elements include:

  1. In order to run the msbuild, you have to have set up the environment, like using a command window started from the visual studio menus.
  2. The build works (after running it correctly...), but seriously one needs the "install" step to organize the files--otherwise they are scattered within the source and build locations. As to the build itself, the CMake command cmake --build build is much easier than figuring out all the MSBuild stuff (build is current build directory where the CMake was pointed).
  3. At least on Windows, the "install" step really should be done by calling CMake again, say with
cmake --install build --config Debug   --prefix "libpqxx/Debug"

Sure, you can invoke the MSBuild on the INSTALL project. But you had to configure it correctly in the very first CMake command. Then all it does is call CMake in an internal batch file.

Now my Easy-PQXX Build for Windows Visual Studio README gives meticulously tested alternatives and explanations. (Even if one does not use the batch files just the command lines as explained.)

jtv commented 4 years ago

@GordonLElliott only just catching up. Ouch — you had your own little Gitlab moment there! Hope your system has recovered. I just added a link to your project in my own README, saying that you need testers and feedback.

Next, I'll want to update my own documentation for the CMake build to use cmake commands instead of make/ninja/msbuild.

GordonLElliott commented 4 years ago

@GordonLElliott only just catching up. Ouch — you had your own little Gitlab moment there! Hope your system has recovered. ......

Yes, took a day at least. First investigate what happened. Then all night to just recover/reinstall files.

But then had to decide what to do. I didn't do what I initially said I was going to do, which was in essence to try to put some protective code into the mechanism used to call into administrator privileges. That method was to use a very cryptic language--only on Microsoft Windows--called "Visual Basic Script". But that was so cryptic that no one would be able to read it and see what was happening. So I dicided to keep that at minimum, and now what is done in that "VBS" is just recursively call the batch file itself with an argument, but running as administrator.

So what I do (as administrator in the batch file) is make sure that there is only one action path if it has any command line argument whatsoever. Then use only simple bug-free commands that only copy files, when on that path, and you can read the batch file and verify that. As I get time I may put some effort into clarifying that aspect.

OH--also thanks for the reference, hope we get someone to do some testing. I have one user but we run such nearly identical environment that it is hardly a test.

AND if you have any interest for the documentation, feel free to copy anything I said or in the project without reference to me.

GordonLElliott commented 4 years ago

A minor issue: In BUILDING-cmake.md you have the line

-DBUILD_DOC=on to include documentation in the build.

That flag (at least on Windows CMake build) is going to use Doxygen to rebuild the documentation itself.

Either on or off, it is going to include the documentation into the installed files. (There may be another flag to remove from the installed files--like @abrownsword was wishing.)

jtv commented 4 years ago

Sounds like solid thinking on the scripting. I hope it takes off.

I think it's time to close this ticket. The original issue was solved, and what we've got is like a discussion forum. And we completely drowned out @chenqingguo who asked a follow-up question which we totally missed. I'm sorry about that, @chenqingguo!

fahmidshibib commented 4 years ago

Hey, can someone tell me where you got the wsock32.lib file?

tt4g commented 4 years ago

Hey, can someone tell me where you got the wsock32.lib file?

@fahmidshibib It's installed on Windows machine.

fahmidshibib commented 4 years ago

Hey, can someone tell me where you got the wsock32.lib file?

@fahmidshibib It's installed on Windows machine.

Hi, I am trying to locate it and trying to add it to my project so that I can remove these errors (even though I am not sure if that lib file is the issue in the first place): (Click on picture for better resolution) image

I trying to run this piece of code in order to test my connection to postgresql image

Please help me resolve this. If this is due to wsock32.lib how do I add it? I looked up the internet and found out that I needed to install WindowsSDK. I did that and I found WSock32.Lib in the installation location "C:\Program Files (x86)\Windows Kits\10\Lib\10.0.19041.0\um\x64"

I added this in Properties>Linker>General>Additional Library Directories

Still ending up with these errors.

Thanks for your help!

tt4g commented 4 years ago

It looks like the linker does not link pqxx. Check the build options.