Postgres found by cmake, but not by make (Mac Ventura)

[ultrasaurus]

attempting to build on M1 Mac, I was able to resolve some, but not all errors.

To make it find postgres on my machine, as well older version of OpenSSL which it seems to require:

cmake -DPostgreSQL_LIBRARY=/usr/local/opt/postgresql@9.5/lib -DPostgreSQL_INCLUDE_DIR=/usr/local/opt/postgresql@9.5/include -DCMAKE_BUILD_TYPE=Release -DOPENSSL_ROOT_DIR=/usr/local/opt/openssl -DCMAKE_INSTALL_PREFIX=. ..

To resolve warnings (below) and also find system includes on MacOS, I modified CMakeLists.txt to add the following lines right below the version check:

cmake_policy(SET CMP0114 NEW)
cmake_policy(SET CMP0135 NEW)
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/")

cmake runs without error:

-- The C compiler identification is AppleClang 12.0.5.12050022
-- The CXX compiler identification is AppleClang 12.0.5.12050022
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- CMake build type: Release
-- Found OpenSSL: /usr/local/opt/openssl/lib/libcrypto.dylib (found version "3.0.7")  
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE  
-- Found PostgreSQL: /usr/local/opt/postgresql@9.5/lib (found version "9.5.25")  
-- Configuring done
-- Generating done
-- Build files have been written to: .../mgmigrate/build

when I run make, I see this error:

CMake Error at /usr/local/Cellar/cmake/3.25.0/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
  Could NOT find PostgreSQL (missing: PostgreSQL_LIBRARY
  PostgreSQL_INCLUDE_DIR)
Call Stack (most recent call first):
  /usr/local/Cellar/cmake/3.25.0/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:600 (_FPHSA_FAILURE_MESSAGE)
  /usr/local/Cellar/cmake/3.25.0/share/cmake/Modules/FindPostgreSQL.cmake:272 (find_package_handle_standard_args)
  cmake/config.cmake:26 (find_package)
  CMakeLists.txt:27 (include)

then the make process continues until it halts at:

[ 64%] No install step for 'postgresql-proj'
[ 65%] Completed 'postgresql-proj'
[ 65%] Built target postgresql-proj
make: *** [all] Error 2

How do I make it so it finds Postgres in make?

warning (resolved with changes to CMakeLists.txt)

CMake Warning (dev) at /usr/local/Cellar/cmake/3.25.0/share/cmake/Modules/ExternalProject.cmake:3955 (message):
  Policy CMP0114 is not set to NEW.  In order to support the Xcode "new build
  system", this project must be updated to set policy CMP0114 to NEW.

  Since CMake is generating for the Xcode "new build system",
  ExternalProject_Add will use policy CMP0114's NEW behavior anyway, but the
  generated build system may not match what the project intends.
Call Stack (most recent call first):
  CMakeLists.txt:52 (ExternalProject_Add)
This warning is for project developers.  Use -Wno-dev to suppress it.

CMake Warning (dev) at /usr/local/Cellar/cmake/3.25.0/share/cmake/Modules/ExternalProject.cmake:3075 (message):
  The DOWNLOAD_EXTRACT_TIMESTAMP option was not given and policy CMP0135 is
  not set.  The policy's OLD behavior will be used.  When using a URL
  download, the timestamps of extracted files should preferably be that of
  the time of extraction, otherwise code that depends on the extracted
  contents might not be rebuilt if the URL changes.  The OLD behavior
  preserves the timestamps from the archive instead, but this is usually not
  what you want.  Update your project to the NEW behavior or specify the
  DOWNLOAD_EXTRACT_TIMESTAMP option with a value of true to avoid this
  robustness issue.
Call Stack (most recent call first):
  /usr/local/Cellar/cmake/3.25.0/share/cmake/Modules/ExternalProject.cmake:4185 (_ep_add_download_command)
  CMakeLists.txt:52 (ExternalProject_Add)
This warning is for project developers.  Use -Wno-dev to suppress it.

CMake Warning (dev) at /usr/local/Cellar/cmake/3.25.0/share/cmake/Modules/ExternalProject.cmake:3955 (message):
  Policy CMP0114 is not set to NEW.  In order to support the Xcode "new build
  system", this project must be updated to set policy CMP0114 to NEW.

  Since CMake is generating for the Xcode "new build system",
  ExternalProject_Add will use policy CMP0114's NEW behavior anyway, but the
  generated build system may not match what the project intends.
Call Stack (most recent call first):
  CMakeLists.txt:73 (ExternalProject_Add)
This warning is for project developers.  Use -Wno-dev to suppress it.

CMake Warning (dev) at /usr/local/Cellar/cmake/3.25.0/share/cmake/Modules/ExternalProject.cmake:3955 (message):
  Policy CMP0114 is not set to NEW.  In order to support the Xcode "new build
  system", this project must be updated to set policy CMP0114 to NEW.

  Since CMake is generating for the Xcode "new build system",
  ExternalProject_Add will use policy CMP0114's NEW behavior anyway, but the
  generated build system may not match what the project intends.
Call Stack (most recent call first):
  CMakeLists.txt:94 (ExternalProject_Add)
This warning is for project developers.  Use -Wno-dev to suppress it.

CMake Warning (dev) at /usr/local/Cellar/cmake/3.25.0/share/cmake/Modules/ExternalProject.cmake:3955 (message):
  Policy CMP0114 is not set to NEW.  In order to support the Xcode "new build
  system", this project must be updated to set policy CMP0114 to NEW.

  Since CMake is generating for the Xcode "new build system",
  ExternalProject_Add will use policy CMP0114's NEW behavior anyway, but the
  generated build system may not match what the project intends.
Call Stack (most recent call first):
  CMakeLists.txt:122 (ExternalProject_Add)
This warning is for project developers.  Use -Wno-dev to suppress it.

CMake Warning (dev) at /usr/local/Cellar/cmake/3.25.0/share/cmake/Modules/ExternalProject.cmake:3955 (message):
  Policy CMP0114 is not set to NEW.  In order to support the Xcode "new build
  system", this project must be updated to set policy CMP0114 to NEW.

  Since CMake is generating for the Xcode "new build system",
  ExternalProject_Add will use policy CMP0114's NEW behavior anyway, but the
  generated build system may not match what the project intends.
Call Stack (most recent call first):
  CMakeLists.txt:145 (ExternalProject_Add)
This warning is for project developers.  Use -Wno-dev to suppress it.

CMake Warning (dev) at /usr/local/Cellar/cmake/3.25.0/share/cmake/Modules/ExternalProject.cmake:3955 (message):
  Policy CMP0114 is not set to NEW.  In order to support the Xcode "new build
  system", this project must be updated to set policy CMP0114 to NEW.

  Since CMake is generating for the Xcode "new build system",
  ExternalProject_Add will use policy CMP0114's NEW behavior anyway, but the
  generated build system may not match what the project intends.
Call Stack (most recent call first):
  CMakeLists.txt:173 (ExternalProject_Add)
This warning is for project developers.  Use -Wno-dev to suppress it.

CMake Warning (dev) at /usr/local/Cellar/cmake/3.25.0/share/cmake/Modules/ExternalProject.cmake:3955 (message):
  Policy CMP0114 is not set to NEW.  In order to support the Xcode "new build
  system", this project must be updated to set policy CMP0114 to NEW.

  Since CMake is generating for the Xcode "new build system",
  ExternalProject_Add will use policy CMP0114's NEW behavior anyway, but the
  generated build system may not match what the project intends.
Call Stack (most recent call first):
  CMakeLists.txt:202 (ExternalProject_Add)
This warning is for project developers.  Use -Wno-dev to suppress it.

[gitbuda]

Hi @ultrasaurus!

Seems like some CMake + OSX-specific problem. I'll try to take a deeper look over the next few days 😄

Is this critical for you? Since it's about migrating data, do you have another way to import the data?

[katarinasupe]

Hi @ultrasaurus! I managed to install mgmigrate on my Mac M1 after having similar issues.

Reproducing the error

Here are the steps I did to reproduce the error:

I installed XCode and Homebrew and ran: brew install git cmake make openssl postgresql

After that, I ran:

mkdir build && cd build
cmake -DCMAKE_BUILD_TYPE=Release -DOPENSSL_ROOT_DIR="$(ls -rd -- /usr/local/Cellar/openssl@1.1/* | head -n 1)" -DCMAKE_INSTALL_PREFIX=. ..

just like in the instructions.

I got the same error as you:

CMake Error at /opt/homebrew/Cellar/cmake/3.25.1/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
  Could NOT find OpenSSL, try to set the path to OpenSSL root folder in the
  system variable OPENSSL_ROOT_DIR (missing: OPENSSL_CRYPTO_LIBRARY
  OPENSSL_INCLUDE_DIR)
Call Stack (most recent call first):
  /opt/homebrew/Cellar/cmake/3.25.1/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:600 (_FPHSA_FAILURE_MESSAGE)
  /opt/homebrew/Cellar/cmake/3.25.1/share/cmake/Modules/FindOpenSSL.cmake:613 (find_package_handle_standard_args)
  CMakeLists.txt:42 (find_package)

Solution that worked for me

I then googled a bit and ran into this SO answer. Hence, I ran the following:

brew upgrade openssl

The output was: Warning: openssl 3.0.7 already installed Then I ran:

mgmigrate % brew link --force openssl

Output:

Linking /opt/homebrew/Cellar/openssl@3/3.0.7... 5517 symlinks created.

If you need to have this software first in your PATH instead consider running:
  echo 'export PATH="/opt/homebrew/opt/openssl@3/bin:$PATH"' >> ~/.zshrc

I decided to go for it and ran:

echo 'export PATH="/opt/homebrew/opt/openssl@3/bin:$PATH"' >> ~/.zshrc

and in the end, ran the last command from the SO answer to check:

pkg-config --modversion openssl

Output: 3.0.7

I removed the build folder, created a new one, and ran:

cmake -DCMAKE_BUILD_TYPE=Release -DOPENSSL_ROOT_DIR="$(ls -rd -- /usr/local/Cellar/openssl@3/* | head -n 1)" -DCMAKE_INSTALL_PREFIX=. ..

The above command is similar to the one in the README, but I changed the version of OpenSSL. After that, the build files were written to the build folder.

Let me know if that works for you.

[gitbuda]

Nice! Thanks 😄 Since it seems it's all about machine config, should be extend the README page and how?

[katarinasupe]

@gitbuda I will extend the README. I will just wait a bit for @ultrasaurus, to see if the provided solution helped.

[ultrasaurus]

yeah, I had to do some work to install an older version of SSL, will try with newer one and let you know

[ultrasaurus]

I needed to run...

export PKG_CONFIG_PATH="/usr/local/opt/openssl@3/lib/pkgconfig"

before this worked:

pkg-config --modversion openssl

For cmake to find my version of postgres, I needed to modify the command as:

cmake  -DPostgreSQL_LIBRARY=/usr/local/opt/postgresql@9.5/lib -DPostgreSQL_INCLUDE_DIR=/usr/local/opt/postgresql@9.5/include -DCMAKE_BUILD_TYPE=Release -DOPENSSL_ROOT_DIR="$(ls -rd -- /usr/local/Cellar/openssl@3/* | head -n 1)" -DCMAKE_INSTALL_PREFIX=. ..

which was successful (as before), yet make failed as before:

[ 51%] No install step for 'postgresql-proj'
[ 53%] Completed 'postgresql-proj'
[ 53%] Built target postgresql-proj
make: *** [all] Error 2

So, I was able to revert my changes to CMakeLists.txt -- progress!

However, I'm still stuck with the same problem where make is not successful.

[katarinasupe]

@ultrasaurus, I did not manage to reproduce your error. Did you install the XCode, as mentioned in README? Did you have any progress in the meantime?

[ultrasaurus]

yes, I've installed XCode and command-line tools. The issue is around configuring a specific version/installation of Postgres.

I think I don't know enough about cmake to get it set up correctly. Is there something else I need to do to carry over the configuration to the make stage? or other suggestions for debugging?

[katarinasupe]

Hi @ultrasaurus, the error message you provided gives too little info, so please:

1. use make -j1 to force single-threaded compilations, so the log messages are not mixed up
2. provide full logs
3. provide the exact versions of CMake, Make, OpenSSL, Postgres and MacOS

Also, I found a similar issue; maybe it helps.

[antejavor]

Hi @ultrasaurus, unfortunately, sad news from my side, due to the increased amount of work, we stopped actively working on mgmigrate, this resulted in mgmigrate being tricky to use. This means this issue won't be resolved in the near term future, once we get back to mgmigrate this will be resolved.

We just added a short [notice] (https://github.com/memgraph/mgmigrate#notice) and archived the repo.

If you need help with migration to Memgraph, consider joining our Discord community, I would love to help you out, and we can hop on a video call to understand the challenges you are facing with the transition to Memgraph.

Feel free to ping me on Discord; my username is also antejavor on Discord.