search

neuronsimulator / nrn

NEURON Simulator
http://nrn.readthedocs.io
Other
406 stars 118 forks source link

Unable to rebuild from scratch with cmake on macOS 13.3 #2326

Closed cstricker closed 1 year ago

cstricker commented 1 year ago

Context

With the release of Xcode 14.3 and macOS 13.3, cmake is now building for sdk 13.3 and not 13.1 anymore. To avoid a few errors, the easiest way is to dump the git repo and rebuild from scratch after re-cloning the source.

Overview of the issue

However, if I do this, I get two errors that disable a build. The issue is that cmake expects CMakeFiles in ../nrn/build/external/catch2 and ../nrn/build/external/iv . Indeed when the errors occur, those two external "packages" have not been downloaded and configured yet.

The output is:

+/Users/stricker/.local/src/neuron/nrnupd:58> git clone https://github.com/neuronsimulator/nrn Cloning into 'nrn'... remote: Enumerating objects: 86377, done. remote: Counting objects: 100% (3277/3277), done. remote: Compressing objects: 100% (1811/1811), done. remote: Total 86377 (delta 1945), reused 2233 (delta 1433), pack-reused 83100 Receiving objects: 100% (86377/86377), 158.25 MiB | 5.65 MiB/s, done. Resolving deltas: 100% (61808/61808), done. +/Users/stricker/.local/src/neuron/nrnupd:59> COMPNRN=1 +/Users/stricker/.local/src/neuron/nrnupd:61> [[ 1 -eq 1 ]] +/Users/stricker/.local/src/neuron/nrnupd:62> cd /Users/stricker/.local/src/neuron/nrn +/Users/stricker/.local/src/neuron/nrnupd:63> mkdir -p build +/Users/stricker/.local/src/neuron/nrnupd:64> pushd build +/Users/stricker/.local/src/neuron/nrnupd:65> cmake .. '-DNRN_ENABLE_INTERVIEWS=ON' '-DNRN_ENABLE_PYTHON=ON' '-DNRN_ENABLE_MPI=ON' '-DNRN_ENABLE_RX3D=ON' '-DNRN_ENABLE_CORENEURON=OFF' '-DNRN_ENABLE_TESTS=ON' '-DNRN_ENABLE_BINARY_SPECIAL=ON' '-DPYTHON_EXECUTABLE=/usr/local/opt/python/Frameworks/Python.framework/Versions/3.11/bin/python3' '-DCMAKE_INSTALL_PREFIX=/Applications/NEURON' '-DCMAKE_C_COMPILER=clang' '-DCMAKE_CXX_COMPILER=clang++' -- The C compiler identification is AppleClang 14.0.3.14030022 -- The CXX compiler identification is AppleClang 14.0.3.14030022 -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Check for working C compiler: /usr/bin/clang - skipped -- Detecting C compile features -- Detecting C compile features - done -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Check for working CXX compiler: /usr/bin/clang++ - skipped -- Detecting CXX compile features -- Detecting CXX compile features - done -- Setting build type to 'RelWithDebInfo' as none was specified. -- Sub-module : missing external/coding-conventions : running git submodule update --init Submodule 'external/coding-conventions' (https://github.com/BlueBrain/hpc-coding-conventions.git) registered for path 'external/coding-conventions' Cloning into '/Users/stricker/.local/src/neuron/nrn/external/coding-conventions'... Submodule path 'external/coding-conventions': checked out 'f8f8d69a66c23978d1c9c5dce62de79466f26e5d' -- Fetching git submodule external/Random123: running git submodule update --init;--recursive;--depth;1 -- external/Random123 Submodule 'external/Random123' (https://github.com/BlueBrain/Random123.git) registered for path 'external/Random123' Cloning into '/Users/stricker/.local/src/neuron/nrn/external/Random123'... From https://github.com/BlueBrain/Random123

-- Found PythonInterp: /usr/local/opt/python/Frameworks/Python.framework/Versions/3.11/bin/python3 (found suitable version "3.11.3", minimum required is "3.8") -- Found PythonInterp: /usr/local/opt/python/Frameworks/Python.framework/Versions/3.11/bin/python3 (found suitable version "3.11.3", minimum required is "3") -- Found PythonLibs: /usr/local/opt/python@3.11/Frameworks/Python.framework/Versions/3.11/lib/libpython3.11.dylib -- Performing Test CMAKE_HAVE_LIBC_PTHREAD -- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success -- Found Threads: TRUE Extracting link flags from target 'Threads::Threads', beware that this can be fragile. Got: Generating link flags from path /usr/local/opt/python@3.11/Frameworks/Python.framework/Versions/3.11/lib/libpython3.11.dylib Got: /usr/local/opt/python@3.11/Frameworks/Python.framework/Versions/3.11/lib/libpython3.11.dylib -Wl,-rpath,/usr/local/opt/python@3.11/Frameworks/Python.framework/Versions/3.11/lib Generating link flags from path /usr/local/Cellar/open-mpi/4.1.5/lib/libmpi.dylib Got: /usr/local/Cellar/open-mpi/4.1.5/lib/libmpi.dylib -Wl,-rpath,/usr/local/Cellar/open-mpi/4.1.5/lib Generating link flags from path /usr/local/lib/libX11.dylib Got: /usr/local/lib/libX11.dylib -Wl,-rpath,/usr/local/lib Generating link flags from path /usr/local/lib/libXext.dylib Got: /usr/local/lib/libXext.dylib -Wl,-rpath,/usr/local/lib -- Sub-project : using catch2 from from /Users/stricker/.local/src/neuron/nrn/external/catch2 CMake Error at cmake/ExternalProjectHelper.cmake:58 (add_subdirectory): The source directory

/Users/stricker/.local/src/neuron/nrn/external/catch2

does not contain a CMakeLists.txt file. Call Stack (most recent call first): CMakeLists.txt:819 (nrn_add_external_project)

CMake Error at CMakeLists.txt:822 (include): include could not find requested file:

Catch

-- Found pytest: /usr/local/opt/python/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pytest (found version "7.3.1") -- Found pytest_cov: /usr/local/opt/python/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pytest_cov (found version "4.0.0") -- Fetching git submodule test/rxd/testdata: running git submodule update --init;--recursive;--depth;1 -- test/rxd/testdata Submodule 'test/rxd/testdata' (https://github.com/neuronsimulator/rxdtestdata) registered for path 'test/rxd/testdata' Cloning into '/Users/stricker/.local/src/neuron/nrn/test/rxd/testdata'... From https://github.com/neuronsimulator/rxdtestdata

Expected result/behavior

Not relevant

NEURON setup

Minimal working example - MWE

CMake build commands:


git clone git@github.com:neuronsimulator/nrn.git
mkdir build && cd build
cmake .. '-DNRN_ENABLE_INTERVIEWS=ON' '-DNRN_ENABLE_PYTHON=ON' '-DNRN_ENABLE_MPI=ON' '-DNRN_ENABLE_RX3D=ON' '-DNRN_ENABLE_CORENEURON=OFF' '-DNRN_ENABLE_TESTS=ON' '-DNRN_ENABLE_BINARY_SPECIAL=ON' '-DPYTHON_EXECUTABLE=/usr/local/opt/python/Frameworks/Python.framework/Versions/3.11/bin/python3' '-DCMAKE_INSTALL_PREFIX=/Applications/NEURON' '-DCMAKE_C_COMPILER=clang' '-DCMAKE_CXX_COMPILER=clang++'```

## Logs

See above

 Otherwise provide the error printed to the terminal or a screenshot of the issue]
nrnhines commented 1 year ago

I'm experiencing the same issue on my M1 after upgrading to macOS Ventura 13.3.1 and installing the latest Xcode 14.3 The last line of

cmake .. -G Ninja -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ -DCMAKE_INSTALL_PREFIX=install -DPYTHON_EXECUTABLE=`which python3.11` -DNRN_ENABLE_RX3D=OFF -DNRN_ENABLE_TESTS=ON

is

-- Configuring incomplete, errors occurred!

The errors are

-- Sub-project : using catch2 from from /Users/hines/neuron/nrn/external/catch2
CMake Error at cmake/ExternalProjectHelper.cmake:58 (add_subdirectory):
  The source directory

    /Users/hines/neuron/nrn/external/catch2

  does not contain a CMakeLists.txt file.
Call Stack (most recent call first):
  CMakeLists.txt:819 (nrn_add_external_project)

CMake Error at CMakeLists.txt:822 (include):
  include could not find requested file:

    Catch

and

-- Sub-project : using iv from from /Users/hines/neuron/nrn/external/iv
CMake Error at cmake/ExternalProjectHelper.cmake:58 (add_subdirectory):
  The source directory

    /Users/hines/neuron/nrn/external/iv

  does not contain a CMakeLists.txt file.
Call Stack (most recent call first):
  CMakeLists.txt:403 (nrn_add_external_project)

a work around for the issue is

cd external
git clone https://github.com/neuronsimulator/iv.git
git clone https://github.com/catchorg/Catch2.git
cd catch2
git checkout  c4e3767e
nrnhines commented 1 year ago

But I am seeing some extra test failures.

      2 - unit_tests::benchmarks (Failed)
     13 - pynrn::basic_tests (Failed)
     14 - coverage_tests::cover_tests (Failed)
     62 - hoctests::test_neurondemo_py (Failed)

62, I believe is just because I did not configure with -DNRN_ENABLE_CORENEURON. And I speculate that the benchmark failure is specific to my M1.

To build with -DNRN_ENABLE_CORENEURON=ON required an explicit

cd external
git clone https://github.com/BlueBrain/mod2c

but now I have 34 failing tests. That is likely not relevant to this issue.

cstricker commented 1 year ago

G’day Michael

thanks for looking into this. Great to hear from you! I hope you are doing very well!

I have just built using the work-around - and that fixes it at my end… but I don’t have an M1 or M2 machine yet… :-( Perhaps it is a blessing at this stage? I have no issues with the tests folder, though.

Take care - and thanks for your prompt reply. Cheers

Christian.

On 17 Apr 2023, at 17:36, nrnhines @.***> wrote:

I'm experiencing the same issue on my M1 after upgrading to macOS Ventura 13.3.1 and installing the latest Xcode 14.3 The last line of

cmake .. -G Ninja -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ -DCMAKE_INSTALL_PREFIX=install -DPYTHON_EXECUTABLE=which python3.11 -DNRN_ENABLE_RX3D=OFF -DNRN_ENABLE_TESTS=ON

is

-- Configuring incomplete, errors occurred!

The errors are

-- Sub-project : using catch2 from from /Users/hines/neuron/nrn/external/catch2 CMake Error at cmake/ExternalProjectHelper.cmake:58 (add_subdirectory): The source directory

/Users/hines/neuron/nrn/external/catch2

does not contain a CMakeLists.txt file. Call Stack (most recent call first): CMakeLists.txt:819 (nrn_add_external_project)

CMake Error at CMakeLists.txt:822 (include): include could not find requested file:

Catch

and

-- Sub-project : using iv from from /Users/hines/neuron/nrn/external/iv CMake Error at cmake/ExternalProjectHelper.cmake:58 (add_subdirectory): The source directory

/Users/hines/neuron/nrn/external/iv

does not contain a CMakeLists.txt file. Call Stack (most recent call first): CMakeLists.txt:403 (nrn_add_external_project)

a work around for the issue is

cd external git clone https://github.com/neuronsimulator/iv.git git clone https://github.com/catchorg/Catch2.git cd catch2

— Reply to this email directly, view it on GitHubhttps://github.com/neuronsimulator/nrn/issues/2326#issuecomment-1510852232, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AD5QLACSH2JJ4AOGLPQPKJ3XBTXJJANCNFSM6AAAAAAW77VIXQ. You are receiving this because you authored the thread.Message ID: @.***>

cstricker commented 1 year ago

G’day Michael

I have a hunch that the following observation may be causing the issue. Having looked around, it turns out that there are two versions of iv and catch2. One in the following location (that you, Michael, indicated for manual download...)

…/nrn/external

and the second one in

…/nrn/build/external

Turns out that the first one is used by cmake and compiled first - and has the CMakeLists.txt file in it - that is the reason why it is possible to do the work-around. However, this one is not populated by the cmake build command that does the git download. Those commands download into …/nrn/build/external - but both do not get the CMakeLists.txt file - i..e are not on the “standard configuration path". That is where the error occurs if iv and catch2 are not manually downloaded into …/nrn/external.

After make -j if you then do a make install, cmake somehow detects an issue and reports

-- GLOB mismatch!

and goes back and does a cmake re-configuration and this time, it creates the CMakeLists.txt file in …/nrn/build/external - and compiles and links the contents in iv and catch2. I noticed this as the times when the respective files are compiled differ on my machine by about 5 minutes.

I hope this helps and cheers

Christian.

On 17 Apr 2023, at 17:36, nrnhines @.***> wrote:

I'm experiencing the same issue on my M1 after upgrading to macOS Ventura 13.3.1 and installing the latest Xcode 14.3 The last line of

cmake .. -G Ninja -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ -DCMAKE_INSTALL_PREFIX=install -DPYTHON_EXECUTABLE=which python3.11 -DNRN_ENABLE_RX3D=OFF -DNRN_ENABLE_TESTS=ON

is

-- Configuring incomplete, errors occurred!

The errors are

-- Sub-project : using catch2 from from /Users/hines/neuron/nrn/external/catch2 CMake Error at cmake/ExternalProjectHelper.cmake:58 (add_subdirectory): The source directory

/Users/hines/neuron/nrn/external/catch2

does not contain a CMakeLists.txt file. Call Stack (most recent call first): CMakeLists.txt:819 (nrn_add_external_project)

CMake Error at CMakeLists.txt:822 (include): include could not find requested file:

Catch

and

-- Sub-project : using iv from from /Users/hines/neuron/nrn/external/iv CMake Error at cmake/ExternalProjectHelper.cmake:58 (add_subdirectory): The source directory

/Users/hines/neuron/nrn/external/iv

does not contain a CMakeLists.txt file. Call Stack (most recent call first): CMakeLists.txt:403 (nrn_add_external_project)

a work around for the issue is

cd external git clone https://github.com/neuronsimulator/iv.git git clone https://github.com/catchorg/Catch2.git cd catch2

— Reply to this email directly, view it on GitHubhttps://github.com/neuronsimulator/nrn/issues/2326#issuecomment-1510852232, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AD5QLACSH2JJ4AOGLPQPKJ3XBTXJJANCNFSM6AAAAAAW77VIXQ. You are receiving this because you authored the thread.Message ID: @.***>