ModuleNotFoundError: No module named 'yaml'

[pramodk]

While building CoreNEURON in vanilla docker Ubuntu 20.04 image with typical build dependencies, I see:

-- Found PythonInterp: /usr/bin/python3 (found suitable version "3.8.5", minimum required is "3.5")
-- Found ClangFormat: /usr/bin/clang-format-11 (found version "11.0.0")
Traceback (most recent call last):
  File "/CoreNeuron/CMake/hpc-coding-conventions/cpp/cmake/cpplib.py", line 448, in <module>
    sys.exit(0 if main() else 1)
  File "/CoreNeuron/CMake/hpc-coding-conventions/cpp/cmake/cpplib.py", line 442, in main
    return result.func(*result.files)
  File "/CoreNeuron/CMake/hpc-coding-conventions/cpp/cmake/cpplib.py", line 397, in do_merge_yaml
    import yaml
ModuleNotFoundError: No module named 'yaml'
-- Setting build type to 'RelWithDebInfo' as none was specified.
-- Could NOT find Random123 (missing: Random123_PROJ)
-- Sub-module Random123 missing: running git submodule update --init --recursive
Submodule 'external/Random123' (https://github.com/BlueBrain/Random123.git) registered for path 'external/Random123'
Cloning into '/CoreNeuron/external/Random123'...
Submodule path 'external/Random123': checked out 'b5cb15614e8f3eb0ff316bdc50c300ddd32fa3c1'
-- Could NOT find CLI11 (missing: CLI11_PROJ)
-- Sub-module CLI11 missing: running git submodule update --init
Submodule 'external/CLI11' (https://github.com/CLIUtils/CLI11.git) registered for path 'external/CLI11'
Cloning into '/CoreNeuron/external/CLI11'...
Submodule path 'external/CLI11': checked out 'dd0d8e4fe729e5b1110232c7a5c9566dad884686'
-- Found Git: /usr/bin/git (found version "2.25.1")
-- Could NOT find Doxygen (missing: DOXYGEN_EXECUTABLE)
-- Doxygen not found, building docs has been disabled
-- Found PythonInterp: /usr/bin/python3 (found version "3.8.5")
-- Found Perl: /usr/bin/perl (found version "5.30.0")
-- Could NOT find MOD2C (missing: MOD2C_PROJ)
-- Sub-module mod2c missing : running git submodule update --init --recursive
Submodule 'external/mod2c' (https://github.com/BlueBrain/mod2c) registered for path 'external/mod2c'
Cloning into '/CoreNeuron/external/mod2c'...
Submodule path 'external/mod2c': checked out 'e02365e61dece84efed8a63d4b550c3ca690d65c'
-- mod2c is used as APPLICATION_NAME
-- Found BISON: /usr/bin/bison (found version "3.5.1")
-- Found FLEX: /usr/bin/flex (found version "2.6.4")
-- Boost not found, unit tests disabled
--
-- Configured CoreNEURON 0.21.0
--
-- Some things you can do now:
-- --------------------+--------------------------------------------------------
-- Command             |   Description
-- --------------------+--------------------------------------------------------
-- make install        | Will install CoreNEURON to: /usr/local
-- make docs           | Build full docs. Calls targets: doxygen, sphinx
-- --------------------+--------------------------------------------------------
--  Build option       | Status
-- --------------------+--------------------------------------------------------
-- C COMPILER          | /usr/bin/cc
-- CXX COMPILER        | /usr/bin/c++
-- COMPILE FLAGS       |  -Wno-unknown-pragmas -g  -O2 -std=c++14
-- Build Type          | SHARED
-- MPI                 | OFF
-- OpenMP              | OFF
-- Use legacy units    | OFF
-- NMODL               | OFF
-- MOD2CPP PATH        | /CoreNeuron/build/bin/mod2c_core
-- GPU Support         | OFF
-- Auto Timeout        | ON
-- Wrap exp()          | OFF
-- SplayTree Queue     | ON
-- NetReceive Buffer   | ON
-- Caliper             | OFF
-- Likwid              | OFF
-- Unit Tests          | ON
-- Reporting           | OFF
-- --------------+--------------------------------------------------------------
--  See documentation : https://github.com/BlueBrain/CoreNeuron/
-- --------------+--------------------------------------------------------------
--
-- Configuring done
-- Generating done
-- Build files have been written to: /CoreNeuron/build

Is yaml necessary dependency?

[alkino]

yaml module is used to merge yaml file. We merge yaml file when people put some .clang-format.extend in the root of the git repository to update the default .clang-format file (or cmake or tidy).

@tristan0x I cannot find a requirements.txt in this repository, how is it handled?

[matz-e]

Could this be optional?

[tristan0x]

Could this be optional?

A quick move is to restrict the import yaml only where it is used. For instance, in this particular case, it would only fail if one provides a .clang-format.extend at the root a project. But yeah, at some point we want to read and write yaml files.

[alkino]

Could this be optional?

A quick move is to restrict the import yaml only where it is used. For instance, in this particular case, it would only fail if one provides a .clang-format.extend at the root a project. But yeah, at some point we want to read and write yaml files.

It already seems needed only when use (the import is at beginning of the function using the module.

[pramodk]

And, it would be nice if CMake try to give clear error message instead of silent error like above. (e.g. FindPythonModule can be used to check if pyyaml exists)

[alkino]

And so make it always needed globally?

[pramodk]

I don't know how the overall workflow works here but what I was thinking of is : 1) you see <....>.extend file exist at the top level 2) pyyaml module doesn't exist 3) then raise an error

[tristan0x]

I agree with @pramodk I would:

1. raise a CMake warning if the PyYaml module is not found whether it is required or not.
2. raise a CMake error wherever the PyYaml module is required.