Open benlee0423 opened 5 months ago
Obviously, this is due to the following path in module. Not sure how to fix it.
Singularity> ls /usr/lib64/python3.9/site-packages/openmpi
ls: cannot access '/usr/lib64/python3.9/site-packages/openmpi': No such file or directory
-- NGen version: 0.1.0
-- Build configuration summary:
-- Generator: Unix Makefiles
-- Build type:
-- System: Linux
-- C Compiler: /usr/bin/cc
-- C Flags:
-- CXX Compiler: /usr/bin/c++
-- CXX Flags:
-- Flags:
-- NGEN_WITH_MPI: OFF
-- NGEN_WITH_NETCDF: ON
-- NGEN_WITH_SQLITE: ON
-- NGEN_WITH_UDUNITS: ON
-- NGEN_WITH_BMI_FORTRAN: ON
-- NGEN_WITH_BMI_C: ON
-- NGEN_WITH_PYTHON: ON
-- NGEN_WITH_ROUTING: ON
-- NGEN_WITH_TESTS: ON
-- NGEN_WITH_COVERAGE: OFF
-- NGEN_QUIET: ON
-- Extern Models:
-- NGEN_WITH_EXTERN_ALL: OFF
-- NGEN_WITH_EXTERN_SLOTH: ON
-- NGEN_WITH_EXTERN_TOPMODEL: ON
-- NGEN_WITH_EXTERN_CFE: ON
-- NGEN_WITH_EXTERN_PET: ON
-- NGEN_WITH_EXTERN_NOAH_OWP_MODULAR: ON
-- Environment summary:
-- Boost:
-- Version: 1.79.0
-- Include: /usr/include
-- NetCDF:
-- Version: 4.8.1
-- Library: /usr/lib64/libnetcdf.so
-- Library (CXX): /usr/local/lib64/libnetcdf-cxx4.so
-- Include: /usr/include
-- Include (CXX): /usr/local/include
-- Parallel: FALSE
-- SQLite:
-- Version: 3.34.1
-- Library: /usr/lib64/libsqlite3.so
-- Include: /usr/include
-- UDUNITS2:
-- Library: /usr/lib64/libudunits2.so
-- Include: /usr/include/udunits2
-- Fortran:
-- BMI_FORTRAN_ISO_C_LIB_PATH:
-- BMI_FORTRAN_ISO_C_LIB_NAME: OFF
-- BMI_FORTRAN_ISO_C_LIB_DIR: OFF
-- Python:
-- Version: 3.9.18
-- Virtual Env:
-- Configuring done
This looks like the similar issue raised by Trupesh. https://github.com/NOAA-OWP/ngen/issues/655
Can you run
ldd /usr/bin/python3.9
In the container runtime?
Singularity> ldd /usr/bin/python3.9
linux-vdso.so.1 (0x00007ffc11394000)
libpython3.9.so.1.0 => /lib64/libpython3.9.so.1.0 (0x00007f71c2c62000)
libc.so.6 => /lib64/libc.so.6 (0x00007f71c2a59000)
libm.so.6 => /lib64/libm.so.6 (0x00007f71c297e000)
/lib64/ld-linux-x86-64.so.2 (0x00007f71c2fd4000)
So this isn't the same issue referenced above (noaa-owp/ngen#655) which is caused by the python interpreter being statically linked.
Best guess is a path problem. This looks suspicious:
-- NumPy Include: /usr/local/lib64/python3.9/site-packages/numpy/core/include
It looks like numpy is installed/found in
/usr/local/
Whereas the python path is
-- Site Library: /usr/lib/python3.9/site-packages
Can you simply open a python interpreter in the container and import numpy?
Able to import numpy in python.
Singularity> python
Python 3.9.18 (main, Jan 4 2024, 00:00:00)
[GCC 11.4.1 20230605 (Red Hat 11.4.1-2)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy
>>> array = numpy.array([1,2,3,4,5])
>>> print(array)
[1 2 3 4 5]
Also numpy is found in the location.
Singularity> ls /usr/local/lib64/python3.9/site-packages/numpy/core/include
numpy
python location
Singularity> whereis python
python: /usr/bin/python
no numpy in /usr/lib/python3.9/site-packages
Singularity> ls /usr/lib/python3.9/site-packages
__pycache__ distutils-precedence.pth mockbuild pip-21.2.3.dist-info python_dateutil-2.8.1-py3.9.egg-info six.py
_distutils_hack dnf packaging pkg_resources setuptools
asciidocapi.py dnf-plugins packaging-20.9.dist-info pyparsing-2.4.7.dist-info setuptools-53.0.0.dist-info
dateutil dnfpluginscore pip pyparsing.py six-1.15.0.dist-info
Singularity> ls -l /usr/bin/python
lrwxrwxrwx 1 root root 16 Mar 22 02:17 /usr/bin/python -> /usr/bin/python3
Have you tried using a virtual environment for building and running ngen with?
No virtual environment is used.
What ngen commit are you building? A pybind update was merged a couple days ago.
noaa-owp/ngen#755
I just built the image with ngen master branch. And, getting the same error.
Getting the same error in docker build as well.
#17 48.05 terminate called after throwing an instance of 'pybind11::error_already_set'
#17 48.05 what(): ModuleNotFoundError: No module named 'numpy'
#17 48.18 [ 64%] Built target test_geojson
#17 48.18 CMake Error at /usr/local/lib64/python3.9/site-packages/cmake/data/share/cmake-3.28/Modules/GoogleTestAddTests.cmake:112 (message):
#17 48.18 Error running test executable.
#17 48.18
#17 48.18 Path: '/ngen/ngen/cmake_build_serial/test/test_routing_pybind'
#17 48.18 Result: Subprocess aborted
#17 48.18 Output:
Build and run are successful with commit id f91e2ea of ngen repo.
Command to run the image
Command to run inside running image
Error message
Some path variables inside running image
Linux: