search

GUDHI / gudhi-devel

The GUDHI library is a generic open source C++ library, with a Python interface, for Topological Data Analysis (TDA) and Higher Dimensional Geometry Understanding.
https://gudhi.inria.fr/
MIT License
245 stars 65 forks source link

Failing test #818

Closed mboisson closed 1 year ago

mboisson commented 1 year ago

Currently building version 3.7.1, and one of the test fails with: 

 5/89 Test  #5: test_subsampling_py_test ....................................................................***Exception: Numerical  0.38 sec
=========================================================================================================================================================================== test session starts ============================================================================================================================================================================
platform linux -- Python 3.10.2, pytest-7.0.1, pluggy-1.0.0
rootdir: /tmp/mboisson/avx2/GUDHI/3.7.1/GCC-9.3.0/gudhi.3.7.1/python
collected 5 items

test/test_subsampling.py ...Fatal Python error: Floating point exception

Current thread 0x00002ba7fce83740 (most recent call first):
  File "/tmp/mboisson/avx2/GUDHI/3.7.1/GCC-9.3.0/gudhi.3.7.1/python/test/test_subsampling.py", line 80 in test_simple_pick_n_random_points
  File "/cvmfs/soft.computecanada.ca/easybuild/software/2020/avx2/Core/pytest/7.0.1/lib/python3.10/site-packages/_pytest/python.py", line 192 in pytest_pyfunc_call
  File "/cvmfs/soft.computecanada.ca/easybuild/software/2020/avx2/Core/pytest/7.0.1/lib/python3.10/site-packages/pluggy/_callers.py", line 39 in _multicall
  File "/cvmfs/soft.computecanada.ca/easybuild/software/2020/avx2/Core/pytest/7.0.1/lib/python3.10/site-packages/pluggy/_manager.py", line 80 in _hookexec
  File "/cvmfs/soft.computecanada.ca/easybuild/software/2020/avx2/Core/pytest/7.0.1/lib/python3.10/site-packages/pluggy/_hooks.py", line 265 in __call__
  File "/cvmfs/soft.computecanada.ca/easybuild/software/2020/avx2/Core/pytest/7.0.1/lib/python3.10/site-packages/_pytest/python.py", line 1718 in runtest
  File "/cvmfs/soft.computecanada.ca/easybuild/software/2020/avx2/Core/pytest/7.0.1/lib/python3.10/site-packages/_pytest/runner.py", line 168 in pytest_runtest_call
  File "/cvmfs/soft.computecanada.ca/easybuild/software/2020/avx2/Core/pytest/7.0.1/lib/python3.10/site-packages/pluggy/_callers.py", line 39 in _multicall
  File "/cvmfs/soft.computecanada.ca/easybuild/software/2020/avx2/Core/pytest/7.0.1/lib/python3.10/site-packages/pluggy/_manager.py", line 80 in _hookexec
  File "/cvmfs/soft.computecanada.ca/easybuild/software/2020/avx2/Core/pytest/7.0.1/lib/python3.10/site-packages/pluggy/_hooks.py", line 265 in __call__
  File "/cvmfs/soft.computecanada.ca/easybuild/software/2020/avx2/Core/pytest/7.0.1/lib/python3.10/site-packages/_pytest/runner.py", line 261 in <lambda>
  File "/cvmfs/soft.computecanada.ca/easybuild/software/2020/avx2/Core/pytest/7.0.1/lib/python3.10/site-packages/_pytest/runner.py", line 340 in from_call
  File "/cvmfs/soft.computecanada.ca/easybuild/software/2020/avx2/Core/pytest/7.0.1/lib/python3.10/site-packages/_pytest/runner.py", line 260 in call_runtest_hook
  File "/cvmfs/soft.computecanada.ca/easybuild/software/2020/avx2/Core/pytest/7.0.1/lib/python3.10/site-packages/_pytest/runner.py", line 221 in call_and_report
  File "/cvmfs/soft.computecanada.ca/easybuild/software/2020/avx2/Core/pytest/7.0.1/lib/python3.10/site-packages/_pytest/runner.py", line 132 in runtestprotocol
  File "/cvmfs/soft.computecanada.ca/easybuild/software/2020/avx2/Core/pytest/7.0.1/lib/python3.10/site-packages/_pytest/runner.py", line 113 in pytest_runtest_protocol
  File "/cvmfs/soft.computecanada.ca/easybuild/software/2020/avx2/Core/pytest/7.0.1/lib/python3.10/site-packages/pluggy/_callers.py", line 39 in _multicall
  File "/cvmfs/soft.computecanada.ca/easybuild/software/2020/avx2/Core/pytest/7.0.1/lib/python3.10/site-packages/pluggy/_manager.py", line 80 in _hookexec
  File "/cvmfs/soft.computecanada.ca/easybuild/software/2020/avx2/Core/pytest/7.0.1/lib/python3.10/site-packages/pluggy/_hooks.py", line 265 in __call__
  File "/cvmfs/soft.computecanada.ca/easybuild/software/2020/avx2/Core/pytest/7.0.1/lib/python3.10/site-packages/_pytest/main.py", line 347 in pytest_runtestloop
  File "/cvmfs/soft.computecanada.ca/easybuild/software/2020/avx2/Core/pytest/7.0.1/lib/python3.10/site-packages/pluggy/_callers.py", line 39 in _multicall
  File "/cvmfs/soft.computecanada.ca/easybuild/software/2020/avx2/Core/pytest/7.0.1/lib/python3.10/site-packages/pluggy/_manager.py", line 80 in _hookexec
  File "/cvmfs/soft.computecanada.ca/easybuild/software/2020/avx2/Core/pytest/7.0.1/lib/python3.10/site-packages/pluggy/_hooks.py", line 265 in __call__
  File "/cvmfs/soft.computecanada.ca/easybuild/software/2020/avx2/Core/pytest/7.0.1/lib/python3.10/site-packages/_pytest/main.py", line 322 in _main
  File "/cvmfs/soft.computecanada.ca/easybuild/software/2020/avx2/Core/pytest/7.0.1/lib/python3.10/site-packages/_pytest/main.py", line 268 in wrap_session
  File "/cvmfs/soft.computecanada.ca/easybuild/software/2020/avx2/Core/pytest/7.0.1/lib/python3.10/site-packages/_pytest/main.py", line 315 in pytest_cmdline_main
  File "/cvmfs/soft.computecanada.ca/easybuild/software/2020/avx2/Core/pytest/7.0.1/lib/python3.10/site-packages/pluggy/_callers.py", line 39 in _multicall
  File "/cvmfs/soft.computecanada.ca/easybuild/software/2020/avx2/Core/pytest/7.0.1/lib/python3.10/site-packages/pluggy/_manager.py", line 80 in _hookexec
  File "/cvmfs/soft.computecanada.ca/easybuild/software/2020/avx2/Core/pytest/7.0.1/lib/python3.10/site-packages/pluggy/_hooks.py", line 265 in __call__
  File "/cvmfs/soft.computecanada.ca/easybuild/software/2020/avx2/Core/pytest/7.0.1/lib/python3.10/site-packages/_pytest/config/__init__.py", line 165 in main
  File "/cvmfs/soft.computecanada.ca/easybuild/software/2020/avx2/Core/pytest/7.0.1/lib/python3.10/site-packages/_pytest/config/__init__.py", line 188 in console_main
  File "/cvmfs/soft.computecanada.ca/easybuild/software/2020/avx2/Core/pytest/7.0.1/lib/python3.10/site-packages/pytest/__main__.py", line 5 in <module>
  File "/cvmfs/soft.computecanada.ca/easybuild/software/2020/avx2/Core/python/3.10.2/lib/python3.10/runpy.py", line 86 in _run_code
  File "/cvmfs/soft.computecanada.ca/easybuild/software/2020/avx2/Core/python/3.10.2/lib/python3.10/runpy.py", line 196 in _run_module_as_main

Extension modules: numpy.core._multiarray_umath, numpy.core._multiarray_tests, numpy.linalg._umath_linalg, numpy.fft._pocketfft_internal, numpy.random._common, numpy.random.bit_generator, numpy.random._bounded_integers, numpy.random._mt19937, numpy.random.mtrand, numpy.random._philox, numpy.random._pcg64, numpy.random._sfc64, numpy.random._generator, gudhi.off_utils, gudhi.simplex_tree, gudhi.rips_complex, gudhi.cubical_complex, gudhi.periodic_cubical_complex, gudhi.reader_utils, gudhi.witness_complex, gudhi.strong_witness_complex, gudhi.nerve_gic, gudhi.subsampling, gudhi.tangential_complex, gudhi.euclidean_witness_complex, gudhi.euclidean_strong_witness_complex (total: 26)

Any pointer on fixing this ?

mglisse commented 1 year ago

Hello, I don't remember ever seeing this error. Does it reproduce every time? Could you tell us more about how you built gudhi (options to cmake, output of cmake)? Is there anything unusual in your platform (virtual machine, compilation and execution on different machines, etc)? Line 80 is assert gudhi.pick_n_random_points(points=[], nb_points=0) == [], does running gudhi.pick_n_random_points(points=[], nb_points=0) after just importing gudhi also crash?

mboisson commented 1 year ago

Yes, it happens every time (until I disable the tests).

This is an install for cluster computing environments. The build is in a virtual machine. The tests are run on the same VM.

The whole build is using EasyBuild, with modules (external libraries), and is therefore rather complicated, but the gist of the commands being run is the following: 

grep "running cmd" /home/mboisson/.local/easybuild/software/2020/avx2/Compiler/gcc9/gudhi/3.7.1/easybuild/easybuild-GUDHI-3.7.1-20230216.163908.log
== 2023-02-16 16:24:22,821 run.py:236 INFO running cmd: tar xzf /home/mboisson/.local/easybuild/sources/g/GUDHI/gudhi.3.7.1.tar.gz
== 2023-02-16 16:24:39,346 run.py:236 INFO running cmd:  cmake  -DCMAKE_INSTALL_PREFIX=/home/mboisson/.local/easybuild/software/2020/avx2/Compiler/gcc9/gudhi/3.7.1 -DCMAKE_BUILD_TYPE=Release -DCMAKE_VERBOSE_MAKEFILE=ON -DCMAKE_FIND_USE_PACKAGE_REGISTRY=OFF -DBoost_NO_BOOST_CMAKE=ON -DBOOST_ROOT=/cvmfs/soft.computecanada.ca/easybuild/software/2020/avx2/Compiler/gcc9/boost/1.72.0 -DBoost_NO_SYSTEM_PATHS=ON  -DCMAKE_VERBOSE_MAKEFILE:BOOL=ON  -DENABLE_XHOST=OFF  -DCMAKE_SKIP_INSTALL_RPATH=ON -DPYTHON_EXECUTABLE=$EBROOTPYTHON/bin/python3 .
== 2023-02-16 16:24:42,853 run.py:236 INFO running cmd:  make  -j 8
== 2023-02-16 16:24:54,769 run.py:236 INFO running cmd:  make install  && mkdir -p /home/mboisson/.local/easybuild/software/2020/avx2/Compiler/gcc9/gudhi/3.7.1/lib/python3.10/site-packages && cd python && pip install --no-deps --no-build-isolation --prefix=/home/mboisson/.local/easybuild/software/2020/avx2/Compiler/gcc9/gudhi/3.7.1 .
== 2023-02-16 16:25:09,677 run.py:236 INFO running cmd: tar xzf /home/mboisson/.local/easybuild/sources/g/GUDHI/gudhi.3.7.1.tar.gz
== 2023-02-16 16:25:23,726 run.py:236 INFO running cmd:  cmake   -DCMAKE_INSTALL_PREFIX=/home/mboisson/.local/easybuild/software/2020/avx2/Compiler/gcc9/gudhi/3.7.1 -DCMAKE_BUILD_TYPE=Release -DCMAKE_VERBOSE_MAKEFILE=ON -DCMAKE_FIND_USE_PACKAGE_REGISTRY=OFF -DBoost_NO_BOOST_CMAKE=ON -DBOOST_ROOT=/cvmfs/soft.computecanada.ca/easybuild/software/2020/avx2/Compiler/gcc9/boost/1.72.0 -DBoost_NO_SYSTEM_PATHS=ON  -DCMAKE_VERBOSE_MAKEFILE:BOOL=ON  -DENABLE_XHOST=OFF  -DCMAKE_SKIP_INSTALL_RPATH=ON -DPYTHON_EXECUTABLE=$EBROOTPYTHON/bin/python3 .
== 2023-02-16 16:25:28,600 run.py:236 INFO running cmd:  make  -j 8
== 2023-02-16 16:31:00,989 run.py:236 INFO running cmd:  make install  && mkdir -p /home/mboisson/.local/easybuild/software/2020/avx2/Compiler/gcc9/gudhi/3.7.1/lib/python3.9/site-packages && cd python && pip install --no-deps --no-build-isolation --prefix=/home/mboisson/.local/easybuild/software/2020/avx2/Compiler/gcc9/gudhi/3.7.1 .
== 2023-02-16 16:31:35,739 run.py:236 INFO running cmd: tar xzf /home/mboisson/.local/easybuild/sources/g/GUDHI/gudhi.3.7.1.tar.gz
== 2023-02-16 16:31:49,786 run.py:236 INFO running cmd:  cmake    -DCMAKE_INSTALL_PREFIX=/home/mboisson/.local/easybuild/software/2020/avx2/Compiler/gcc9/gudhi/3.7.1 -DCMAKE_BUILD_TYPE=Release -DCMAKE_VERBOSE_MAKEFILE=ON -DCMAKE_FIND_USE_PACKAGE_REGISTRY=OFF -DBoost_NO_BOOST_CMAKE=ON -DBOOST_ROOT=/cvmfs/soft.computecanada.ca/easybuild/software/2020/avx2/Compiler/gcc9/boost/1.72.0 -DBoost_NO_SYSTEM_PATHS=ON  -DCMAKE_VERBOSE_MAKEFILE:BOOL=ON  -DENABLE_XHOST=OFF  -DCMAKE_SKIP_INSTALL_RPATH=ON -DPYTHON_EXECUTABLE=$EBROOTPYTHON/bin/python3 .
== 2023-02-16 16:31:54,793 run.py:236 INFO running cmd:  make  -j 8
== 2023-02-16 16:37:28,439 run.py:236 INFO running cmd:  make install  && mkdir -p /home/mboisson/.local/easybuild/software/2020/avx2/Compiler/gcc9/gudhi/3.7.1/lib/python3.8/site-packages && cd python && pip install --no-deps --no-build-isolation --prefix=/home/mboisson/.local/easybuild/software/2020/avx2/Compiler/gcc9/gudhi/3.7.1 .
== 2023-02-16 16:38:20,622 run.py:236 INFO running cmd: python -c 'import gudhi'
== 2023-02-16 16:38:34,991 run.py:236 INFO running cmd: python -c 'import gudhi'
== 2023-02-16 16:38:49,693 run.py:236 INFO running cmd: python -c 'import gudhi'
== 2023-02-16 16:39:08,687 run.py:236 INFO running cmd: gcc -v
== 2023-02-16 16:39:08,696 run.py:236 INFO running cmd: ldd --version

This is now with the tests disabled.

mglisse commented 1 year ago

Actually, testing with python3 -c 'import gudhi; gudhi.pick_n_random_points(points=[], nb_points=0)', I can reproduce the issue with several official packages (conda/pip), but not all. I'll see what I can figure out.

mglisse commented 1 year ago

Ah, it was starting to ring a bell, and indeed: https://gcc.gnu.org/git/?p=gcc.git;a=commit;h=813ad9c4dd5a779f12ad2abf710c6e75a3117ef0

This was a bug in older versions of gcc. If at all possible, you should use a more recent compiler. If not, look at the file pick_n_random_points.h in gudhi, and change #if __cplusplus >= 201703L to #if 0 to avoid calling this broken code. The problem only affects strange code (asking for 0 random points from an empty range), so it doesn't seem too bad.

mboisson commented 1 year ago

Ah, thanks for the pointer. I'll see if we can backport the stdlib patch, or patch to skip this.

I note that the stdlib patch was only integrated in GCC version 11.3.0. Looking at the header files, all of our versions priori to GCC 11 have the same bug.

mglisse commented 1 year ago

Note that it was backported to other branches, so for instance gcc-9.5 is fine, even 8.5 should work, but you are using an old version (9.3) of gcc-9...

mboisson commented 1 year ago

Note that it was backported to other branches, so for instance gcc-9.5 is fine, even 8.5 should work, but you are using an old version (9.3) of gcc-9...

Quite possible indeed. Updating a compiler is not a trivial matter on a cluster, so we tend to pick one minor version per major version and stick with it forever.

mboisson commented 1 year ago

I confirm that building with newer stdlib works. You can close this issue.