search

OpenMDAO / OpenMDAO

OpenMDAO repository.
http://openmdao.org
Other
537 stars 244 forks source link

Unreliable test: test_feature_cache_linear #1769

Closed swryan closed 3 years ago

swryan commented 3 years ago

Summary of Issue

Unreliable test: test_feature_cache_linear_solution.py:CacheLinearTestCase.test_feature_cache_linear

Issue Type

Description

This test fails on the cluster and on a personal Ubuntu VM (19.10) with recent versions of OpenMDAO, including 3.4.1, and latest versions of Python, NumPy and SciPy installed via Conda. It does not fail on CI.

Example

/mdao/u/swryan/benchmark/runner/openmdao_repos/OpenMDAO/openmdao/core/tests/test_feature_cache_linear_solution.py:61: RuntimeWarning:invalid value encountered in sqrt
Traceback (most recent call last):
  File "/mdao/u/swryan/anaconda3/envs/openmdao_20201113-005300/lib/python3.9/site-packages/testflo/test.py", line 425, in _try_call
    func()
  File "/mdao/u/swryan/benchmark/runner/openmdao_repos/OpenMDAO/openmdao/core/tests/test_feature_cache_linear_solution.py", line 110, in test_feature_cache_linear
    assert_near_equal(p['obj.y'], 0.25029766, 1e-3)
  File "/mdao/u/swryan/benchmark/runner/openmdao_repos/OpenMDAO/openmdao/utils/assert_utils.py", line 440, in assert_near_equal
    raise ValueError('actual %s, desired %s' % (actual, desired))
ValueError: actual [nan], desired [0.25029766]

Environment

[16:18:30](release)   ~ >> lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 19.10
Release:    19.10
Codename:   eoan
[16:18:37](release)   ~ >> conda list
# packages in environment at /home/swryan/anaconda3/envs/release:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                 conda_forge    conda-forge
_openmp_mutex             4.5                       1_gnu    conda-forge
alabaster                 0.7.12                   pypi_0    pypi
appdirs                   1.4.4                    pypi_0    pypi
babel                     2.9.0                    pypi_0    pypi
baron                     0.9                      pypi_0    pypi
bokeh                     2.2.3                    pypi_0    pypi
c-ares                    1.16.1               h516909a_3    conda-forge
ca-certificates           2020.11.8            ha878542_0    conda-forge
certifi                   2020.11.8        py39hf3d152e_0    conda-forge
chardet                   3.0.4                    pypi_0    pypi
colorama                  0.4.4                    pypi_0    pypi
coverage                  4.5.4                    pypi_0    pypi
cycler                    0.10.0                   pypi_0    pypi
cython                    0.29.21          py39hfbd4741_1    conda-forge
decorator                 4.4.2                    pypi_0    pypi
docutils                  0.16                     pypi_0    pypi
hdf5                      1.10.6          nompi_h1022a3e_1110    conda-forge
hypre                     2.18.2               h3c37bf7_1    conda-forge
idna                      2.10                     pypi_0    pypi
imagesize                 1.2.0                    pypi_0    pypi
jinja2                    2.11.2                   pypi_0    pypi
kiwisolver                1.3.1                    pypi_0    pypi
krb5                      1.17.1               hfafb76e_3    conda-forge
ld_impl_linux-64          2.35.1               hed1e6ac_0    conda-forge
libblas                   3.9.0                2_openblas    conda-forge
libcblas                  3.9.0                2_openblas    conda-forge
libcurl                   7.71.1               hcdd3856_8    conda-forge
libedit                   3.1.20191231         he28a2e2_2    conda-forge
libev                     4.33                 h516909a_1    conda-forge
libffi                    3.2.1             he1b5a44_1007    conda-forge
libgcc-ng                 9.3.0               h5dbcf3e_17    conda-forge
libgfortran-ng            9.3.0               he4bcb1c_17    conda-forge
libgfortran5              9.3.0               he4bcb1c_17    conda-forge
libgomp                   9.3.0               h5dbcf3e_17    conda-forge
liblapack                 3.9.0                2_openblas    conda-forge
libnghttp2                1.41.0               h8cfc5f6_2    conda-forge
libopenblas               0.3.12          pthreads_h4812303_1    conda-forge
libssh2                   1.9.0                hab1572f_5    conda-forge
libstdcxx-ng              9.3.0               h2ae2ef3_17    conda-forge
markupsafe                1.1.1                    pypi_0    pypi
matplotlib                3.3.3                    pypi_0    pypi
metis                     5.1.0             he1b5a44_1006    conda-forge
mock                      4.0.2                    pypi_0    pypi
mpi                       1.0                     openmpi    anaconda
mpi4py                    3.0.3            py39hd96e10c_2    conda-forge
mumps-include             5.2.1               ha770c72_10    conda-forge
mumps-mpi                 5.2.1               h4f59553_10    conda-forge
ncurses                   6.2                  h58526e2_3    conda-forge
networkx                  2.5                      pypi_0    pypi
numpy                     1.19.4           py39h57d35e7_1    conda-forge
numpydoc                  1.1.0                    pypi_0    pypi
openmdao                  3.4.1                    pypi_0    pypi
openmpi                   4.0.5                h2c4babf_3    conda-forge
openssl                   1.1.1h               h516909a_0    conda-forge
packaging                 20.4                     pypi_0    pypi
parameterized             0.7.4                    pypi_0    pypi
parmetis                  4.0.3             hc401efc_1005    conda-forge
pcre                      8.44                 he1b5a44_0    conda-forge
petsc                     3.14.1               hc914707_0    conda-forge
petsc4py                  3.14.0           py39hd899e5a_1    conda-forge
pillow                    8.0.1                    pypi_0    pypi
pip                       20.2.4                     py_0    conda-forge
ptscotch                  6.0.9                h9271a0d_1    conda-forge
pycodestyle               2.6.0                    pypi_0    pypi
pydocstyle                2.0.0                    pypi_0    pypi
pydoe2                    1.3.0                    pypi_0    pypi
pyee                      7.0.4                    pypi_0    pypi
pygments                  2.7.2                    pypi_0    pypi
pyoptsparse               2.1.5                    pypi_0    pypi
pyparsing                 2.4.7                    pypi_0    pypi
pyppeteer                 0.2.2                    pypi_0    pypi
python                    3.9.0           h2a148a8_4_cpython    conda-forge
python-dateutil           2.8.1                    pypi_0    pypi
python_abi                3.9                      1_cp39    conda-forge
pytz                      2020.4                   pypi_0    pypi
pyyaml                    5.3.1                    pypi_0    pypi
readline                  8.0                  he28a2e2_2    conda-forge
redbaron                  0.9.2                    pypi_0    pypi
requests                  2.25.0                   pypi_0    pypi
rply                      0.7.7                    pypi_0    pypi
scalapack                 2.0.2             hc7fdd6d_1009    conda-forge
scipy                     1.5.3            py39hee8e79c_0    conda-forge
scotch                    6.0.9                h0eec0ba_1    conda-forge
setuptools                49.6.0           py39h079e4ff_2    conda-forge
six                       1.15.0                   pypi_0    pypi
snowballstemmer           2.0.0                    pypi_0    pypi
sphinx                    3.3.1                    pypi_0    pypi
sphinxcontrib-applehelp   1.0.2                    pypi_0    pypi
sphinxcontrib-devhelp     1.0.2                    pypi_0    pypi
sphinxcontrib-htmlhelp    1.0.3                    pypi_0    pypi
sphinxcontrib-jsmath      1.0.1                    pypi_0    pypi
sphinxcontrib-qthelp      1.0.3                    pypi_0    pypi
sphinxcontrib-serializinghtml 1.1.4                    pypi_0    pypi
sqlite                    3.33.0               h4cf870e_1    conda-forge
sqlitedict                1.7.0                    pypi_0    pypi
suitesparse               5.6.0                h717dc36_0    conda-forge
superlu                   5.2.2                he1ec49c_0    conda-forge
swig                      4.0.2                he1b5a44_0    conda-forge
tbb                       2020.2               hc9558a2_0    conda-forge
testflo                   1.4.2                    pypi_0    pypi
tk                        8.6.10               hed695b0_1    conda-forge
tornado                   6.1                      pypi_0    pypi
tqdm                      4.51.0                   pypi_0    pypi
typing-extensions         3.7.4.3                  pypi_0    pypi
tzdata                    2020d                h516909a_0    conda-forge
urllib3                   1.26.2                   pypi_0    pypi
websockets                8.1                      pypi_0    pypi
wheel                     0.35.1             pyh9f0ad1d_0    conda-forge
xz                        5.2.5                h516909a_1    conda-forge
zlib                      1.2.11            h516909a_1010    conda-forge
[16:18:41](release)   ~ >>
swryan commented 3 years ago

This appears to be related to different builds of some of the underlying libraries depending on whether conda-forge is included in the configured channels, e.g. due to conda config --add conda-forge.

We use the conda-forge build of petsc4py but care must be taken that other libraries are not installed from there by ensuring that conda-forge is not in .condarc and by installing petsc4py using conda install -c conda-forge as seen in the OpenMDAO travis.yml