Python tests fail when using shared libraries

[hakonhagland]

On Ubuntu 23.10 I tested Python bindings using shared libraries like this:

#! /bin/bash

NPROC=5  # Number of build threads when running "make"
# export PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

for repo in opm-common opm-grid opm-models opm-simulators
do
    git clone git@github.com:OPM/"${repo}".git
done
global_cmake_flags="-DBUILD_SHARED_LIBS=ON"

build_opm_common() {
   local flags="-DOPM_ENABLE_PYTHON=ON $global_cmake_flags"
   cd opm-common
   mkdir build
   cd build
   cmake $flags ..
   make -j$NPROC
   cd ../..
}

build_opm_grid() {
   local flags="$global_cmake_flags"
   cd opm-grid
   mkdir build
   cd build
   cmake $flags ..
   make -j$NPROC
   cd ../..
}

build_opm_models() {
   local flags="$global_cmake_flags"
   cd opm-models
   mkdir build
   cd build
   cmake $flags ..
   make -j$NPROC
   cd ../..
}

build_opm_simulators() {
   local flags="-DOPM_ENABLE_PYTHON=ON $global_cmake_flags"
   cd opm-simulators
   mkdir build
   cd build
   cmake $flags ..
   make -j$NPROC
   cd ../..
}

build_opm_common
build_opm_grid
build_opm_models
build_opm_simulators

Then, after the build completed:

$ cd opm-simulators/build
$ ctest -R python_basic --output-on-failure
Test project /home/hakon/test/opm15/opm-simulators/build
    Start 54: python_basic
1/1 Test #54: python_basic .....................***Failed    3.80 sec
**********************************************************************
*                                                                    *
*                      This is flow 2024.10-pre                      *
*                                                                    *
* Flow is a simulator for fully implicit three-phase black-oil flow, *
*             including solvent and polymer capabilities.            *
*          For more information, see https://opm-project.org         *
*                                                                    *
**********************************************************************

Using 1 MPI processes with 2 OMP threads on each 

Reading deck file 'SPE1CASE1.DATA'
    0 Reading RUNSPEC  in SPE1CASE1.DATA line 20
    1 Reading TITLE    in SPE1CASE1.DATA line 23
    2 Reading DIMENS   in SPE1CASE1.DATA line 26
    3 Reading EQLDIMS  in SPE1CASE1.DATA line 31
    4 Reading TABDIMS  in SPE1CASE1.DATA line 36
    5 Reading OIL      in SPE1CASE1.DATA line 39
    6 Reading GAS      in SPE1CASE1.DATA line 40
    7 Reading WATER    in SPE1CASE1.DATA line 41
    8 Reading DISGAS   in SPE1CASE1.DATA line 42
    9 Reading FIELD    in SPE1CASE1.DATA line 46
   10 Reading START    in SPE1CASE1.DATA line 48
   11 Reading WELLDIMS in SPE1CASE1.DATA line 51
   12 Reading UNIFOUT  in SPE1CASE1.DATA line 62
   13 Reading GRID     in SPE1CASE1.DATA line 64
   14 Reading INIT     in SPE1CASE1.DATA line 71
   15 Reading NOECHO   in SPE1CASE1.DATA line 75
   16 Reading DX       in SPE1CASE1.DATA line 77
   17 Reading DY       in SPE1CASE1.DATA line 80
   18 Reading DZ       in SPE1CASE1.DATA line 83
   19 Reading TOPS     in SPE1CASE1.DATA line 87
   20 Reading PORO     in SPE1CASE1.DATA line 91
   21 Reading PERMX    in SPE1CASE1.DATA line 95
   22 Reading PERMY    in SPE1CASE1.DATA line 99
   23 Reading PERMZ    in SPE1CASE1.DATA line 103
   24 Reading ECHO     in SPE1CASE1.DATA line 107
   25 Reading PROPS    in SPE1CASE1.DATA line 109
   26 Reading PVTW     in SPE1CASE1.DATA line 112
   27 Reading ROCK     in SPE1CASE1.DATA line 125
   28 Reading SWOF     in SPE1CASE1.DATA line 132
   29 Reading SGOF     in SPE1CASE1.DATA line 161
   30 Reading DENSITY  in SPE1CASE1.DATA line 191
   31 Reading PVDG     in SPE1CASE1.DATA line 201
   32 Reading PVTO     in SPE1CASE1.DATA line 222
   33 Reading SOLUTION in SPE1CASE1.DATA line 252
   34 Reading EQUIL    in SPE1CASE1.DATA line 255
   35 Reading RSVD     in SPE1CASE1.DATA line 275
   36 Reading SUMMARY  in SPE1CASE1.DATA line 283
   37 Reading FOPR     in SPE1CASE1.DATA line 287
   38 Reading WGOR     in SPE1CASE1.DATA line 291
   39 Reading FGOR     in SPE1CASE1.DATA line 296
   40 Reading BPR      in SPE1CASE1.DATA line 299
   41 Reading BGSAT    in SPE1CASE1.DATA line 305
   42 Reading WBHP     in SPE1CASE1.DATA line 318
   43 Reading WGIR     in SPE1CASE1.DATA line 322
   44 Reading WGIT     in SPE1CASE1.DATA line 326
   45 Reading WGPR     in SPE1CASE1.DATA line 330
   46 Reading WGPT     in SPE1CASE1.DATA line 334
   47 Reading WOIR     in SPE1CASE1.DATA line 338
   48 Reading WOIT     in SPE1CASE1.DATA line 342
   49 Reading WOPR     in SPE1CASE1.DATA line 346
   50 Reading WOPT     in SPE1CASE1.DATA line 350
   51 Reading WWIR     in SPE1CASE1.DATA line 354
   52 Reading WWIT     in SPE1CASE1.DATA line 358
   53 Reading WWPR     in SPE1CASE1.DATA line 362
   54 Reading WWPT     in SPE1CASE1.DATA line 366
   55 Reading SCHEDULE in SPE1CASE1.DATA line 370
   56 Reading RPTSCHED in SPE1CASE1.DATA line 372
   57 Reading RPTRST   in SPE1CASE1.DATA line 375
   58 Reading DRSDT    in SPE1CASE1.DATA line 380
   59 Reading WELSPECS in SPE1CASE1.DATA line 385
   60 Reading COMPDAT  in SPE1CASE1.DATA line 394
   61 Reading WCONPROD in SPE1CASE1.DATA line 404
   62 Reading WCONINJE in SPE1CASE1.DATA line 414
   63 Reading TSTEP    in SPE1CASE1.DATA line 423

Warning: Unsupported keywords or keyword items:

  NOECHO: keyword not supported
  In file: SPE1CASE1.DATA, line 75

  ECHO: keyword not supported
  In file: SPE1CASE1.DATA, line 107

Creating grid from keywords DX, DY, DZ and TOPS
3 fluid phases are active

Processing dynamic information from
SPE1CASE1.DATA line 370
Initializing report step 0/12 at 2015-01-01 0 DAYS line 370
Processing keyword RPTSCHED at line 372
Processing keyword RPTRST at line 375
Processing keyword DRSDT at line 380
Processing keyword WELSPECS at line 385
Processing keyword COMPDAT at line 394
Processing keyword WCONPROD at line 404
Processing keyword WCONINJE at line 414
Complete report step 1 (31 DAYS) at 2015-02-01 (0 DAYS)

Initializing report step 2/12 at 2015-02-01 (0 DAYS) line 423
Complete report step 2 (28 DAYS) at 2015-03-01 (31 DAYS)

Initializing report step 3/12 at 2015-03-01 (31 DAYS) line 423
Complete report step 3 (31 DAYS) at 2015-04-01 (59 DAYS)

Initializing report step 4/12 at 2015-04-01 (59 DAYS) line 423
Complete report step 4 (30 DAYS) at 2015-05-01 (90 DAYS)

Initializing report step 5/12 at 2015-05-01 (90 DAYS) line 423
Complete report step 5 (31 DAYS) at 2015-06-01 (120 DAYS)

Processing grid
Total number of active cells: 300 / total pore volume: 534322820 RB

===============Saturation Functions Diagnostics===============

System:  Black-oil system.
Relative permeability input format: Saturation Family I.
Number of saturation regions: 1

python3: /home/hakon/test/opm15/opm-models/opm/models/blackoil/blackoilprimaryvariables.hh:437: void Opm::BlackOilPrimaryVariables<TypeTag>::assignNaive(const FluidState&) [with FluidState = Opm::BlackOilFluidState<double, Opm::BlackOilFluidSystem<double, Opm::BlackOilDefaultIndexTraits>, true, false, true, false, false, false, false, 3>; TypeTag = Opm::Properties::TTag::FlowProblemTPFA]: Assertion `FluidSystem::phaseIsActive(waterPhaseIdx)' failed.
[hakon-Precision-3570:539868] *** Process received signal ***
[hakon-Precision-3570:539868] Signal: Aborted (6)
[hakon-Precision-3570:539868] Signal code:  (-6)
[hakon-Precision-3570:539868] [ 0] /lib/x86_64-linux-gnu/libc.so.6(+0x42990)[0x7d35cfc42990]
[hakon-Precision-3570:539868] [ 1] /lib/x86_64-linux-gnu/libc.so.6(pthread_kill+0x11b)[0x7d35cfc99a1b]
[hakon-Precision-3570:539868] [ 2] /lib/x86_64-linux-gnu/libc.so.6(raise+0x16)[0x7d35cfc428e6]
[hakon-Precision-3570:539868] [ 3] /lib/x86_64-linux-gnu/libc.so.6(abort+0xd7)[0x7d35cfc268b7]
[hakon-Precision-3570:539868] [ 4] /lib/x86_64-linux-gnu/libc.so.6(+0x267db)[0x7d35cfc267db]
[hakon-Precision-3570:539868] [ 5] /lib/x86_64-linux-gnu/libc.so.6(+0x39206)[0x7d35cfc39206]
[hakon-Precision-3570:539868] [ 6] /home/hakon/test/opm15/opm-simulators/build/python/opm/simulators/simulators.cpython-311-x86_64-linux-gnu.so(+0x13dfab)[0x7d35ceb3dfab]
[hakon-Precision-3570:539868] [ 7] /home/hakon/test/opm15/opm-simulators/build/python/opm/simulators/simulators.cpython-311-x86_64-linux-gnu.so(+0x2a85c9)[0x7d35ceca85c9]
[hakon-Precision-3570:539868] [ 8] /home/hakon/test/opm15/opm-simulators/build/python/opm/simulators/simulators.cpython-311-x86_64-linux-gnu.so(+0xfeef6)[0x7d35ceafeef6]
[hakon-Precision-3570:539868] [ 9] /home/hakon/test/opm15/opm-simulators/build/python/opm/simulators/simulators.cpython-311-x86_64-linux-gnu.so(+0x122ecf)[0x7d35ceb22ecf]
[hakon-Precision-3570:539868] [10] /home/hakon/test/opm15/opm-simulators/build/python/opm/simulators/simulators.cpython-311-x86_64-linux-gnu.so(+0xe9cad)[0x7d35ceae9cad]
[hakon-Precision-3570:539868] [11] /usr/bin/python3[0x512776]
[hakon-Precision-3570:539868] [12] /usr/bin/python3(_PyObject_MakeTpCall+0x22b)[0x4e0bab]
[hakon-Precision-3570:539868] [13] /usr/bin/python3(_PyEval_EvalFrameDefault+0x88a)[0x4f627a]
[hakon-Precision-3570:539868] [14] /usr/bin/python3[0x54e402]
[hakon-Precision-3570:539868] [15] /usr/bin/python3[0x54db12]
[hakon-Precision-3570:539868] [16] /usr/bin/python3(_PyEval_EvalFrameDefault+0x4794)[0x4fa184]
[hakon-Precision-3570:539868] [17] /usr/bin/python3(_PyObject_FastCallDictTstate+0x217)[0x4e8ef7]
[hakon-Precision-3570:539868] [18] /usr/bin/python3(_PyObject_Call_Prepend+0x61)[0x52f3b1]
[hakon-Precision-3570:539868] [19] /usr/bin/python3[0x632f36]
[hakon-Precision-3570:539868] [20] /usr/bin/python3(_PyObject_MakeTpCall+0x22b)[0x4e0bab]
[hakon-Precision-3570:539868] [21] /usr/bin/python3(_PyEval_EvalFrameDefault+0x88a)[0x4f627a]
[hakon-Precision-3570:539868] [22] /usr/bin/python3[0x54e402]
[hakon-Precision-3570:539868] [23] /usr/bin/python3[0x54db12]
[hakon-Precision-3570:539868] [24] /usr/bin/python3(_PyEval_EvalFrameDefault+0x4794)[0x4fa184]
[hakon-Precision-3570:539868] [25] /usr/bin/python3(_PyObject_FastCallDictTstate+0x217)[0x4e8ef7]
[hakon-Precision-3570:539868] [26] /usr/bin/python3(_PyObject_Call_Prepend+0x61)[0x52f3b1]
[hakon-Precision-3570:539868] [27] /usr/bin/python3[0x632f36]
[hakon-Precision-3570:539868] [28] /usr/bin/python3(_PyObject_MakeTpCall+0x22b)[0x4e0bab]
[hakon-Precision-3570:539868] [29] /usr/bin/python3(_PyEval_EvalFrameDefault+0x88a)[0x4f627a]
[hakon-Precision-3570:539868] *** End of error message ***
Subprocess aborted

0% tests passed, 1 tests failed out of 1

Total Test time (real) =   3.81 sec

The following tests FAILED:
     54 - python_basic (Failed)
Errors while running CTest

[hakonhagland]

These tests works of course fine when not building shared libraries. Since the Jenkin build is not using shared libraries when testing the Python bindings, this issue could have been present for a long time.

[hakonhagland]

@blattms Could this issue be related to #5358 ?

[hakonhagland]

Assertion `FluidSystem::phaseIsActive(waterPhaseIdx)' failed.

I suspect this is an issue with static variables in shared libraries (opmsimulators.so, opmcommon.so). More specifically the static variable phaseIsActive_ in class BlackOilFluidSystem , see line 38 in BlackOilFluidSystem.hpp in opm-common, seems to suddenly change address in GDB.

[hakonhagland]

the static variable phaseIsActive_ in class BlackOilFluidSystem seems to suddenly change address in GDB.

It appears that static variable is undefined in libopmcommon.so:

$ nm -D lib/libopmcommon.so | grep phaseIsActive_ | c++filt
0000000002cd8681 u Opm::BlackOilFluidSystem<double, Opm::BlackOilDefaultIndexTraits>::phaseIsActive_
0000000002cd8685 u Opm::BlackOilFluidSystem<float, Opm::BlackOilDefaultIndexTraits>::phaseIsActive_

which might be the problem.

[blattms]

I have also reported problems in #5322. Is this the same assertion here?

[blattms]

@blattms Could this issue be related to #5358 ?

I am not expert, but it seems to look different to me.

[hakonhagland]

@blattms I think issue should be solved by https://github.com/OPM/opm-common/pull/4087

[blattms]

Thanks a lot for the fix.