Typedef redefinition error

[keceli]

I see the error below while importing a module with cppyy bindings only on a system with OpenMPI 4.1.1. I am not sure if the problem is the MPI version or something else, but on my laptop with OpenMPI 4.0.3, I don't get this error. I'd appreciate any suggestions.

In file included from input_line_18:4:
In file included from /home/keceli/nwx/NWChemEx/build_nwx_gcc10_py3.9/_deps/madworld-src/src/madness/world/worldrmi.h:35:
In file included from /home/keceli/nwx/NWChemEx/build_nwx_gcc10_py3.9/_deps/madworld-src/src/madness/world/safempi.h:56:
In file included from /soft/libraries/mpi/openmpi/4.1.1/include/mpi.h:233:
/gpfs/jlse-fs0/users/keceli/gcc-10.4.0/install/bin/../lib/gcc/x86_64-pc-linux-gnu/10.4.0/include/stddef.h:426:3: error: typedef
      redefinition with different types ('struct max_align_t' vs 'struct max_align_t')
} max_align_t;
  ^
/tmp/pip-build-env-miu43i2j/normal/lib/python3.9/site-packages/cppyy_backend/etc/cling/lib/clang/9.0.1/include/__stddef_max_align_t.h:24:3: note: 
      previous definition is here
} max_align_t;

[wlav]

I'm willing to bet that it's not due to mpl.h per se, but a mismatch between the version of gcc enabled when building the precompiled header (which brings in that clang version) and now using the environment with mpi. Try rebuilding:

$ CLING_REBUILD_PCH=1 python -c "import cppyy"

Aside, the use of automatic I/O is still sitting in my inbox. :) Just that Numba support currently has priority.

[keceli]

I tried rebuilding, it didn't help. I also tried using both conda and pip installations without success. I run cppyy tests and got the following errors. I am not sure if they are related to the problem above, but wanted to report it here.

This is on openSUSE Leap 15.3. cppyy 2.4.0 installed with conda and Python 3.10.4. Tests are compiled with gcc 10.4.0-16.

========================================================================== FAILURES ==========================================================================
_____________________________________________________________ TestFRAGILE.test20_capture_output ______________________________________________________________

self = <test.test_fragile.TestFRAGILE object at 0x7f54772753f0>

    def test20_capture_output(self):
        """Capture cerr into a string"""

        import cppyy

        cppyy.cppdef(r"""\
        namespace capture {
        void say_hello() {
           std::cerr << "Hello, World\n";
        }

        void rdbuf_wa(std::ostream& o, std::basic_stringbuf<char>* b) {
           o.rdbuf(b);
        } }""")

        capture = cppyy.gbl.std.ostringstream()
        oldbuf = cppyy.gbl.std.cerr.rdbuf()

        try:
            cppyy.gbl.capture.rdbuf_wa(cppyy.gbl.std.cerr, capture.rdbuf())
            cppyy.gbl.capture.say_hello()
        finally:
            cppyy.gbl.std.cerr.rdbuf(oldbuf)

>       assert capture.str() == "Hello, World\n"
E       AssertionError: assert b'' == 'Hello, World\n'
E        +  where b'' = <cppyy.CPPOverload object at 0x7f547697f840>()
E        +    where <cppyy.CPPOverload object at 0x7f547697f840> = <cppyy.gbl.std.basic_ostringstream<char> object at 0x556be0573ae0>.str

test_fragile.py:496: AssertionError
_________________________________________________________________ TestREGRESSION.test04_avx __________________________________________________________________

self = <test.test_regression.TestREGRESSION object at 0x7f5477385180>

    def test04_avx(self):
        """Test usability of AVX by default."""

        import cppyy, subprocess

        has_avx = False
        try:
            f = open('/proc/cpuinfo', 'r')
            for line in f.readlines():
                if 'avx' in line:
                    has_avx = True
                    break
            f.close()
        except Exception:
            try:
                cli_arg = subprocess.check_output(['sysctl', 'machdep.cpu.features'])
                has_avx = 'avx' in cli_arg.decode("utf-8").strip().lower()
            except Exception:
                pass

        if has_avx:
>           assert cppyy.cppdef('int check_avx() { return (int) __AVX__; }')

test_regression.py:123: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

src = 'int check_avx() { return (int) __AVX__; }'

    def cppdef(src):
        """Declare C++ source <src> to Cling."""
        with _stderr_capture() as err:
            errcode = gbl.gInterpreter.Declare(src)
        if not errcode:
>           raise SyntaxError('Failed to parse the given C++ code%s' % err.err)
E           SyntaxError: Failed to parse the given C++ code
E           input_line_1926:1:32: error: use of undeclared identifier '__AVX__'
E           int check_avx() { return (int) __AVX__; }
E                                          ^
E           

/home/keceli/miniconda3/envs/nwx_dev/lib/python3.10/site-packages/cppyy/__init__.py:230: SyntaxError
____________________________________________________ TestREGRESSION.test33_explicit_template_in_namespace ____________________________________________________

self = <test.test_regression.TestREGRESSION object at 0x7f54773874c0>

    def test33_explicit_template_in_namespace(self):
        """Lookup of explicit template in namespace"""

        import cppyy

        cppyy.cppdef("""\
        namespace libchemist {
        namespace type {
            template<typename T> class tensor {};
        } // namespace type

        template<typename element_type = double, typename tensor_type = type::tensor<element_type>>
        class CanonicalMO {};

        template class CanonicalMO<double, type::tensor<double>>;

        auto produce() {
            return std::make_tuple(10., type::tensor<double>{});
        }

        } // namespace libchemist

        namespace property_types {
        namespace type {
            template<typename T>
            using canonical_mos = libchemist::CanonicalMO<T>;
        }

        auto produce() {
            return std::make_tuple(5., type::canonical_mos<double>{});
        }

        template<typename element_type = double, typename orbital_type = type::canonical_mos<element_type>>
        class ReferenceWavefunction {};

        template class ReferenceWavefunction<double>;

        template<class T>
        auto run_as() {
            return std::make_tuple(20., T{});
        } } // namespace type, property_types
        """)

        assert cppyy.gbl.property_types.type.canonical_mos['double']
        assert cppyy.gbl.std.get[0](cppyy.gbl.libchemist.produce())     == 10.
>       assert cppyy.gbl.std.get[0](cppyy.gbl.property_types.produce()) ==  5.
E       TypeError: Could not find "get<0>" (set cppyy.set_debug() for C++ errors):
E         double& std::get(std::tuple<double,libchemist::type::tensor<double> >& __t) =>
E           TypeError: could not convert argument 1
E         Failed to instantiate "get<0>(NoneType)"

test_regression.py:986: TypeError
_______________________________________________ TestTEMPLATES.test32_template_of_function_with_templated_args ________________________________________________

self = <test.test_templates.TestTEMPLATES object at 0x7f5477395c90>

    def test32_template_of_function_with_templated_args(self):
        """Lookup of templates of function with templated args used to fail"""

        import cppyy

        cppyy.cppdef("""\
        namespace parenthesis {
        template<class T>
        class F;

        template<class T>
        class V;

        using i = F<void (int)>;
        using v = F<void (V<int>)>;

        using ii = F<void (int,int)>;
        using iv = F<void (int,V<int>)>;
        using vi = F<void (V<int>,int)>;
        using vv = F<void (V<int>,V<int>)>;

        using iii = F<void (int,int,int)>;
        using ivi = F<void (int,V<int>,int)>;
        using vii = F<void (V<int>,int,int)>;
        using vvi = F<void (V<int>,V<int>,int)>;

        using iiv = F<void (int,int,V<int>)>;
        using ivv = F<void (int,V<int>,V<int>)>;
        using viv = F<void (V<int>,int,V<int>)>;
        using vvv = F<void (V<int>,V<int>,V<int>)>;
        }""")

        ns = cppyy.gbl.parenthesis

        for t in ['i','v',
                  'ii', 'iv', 'vi', 'vv',
                  'iii', 'ivi', 'vii', 'vvi',
                  'iiv', 'ivv', 'viv', 'vvv']:
>           assert getattr(ns, t)
E           AttributeError: <namespace cppyy.gbl.parenthesis at 0x556be940c570> has no attribute 'v'. Full details:
E             type object 'parenthesis' has no attribute 'v'
E             'parenthesis::v' is not a known C++ class
E             'v' is not a known C++ template
E             'v' is not a known C++ enum

test_templates.py:988: AttributeError
__________________________________________________ TestTEMPLATED_TYPEDEFS.test05_type_deduction_and_extern ___________________________________________________

self = <test.test_templates.TestTEMPLATED_TYPEDEFS object at 0x7f5477394b80>

    def test05_type_deduction_and_extern(self):
        """Usage of type reducer with extern template"""

        import cppyy
        import sys

        cppyy.cppdef("""\
        namespace FailedTypeDeducer {
        template<class T>
        class A {
        public:
            T result() { return T{5}; }
        };

        extern template class A<int>;
        }""")

        if sys.platform != 'darwin':   # feature disabled
>           assert cppyy.gbl.FailedTypeDeducer.A[int]().result()  == 42
E           assert 5 == 42
E            +  where 5 = <cppyy.CPPOverload object at 0x7f5476780300>()
E            +    where <cppyy.CPPOverload object at 0x7f5476780300> = <cppyy.gbl.FailedTypeDeducer.A<int> object at 0x556be9611fa0>.result
E            +      where <cppyy.gbl.FailedTypeDeducer.A<int> object at 0x556be9611fa0> = <class cppyy.gbl.FailedTypeDeducer.A<int> at 0x556be7428f50>()

test_templates.py:1197: AssertionError
================================================================== short test summary info ===================================================================
FAILED test_fragile.py::TestFRAGILE::test20_capture_output - AssertionError: assert b'' == 'Hello, World\n'
FAILED test_regression.py::TestREGRESSION::test04_avx - SyntaxError: Failed to parse the given C++ code
FAILED test_regression.py::TestREGRESSION::test33_explicit_template_in_namespace - TypeError: Could not find "get<0>" (set cppyy.set_debug() for C++ errors):
FAILED test_templates.py::TestTEMPLATES::test32_template_of_function_with_templated_args - AttributeError: <namespace cppyy.gbl.parenthesis at 0x556be940c5...
FAILED test_templates.py::TestTEMPLATED_TYPEDEFS::test05_type_deduction_and_extern - assert 5 == 42
========================================================= 5 failed, 461 passed, 24 skipped in 45.30s =========================================================

[keceli]

Turns out the errors with the tests here are due to a problem in my environment. Sorry for the noise.

For the main problem of the issue, I installed mpich instead of openmpi and got a different error: GLIBCXX_3.4.29 not found. I added libstdc++.so.6 manually to conda lib as described here and the problem is solved. I am still not sure why I cannot get it working with openmpi, but I am good with this solution.

[wlav]

There's a big conference in a week from now (ask Bert, he's organizing ;) ), so I'm slow to respond for a while longer.

Just want to note that libstdc++.so is loaded explicitly from the environment here: https://github.com/wlav/cppyy/blob/master/python/cppyy/_stdcpp_fix.py and there's an open issue explaining possible problems with that approach: https://github.com/wlav/cppyy/issues/59

[wlav]

For issues b/c of the libstdc++ version, this is probably relevant: https://github.com/wlav/cppyy/issues/97#issuecomment-1263443183. With the specific solution being to create a ~/.condarc:

channels: [conda-forge]
pip_interop_enabled: true
channel_priority: strict

to ensure GCC is picked up from conda-forge.