Open yurivict opened 2 weeks ago
Hi @yurivict,
there is a short comment on that, namely here https://docs.arbor-sim.org/en/stable/contrib/test.html#python-tests It has proven practical, however, to run the following from the cmake build dir in the development cycle:
PYTHONPATH=`pwd`/python python3 -munittest discover -v -s ../python
The above command fails after some tests succeeding:
cd /usr/ports/science/py-arbor/work-py311/.build && /usr/bin/env NINJA_STATUS="[%p %s/%t] " XDG_DATA_HOME=/usr/ports/science/py-arbor/work-py311 XDG_CONFIG_HOME=/usr/ports/science/py-arbor/work-py311 XDG_CACHE_HOME=/usr/ports/science/py-arbor/work-py311/.cache HOME=/usr/ports/science/py-arbor/work-py311 PATH=/usr/local/libexec/ccache:/usr/ports/science/py-arbor/work-py311/.bin:/home/yuri/bin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin PKG_CONFIG_LIBDIR=/usr/ports/science/py-arbor/work-py311/.pkgconfig:/usr/local/libdata/pkgconfig:/usr/local/share/pkgconfig:/usr/libdata/pkgconfig MK_DEBUG_FILES=no MK_KERNEL_SYMBOLS=no SHELL=/bin/sh NO_LINT=YES DESTDIR=/usr/ports/science/py-arbor/work-py311/stage PREFIX=/usr/local LOCALBASE=/usr/local CC="cc" CFLAGS="-O2 -pipe -fstack-protector-strong -isystem /usr/local/include -fno-strict-aliasing " CPP="cpp" CPPFLAGS="-isystem /usr/local/include" LDFLAGS=" -fstack-protector-strong -L/usr/local/lib " LIBS="" CXX="c++" CXXFLAGS="-O2 -pipe -fstack-protector-strong -isystem /usr/local/include -fno-strict-aliasing -fPIC -isystem /usr/local/include " CCACHE_DIR="/tmp/.ccache" BSD_INSTALL_PROGRAM="install -s -m 555" BSD_INSTALL_LIB="install -s -m 0644" BSD_INSTALL_SCRIPT="install -m 555" BSD_INSTALL_DATA="install -m 0644" BSD_INSTALL_MAN="install -m 444" PYTHONPATH=/usr/ports/science/py-arbor/work-py311/stage/usr/local/lib/python3.11/site-packages /usr/local/bin/python3.11 -munittest discover -v -s /usr/ports/science/py-arbor/work-py311/arbor-0.10.1/python
test.unit.test_catalogues (unittest.loader._FailedTest.test.unit.test_catalogues) ... ERROR
test_spike_clearing (test.unit.test_clear_samplers.TestClearSamplers.test_spike_clearing) ... ok
test_context (test.unit.test_contexts.TestContexts.test_context) ... ok
test_context_allocation (test.unit.test_contexts.TestContexts.test_context_allocation) ... ok
test_default_allocation (test.unit.test_contexts.TestContexts.test_default_allocation) ... ok
test_default_context (test.unit.test_contexts.TestContexts.test_default_context) ... ok
test_exceptions_allocation (test.unit.test_contexts.TestContexts.test_exceptions_allocation) ... ok
test_set_allocation (test.unit.test_contexts.TestContexts.test_set_allocation) ... ok
test_iclamp (test.unit.test_decor.TestDecorClasses.test_iclamp) ... ok
test_domain_decomposition_exceptions (test.unit.test_domain_decompositions.TestDomain_Decompositions.test_domain_decomposition_exceptions) ... ok
test_domain_decomposition_heterogenous_CPU (test.unit.test_domain_decompositions.TestDomain_Decompositions.test_domain_decomposition_heterogenous_CPU) ... ok
test_domain_decomposition_heterogenous_GPU (test.unit.test_domain_decompositions.TestDomain_Decompositions.test_domain_decomposition_heterogenous_GPU) ... skipped 'GPU not enabled'
test_domain_decomposition_hints (test.unit.test_domain_decompositions.TestDomain_Decompositions.test_domain_decomposition_hints) ... ok
test_domain_decomposition_homogenous_CPU (test.unit.test_domain_decompositions.TestDomain_Decompositions.test_domain_decomposition_homogenous_CPU) ... ok
test_domain_decomposition_homogenous_GPU (test.unit.test_domain_decompositions.TestDomain_Decompositions.test_domain_decomposition_homogenous_GPU) ... skipped 'GPU not enabled'
test_event_generator_explicit_schedule (test.unit.test_event_generators.TestEventGenerator.test_event_generator_explicit_schedule) ... ok
test_event_generator_poisson_schedule (test.unit.test_event_generators.TestEventGenerator.test_event_generator_poisson_schedule) ... ok
test_event_generator_regular_schedule (test.unit.test_event_generators.TestEventGenerator.test_event_generator_regular_schedule) ... ok
test_gid_index_ctor_cell_member (test.unit.test_identifiers.TestCellMembers.test_gid_index_ctor_cell_member) ... ok
test_set_gid_index_cell_member (test.unit.test_identifiers.TestCellMembers.test_set_gid_index_cell_member) ... ok
test_fileio (test.unit.test_io.TestAccIo.test_fileio) ... ok
test_nameio (test.unit.test_io.TestAccIo.test_nameio) ... ok
test_pathio (test.unit.test_io.TestAccIo.test_pathio) ... ok
test_stringio (test.unit.test_io.TestAccIo.test_stringio) ... ok
test_fileio (test.unit.test_io.TestAscIo.test_fileio) ... ok
test_nameio (test.unit.test_io.TestAscIo.test_nameio) ... ok
test_pathio (test.unit.test_io.TestAscIo.test_pathio) ... ok
test_stringio (test.unit.test_io.TestAscIo.test_stringio) ... ok
test_serialize (test.unit.test_io.TestSerdes.test_serialize) ... ok
test_fileio (test.unit.test_io.TestSwcArborIo.test_fileio) ... ok
test_nameio (test.unit.test_io.TestSwcArborIo.test_nameio) ... ok
test_pathio (test.unit.test_io.TestSwcArborIo.test_pathio) ... ok
test_stringio (test.unit.test_io.TestSwcArborIo.test_stringio) ... ok
test_fileio (test.unit.test_io.TestSwcNeuronIo.test_fileio) ... ok
test_nameio (test.unit.test_io.TestSwcNeuronIo.test_nameio) ... ok
test_pathio (test.unit.test_io.TestSwcNeuronIo.test_pathio) ... ok
test_stringio (test.unit.test_io.TestSwcNeuronIo.test_stringio) ... ok
test_compose (test.unit.test_morphology.TestPlacePwlin.test_compose) ... ok
test_identity (test.unit.test_morphology.TestPlacePwlin.test_identity) ... ok
test_mpoint (test.unit.test_morphology.TestPlacePwlin.test_mpoint) ... ok
test_place_pwlin_id (test.unit.test_morphology.TestPlacePwlin.test_place_pwlin_id) ... ok
test_place_pwlin_isometry (test.unit.test_morphology.TestPlacePwlin.test_place_pwlin_isometry) ... ok
test_rotation (test.unit.test_morphology.TestPlacePwlin.test_rotation) ... ok
test_translation (test.unit.test_morphology.TestPlacePwlin.test_translation) ... ok
test_multiple_connections_rr_halt (test.unit.test_multiple_connections.TestMultipleConnections.test_multiple_connections_rr_halt) ... ok
test_multiple_connections_rr_no_halt (test.unit.test_multiple_connections.TestMultipleConnections.test_multiple_connections_rr_no_halt) ... ok
test_multiple_connections_uni (test.unit.test_multiple_connections.TestMultipleConnections.test_multiple_connections_uni) ... ok
test_probe_addr_metadata (test.unit.test_probes.TestCableProbes.test_probe_addr_metadata) ... ok
test_probe_addr_metadata (test.unit.test_probes.TestLifProbes.test_probe_addr_metadata) ... ok
test_probe_result (test.unit.test_probes.TestLifProbes.test_probe_result) ... ok
test_summary (test.unit.test_profiling.TestProfiling.test_summary) ... skipped 'run test only with profiling support'
test_support (test.unit.test_profiling.TestProfiling.test_support) ... ok
test_events_explicit_schedule (test.unit.test_schedules.TestExplicitSchedule.test_events_explicit_schedule) ... ok
test_exceptions_explicit_schedule (test.unit.test_schedules.TestExplicitSchedule.test_exceptions_explicit_schedule) ... ok
test_times_contor_explicit_schedule (test.unit.test_schedules.TestExplicitSchedule.test_times_contor_explicit_schedule) ... ok
test_events_poisson_schedule (test.unit.test_schedules.TestPoissonSchedule.test_events_poisson_schedule) ... ok
test_exceptions_poisson_schedule (test.unit.test_schedules.TestPoissonSchedule.test_exceptions_poisson_schedule) ... ok
test_freq_poisson_schedule (test.unit.test_schedules.TestPoissonSchedule.test_freq_poisson_schedule) ... ok
test_freq_seed_contor_poisson_schedule (test.unit.test_schedules.TestPoissonSchedule.test_freq_seed_contor_poisson_schedule) ... ok
test_freq_tstart_contor_poisson_schedule (test.unit.test_schedules.TestPoissonSchedule.test_freq_tstart_contor_poisson_schedule) ... ok
test_tstart_freq_seed_contor_poisson_schedule (test.unit.test_schedules.TestPoissonSchedule.test_tstart_freq_seed_contor_poisson_schedule) ... ok
test_tstop_poisson_schedule (test.unit.test_schedules.TestPoissonSchedule.test_tstop_poisson_schedule) ... ok
test_events_regular_schedule (test.unit.test_schedules.TestRegularSchedule.test_events_regular_schedule) ... ok
test_exceptions_regular_schedule (test.unit.test_schedules.TestRegularSchedule.test_exceptions_regular_schedule) ... ok
test_none_ctor_regular_schedule (test.unit.test_schedules.TestRegularSchedule.test_none_ctor_regular_schedule) ... ok
test_set_tstart_dt_tstop_regular_schedule (test.unit.test_schedules.TestRegularSchedule.test_set_tstart_dt_tstop_regular_schedule) ... ok
test_tstart_dt_tstop_ctor_regular_schedule (test.unit.test_schedules.TestRegularSchedule.test_tstart_dt_tstop_ctor_regular_schedule) ... ok
test_spikes_sorted (test.unit.test_spikes.TestSpikes.test_spikes_sorted) ... ok
======================================================================
ERROR: test.unit.test_catalogues (unittest.loader._FailedTest.test.unit.test_catalogues)
----------------------------------------------------------------------
ImportError: Failed to import test module: test.unit.test_catalogues
Traceback (most recent call last):
File "/usr/local/lib/python3.11/unittest/loader.py", line 419, in _find_test_path
module = self._get_module_from_name(name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/unittest/loader.py", line 362, in _get_module_from_name
__import__(name)
File "/usr/ports/science/py-arbor/work-py311/arbor-0.10.1/python/test/unit/test_catalogues.py", line 42, in <module>
class TestCatalogues(unittest.TestCase):
File "/usr/ports/science/py-arbor/work-py311/arbor-0.10.1/python/test/unit/test_catalogues.py", line 47, in TestCatalogues
@fixtures.dummy_catalogue()
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/ports/science/py-arbor/work-py311/arbor-0.10.1/python/test/fixtures.py", line 55, in decorator
fixture = fixture_factory(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/ports/science/py-arbor/work-py311/arbor-0.10.1/python/test/fixtures.py", line 40, in inject_fixture
return func(*bound.args, **bound.kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/ports/science/py-arbor/work-py311/arbor-0.10.1/python/test/fixtures.py", line 173, in dummy_catalogue
cat_path = _build_cat("dummy", path)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/ports/science/py-arbor/work-py311/arbor-0.10.1/python/test/fixtures.py", line 40, in inject_fixture
return func(*bound.args, **bound.kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/ports/science/py-arbor/work-py311/arbor-0.10.1/python/test/fixtures.py", line 161, in _build_cat
_build_cat_local(name, path)
File "/usr/ports/science/py-arbor/work-py311/arbor-0.10.1/python/test/fixtures.py", line 120, in _build_cat_local
raise _BuildCatError(
test.fixtures._BuildCatError: Tests can't build catalogue 'dummy' from '/usr/ports/science/py-arbor/work-py311/arbor-0.10.1/test/unit/dummy':
Traceback (most recent call last):
File "/usr/local/bin/arbor-build-catalogue", line 318, in <module>
sp.run(make_cmd, shell=True, check=True, capture_output = True, text = True)
File "/usr/local/lib/python3.11/subprocess.py", line 571, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command 'make' returned non-zero exit status 2.
Error:
c++: error: no input files
/bin/sh: -pipe: not found
/bin/sh: -fstack-protector-strong: not found
/bin/sh: -isystem: not found
/bin/sh: /usr/local/include: Permission denied
/bin/sh: -fno-strict-aliasing: not found
/bin/sh: -isystem: not found
/bin/sh: /usr/local/include: Permission denied
make[3]: *** [CMakeFiles/dummy-catalogue.dir/build.make:83: CMakeFiles/dummy-catalogue.dir/generated/dummy/dummy_catalogue.cpp.o] Error 126
make[2]: *** [CMakeFiles/Makefile2:83: CMakeFiles/dummy-catalogue.dir/all] Error 2
make[1]: *** [Makefile:91: all] Error 2
Building catalogue 'dummy' from mechanisms in /usr/ports/science/py-arbor/work-py311/arbor-0.10.1/test/unit/dummy
* NMODL
* dummy
Build log:
make[1]: Entering directory '/tmp/tmp_d0w1s6y/build'
[ 25%] modcc generating: /tmp/tmp_d0w1s6y/build/generated/dummy/dummy_catalogue.cpp;/tmp/tmp_d0w1s6y/build/generated/dummy/dummy_cpu.cpp
[ 50%] Building CXX object CMakeFiles/dummy-catalogue.dir/generated/dummy/dummy_catalogue.cpp.o
make[1]: Leaving directory '/tmp/tmp_d0w1s6y/build'
----------------------------------------------------------------------
Ran 68 tests in 0.064s
FAILED (errors=1, skipped=3)
*** Error code 1
Does it still fail after traversing to the arbor checkout and using this instead:
PYTHONPATH=<path/to/build> python3 -munittest discover -v -s python
? Reasoning: Some of our tests build a shared object plugin and it seems the fixture is misbehaving such that artifact is not found
Coming back to the error log, I'd also suspect something is off with the invocation: Quoting the log
c++: error: no input files
/bin/sh: -pipe: not found
/bin/sh: -fstack-protector-strong: not found
/bin/sh: -isystem: not found
/bin/sh: /usr/local/include: Permission denied
/bin/sh: -fno-strict-aliasing: not found
/bin/sh: -isystem: not found
/bin/sh: /usr/local/include: Permission denied
Possibly skip the relevant tests (those loading plugins) and/or check whether you can build a
plugin like so (after make install
; otherwise you'll have to adjust at least PYTHONPATH
and PATH
)
arbor-build-catalogue dummy [...]/test/unit/dummy/
Output should be similar to this
Building catalogue 'dummy' from mechanisms in [...]/test/unit/dummy
* NMODL
* dummy
Catalogue has been built and copied to [...]/dummy-catalogue.so
The Build/install page should explain how to run Python exension tests.
Running pytest in any of the directories test/, test/unit_distributed/ fails like this:
Version: 0.10.1 Python-3.11 FreeBSD 14.1