synsense / sinabs

A deep learning library for spiking neural networks which is based on PyTorch, focuses on fast training and supports inference on neuromorphic hardware.
https://sinabs.readthedocs.io
GNU Affero General Public License v3.0
80 stars 8 forks source link

Sinabs test that require hardware are skipped if no hardware is found but fail if a wrong hardware is found #242

Open ssinhaleite opened 4 months ago

ssinhaleite commented 4 months ago

The behavior of having no hardware or the wrong hardware should be similar. Tests could throw a warning saying the 'correct type of hardware could not be found', but it should not fail because the correct hardware is not there (if the tests pass without the hardware plugged in).

vleite@vleite-desktop:~/Software/sinabs$ pytest 
================================================================= test session starts =================================================================
platform linux -- Python 3.10.12, pytest-7.4.4, pluggy-1.2.0
Test order randomisation NOT enabled. Enable with --random-order or --random-order-bucket=<bucket_type>
rootdir: /home/vleite/Software/sinabs
plugins: anyio-4.3.0, random-order-1.1.0, hydra-core-1.3.2, xdist-3.3.1, cov-4.1.0, test-groups-1.0.3
collected 366 items / 2 errors                                                                                                                        

======================================================================= ERRORS ========================================================================
______________________________________________ ERROR collecting tests/test_dynapcnn/test_neuron_leak.py _______________________________________________
tests/test_dynapcnn/test_neuron_leak.py:64: in <module>
    not is_any_samna_device_connected(), reason="No samna device found!"
tests/test_dynapcnn/hw_utils.py:55: in is_any_samna_device_connected
    return len(find_open_devices()) > 0
tests/test_dynapcnn/hw_utils.py:50: in find_open_devices
    dev_dict.update({reverse_dict[dev_info.device_type_name]: dev_info})
E   KeyError: 'XyloAudio3TestBoard'
_________________________________________ ERROR collecting tests/test_dynapcnn/test_single_neuron_hardware.py _________________________________________
tests/test_dynapcnn/test_single_neuron_hardware.py:15: in <module>
    not is_any_samna_device_connected(), reason="No samna device found!"
tests/test_dynapcnn/hw_utils.py:55: in is_any_samna_device_connected
    return len(find_open_devices()) > 0
tests/test_dynapcnn/hw_utils.py:50: in find_open_devices
    dev_dict.update({reverse_dict[dev_info.device_type_name]: dev_info})
E   KeyError: 'XyloAudio3TestBoard'
=============================================================== short test summary info ===============================================================
ERROR tests/test_dynapcnn/test_neuron_leak.py - KeyError: 'XyloAudio3TestBoard'
ERROR tests/test_dynapcnn/test_single_neuron_hardware.py - KeyError: 'XyloAudio3TestBoard'
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 2 errors during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
================================================================== 2 errors in 1.61s ==================================================================