[Bug] Tests failing on aarch64: `test_welch_psd_multidim_input`, `test_welch_cohere_multidim_input`

sanjayankur31 commented 2 years ago

Describe the bug

When building 0.11.0 for Fedora, these two tests fail on aarch64

To Reproduce

Buiild and run pytest on an aarch64 system

Expected behavior

Tests should pass

Environment

OS (e.g., Linux): Fedora Linux (rawhide, 37)
How you installed elephant (conda, pip, source): From source
Python version: 3.10
neo python package version: 0.10.2
elephant python package version: 0.11.0
(Any additional python package you want to include here)

Logs:

Output for convenience:

=================================== FAILURES ===================================
________________ WelchPSDTestCase.test_welch_psd_multidim_input ________________
self = <elephant.test.test_spectral.WelchPSDTestCase testMethod=test_welch_psd_multidim_input>
    def test_welch_psd_multidim_input(self):
        # generate multidimensional data
        num_channel = 4
        data_length = 5000
        sampling_period = 0.001
        noise = np.random.normal(size=(num_channel, data_length))
        data_np = np.array(noise)
        # Since row-column order in AnalogSignal is different from the
        # conventional one, `data_np` needs to be transposed when its used to
        # define an AnalogSignal
        data_neo = n.AnalogSignal(data_np.T,
                                  sampling_period=sampling_period * pq.s,
                                  units='mV')
        data_neo_1dim = n.AnalogSignal(data_np[0],
                                       sampling_period=sampling_period * pq.s,
                                       units='mV')

        # check if the results from different input types are identical
        freqs_np, psd_np = elephant.spectral.welch_psd(data_np,
                                                       fs=1 / sampling_period)
        freqs_neo, psd_neo = elephant.spectral.welch_psd(data_neo)
        freqs_neo_1dim, psd_neo_1dim = elephant.spectral.welch_psd(
            data_neo_1dim)
        self.assertTrue(np.all(freqs_np == freqs_neo))
        self.assertTrue(np.all(psd_np == psd_neo))
>       self.assertTrue(np.all(psd_neo_1dim == psd_neo[0]))
E       AssertionError: False is not true
elephant/test/test_spectral.py:159: AssertionError
_____________ WelchCohereTestCase.test_welch_cohere_multidim_input _____________
self = <elephant.test.test_spectral.WelchCohereTestCase testMethod=test_welch_cohere_multidim_input>
    def test_welch_cohere_multidim_input(self):
        # generate multidimensional data
        num_channel = 4
        data_length = 5000
        sampling_period = 0.001
        x_np = np.array(np.random.normal(size=(num_channel, data_length)))
        y_np = np.array(np.random.normal(size=(num_channel, data_length)))
        # Since row-column order in AnalogSignal is different from the
        # convention in NumPy/SciPy, `data_np` needs to be transposed when its
        # used to define an AnalogSignal
        x_neo = n.AnalogSignal(x_np.T, units='mV',
                               sampling_period=sampling_period * pq.s)
        y_neo = n.AnalogSignal(y_np.T, units='mV',
                               sampling_period=sampling_period * pq.s)
        x_neo_1dim = n.AnalogSignal(x_np[0], units='mV',
                                    sampling_period=sampling_period * pq.s)
        y_neo_1dim = n.AnalogSignal(y_np[0], units='mV',
                                    sampling_period=sampling_period * pq.s)

        # check if the results from different input types are identical
        freqs_np, coherency_np, phase_lag_np = elephant.spectral \
            .welch_coherence(x_np, y_np, fs=1 / sampling_period)
        freqs_neo, coherency_neo, phase_lag_neo = \
            elephant.spectral.welch_coherence(x_neo, y_neo)
        freqs_neo_1dim, coherency_neo_1dim, phase_lag_neo_1dim = \
            elephant.spectral.welch_coherence(x_neo_1dim, y_neo_1dim)
        self.assertTrue(np.all(freqs_np == freqs_neo))
        self.assertTrue(np.all(coherency_np.T == coherency_neo))
        self.assertTrue(np.all(phase_lag_np.T == phase_lag_neo))
>       self.assertTrue(
            np.all(coherency_neo_1dim[:, 0] == coherency_neo[:, 0]))
E       AssertionError: False is not true
elephant/test/test_spectral.py:468: AssertionError

The tests passed on all other arches we support: x86_64, i686, s390x, ppc64le. Here's the temporary build (will be garbage collected in a few days):

https://koji.fedoraproject.org/koji/taskinfo?taskID=85975318

Moritz-Alexander-Kern commented 2 years ago

Hey, thanks for reporting this @sanjayankur31.

We will look into this in connection with issue #410 , since both issues are related to an execution on a different architecture. So far, neither architecture has been the focus of our attention during elephant development, but we consider taking this into account for the future.

sanjayankur31 commented 2 years ago

Thanks very much. Please do let me know if I can help in any way. I guess it's also OK for elephant to not support these arches if the dev team does not want to target them, just as long as this is noted somewhere so that users are aware.

Moritz-Alexander-Kern commented 1 year ago

add note to docs: elephant currently only has active support for the x86_64 architecture.
more recent build of elephant succeeded, see: https://koji.fedoraproject.org/koji/taskinfo?taskID=91845461
this might be related to scipy

Feel free to reopen this, if it comes up again.

sanjayankur31 commented 1 year ago

Another on aarch64 with 0.12.0, noting here for records:

______ MultitaperCoherenceTestCase.test_multitaper_cohere_perfect_cohere _______
self = <elephant.test.test_spectral.MultitaperCoherenceTestCase testMethod=test_multitaper_cohere_perfect_cohere>
    def test_multitaper_cohere_perfect_cohere(self):
        # Generate dummy data
        data_length = 10000
        sampling_period = 0.001
        signal_freq = 100.0
        noise = np.random.normal(size=(1, data_length))
        time_points = np.arange(0, data_length * sampling_period,
                                sampling_period)
        signal = np.cos(2 * np.pi * signal_freq * time_points) + noise

        # Estimate coherence and phase lag with the multitaper method
        freq1, coh, phase_lag = elephant.spectral.multitaper_coherence(
            signal,
            signal,
            fs=1/sampling_period,
            n_segments=16)

        self.assertTrue((coh == np.ones(coh.size)).all())
>       self.assertTrue((phase_lag == np.zeros(phase_lag.size)).all())
E       AssertionError: False is not true
elephant/test/test_spectral.py:867: AssertionError

mdenker commented 1 year ago

Thank you for reporting this. Indeed, numerical precision is an issue for these unit tests. We'll try to come up with a better solution for handling these cases.

Moritz-Alexander-Kern commented 3 months ago

Addressed in #573 .

Feel free to reopen at any time.

NeuralEnsemble / elephant

[Bug] Tests failing on aarch64: `test_welch_psd_multidim_input`, `test_welch_cohere_multidim_input` #479