1.2.0: failing test on s390x (Big endian): ERROR collecting test/test_subsystem_phi_max.py

[sanjayankur31]

Hello,

We're seeing a test failing on s390x while building pyphi for NeuroFedora:

CPU info:
Architecture:                    s390x
CPU op-mode(s):                  32-bit, 64-bit
Byte Order:                      Big Endian
CPU(s):                          3
On-line CPU(s) list:             0-2
Thread(s) per core:              1
Core(s) per socket:              1
Socket(s) per book:              1
Book(s) per drawer:              1
Drawer(s):                       3
NUMA node(s):                    1
Vendor ID:                       IBM/S390
Machine type:                    2964
CPU dynamic MHz:                 5000
CPU static MHz:                  5000
BogoMIPS:                        3033.00
Hypervisor:                      KVM/Linux
Hypervisor vendor:               KVM
Virtualization type:             full
Dispatching mode:                horizontal
L1d cache:                       384 KiB
L1i cache:                       288 KiB
L2d cache:                       6 MiB
L2i cache:                       6 MiB
L3 cache:                        64 MiB
L4 cache:                        480 MiB
NUMA node0 CPU(s):               0-2
Vulnerability Itlb multihit:     Not affected
Vulnerability L1tf:              Not affected
Vulnerability Mds:               Not affected
Vulnerability Meltdown:          Not affected
Vulnerability Spec store bypass: Not affected
Vulnerability Spectre v1:        Mitigation; __user pointer sanitization
Vulnerability Spectre v2:        Mitigation; execute trampolines
Vulnerability Srbds:             Not affected
Vulnerability Tsx async abort:   Not affected
Flags:                           esan3 zarch stfle msa ldisp eimm dfp edat etf3eh highgprs te vx sie

The build logs are below:

+ py.test-3.9
============================= test session starts ==============================
platform linux -- Python 3.9.0rc1, pytest-6.0.2, py-1.9.0, pluggy-0.13.1 -- /usr/bin/python3
cachedir: .pytest_cache
rootdir: /builddir/build/BUILD/pyphi-1.2.0, configfile: pytest.ini
plugins: lazy-fixture-0.6.3
collecting ... collected 482 items / 1 error / 481 selected
==================================== ERRORS ====================================
_______________ ERROR collecting test/test_subsystem_phi_max.py ________________
test/test_subsystem_phi_max.py:63: in <module>
    expected_mips = {
test/test_subsystem_phi_max.py:64: in <dictcomp>
    cut: {
test/test_subsystem_phi_max.py:65: in <dictcomp>
    direction: {
test/test_subsystem_phi_max.py:66: in <dictcomp>
    mechanism: subsystem[cut].find_mip(direction, mechanism, purview)
pyphi/subsystem.py:585: in find_mip
    phi, partitioned_repertoire = self.evaluate_partition(
pyphi/subsystem.py:535: in evaluate_partition
    phi = repertoire_distance(
pyphi/distance.py:281: in repertoire_distance
    dist = directional_emd(direction, r1, r2)
pyphi/distance.py:266: in directional_emd
    return round(func(d1, d2), config.PRECISION)
pyphi/distance.py:143: in hamming_emd
    return emd(d1, d2, _hamming_matrix(N))
pyemd/emd.pyx:49: in pyemd.emd.emd
    ???
E   ValueError: Little-endian buffer not supported on big-endian compiler
=========================== short test summary info ============================
ERROR test/test_subsystem_phi_max.py - ValueError: Little-endian buffer not s...
!!!!!!!!!!!!!!!!!!!!!!!!!! stopping after 1 failures !!!!!!!!!!!!!!!!!!!!!!!!!!!
=============================== 1 error in 1.14s ===============================

The complete build logs are attached:

pyphi-1.2.0-buildlog.txt pyphi-1.2.0-rootlog.txt

[sanjayankur31]

Another one:

=================================== FAILURES ===================================
___________________________ [doctest] 2014paper.rst ____________________________
181     >>> subsystem.cause_repertoire((A,), (B, C, D))
182     array([[[[0.125, 0.125],
183              [0.125, 0.125]],
184     <BLANKLINE>
185             [[0.125, 0.125],
186              [0.125, 0.125]]]])
187 
188 And this gives us zero cause information:
189 
190     >>> subsystem.cause_info((A,), (B, C, D))
UNEXPECTED EXCEPTION: ValueError('Little-endian buffer not supported on big-endian compiler')
Traceback (most recent call last):
  File "/usr/lib64/python3.9/doctest.py", line 1336, in __run
    exec(compile(example.source, filename, "single",
  File "<doctest 2014paper.rst[12]>", line 1, in <module>
  File "/builddir/build/BUILD/pyphi-1.2.0/pyphi/subsystem.py", line 485, in cause_info
    return repertoire_distance(
  File "/builddir/build/BUILD/pyphi-1.2.0/pyphi/distance.py", line 281, in repertoire_distance
    dist = directional_emd(direction, r1, r2)
  File "/builddir/build/BUILD/pyphi-1.2.0/pyphi/distance.py", line 266, in directional_emd
    return round(func(d1, d2), config.PRECISION)
  File "/builddir/build/BUILD/pyphi-1.2.0/pyphi/distance.py", line 143, in hamming_emd
    return emd(d1, d2, _hamming_matrix(N))
  File "pyemd/emd.pyx", line 49, in pyemd.emd.emd
ValueError: Little-endian buffer not supported on big-endian compiler
/builddir/build/BUILD/pyphi-1.2.0/docs/examples/2014paper.rst:190: UnexpectedException
------------------------------ Captured log setup ------------------------------

[wmayner]

Thanks, this is a problem in another one of my libraries, pyemd. I'll have a look.

[musicinmybrain]

I just verified that this is still a problem with pyphi 1.2.1 and the latest dependencies in Fedora.