Is arch support for NEURON limited to `x86_64`?

[sanjayankur31]

Hi there,

I see that the path to the NEURON libnrnmech.so file directory is hard coded here to x86_64 only:

https://github.com/unibe-cns/NEAT/blob/master/neat/tools/simtools/neuron/neuronmodel.py#L59

On other arches, though, for example i686 or ppc or s390x and so on, the special directory will not be called x86_64.

Does this mean that NEAT only supports x86_64, or can we replace this with platform.machine() to let it function on all arches?

Thanks very much

[sanjayankur31]

With this change, all tests pass on multiple architectures:

https://github.com/sanjayankur31/NEAT/commit/2c0591d9255a958ec0a44041926685232dfcb7e2

• x86_64
• s390x
• ppc64le
• aarch64

Here's a test build on the Fedora community build system:

https://koji.fedoraproject.org/koji/taskinfo?taskID=86172865

On i686, the libnrnmech.so file is found too, but tests fail for different reasons. Errors below:

+ /usr/bin/pytest ../
============================= test session starts ==============================
platform linux -- Python 3.10.4, pytest-7.0.1, pluggy-1.0.0
rootdir: /builddir/build/BUILD/NEAT-0.9.2
collected 80 items
../tests/test_cnet.py FFFFF                                              [  6%]
../tests/test_compartmentfitter.py ..........                            [ 18%]
../tests/test_compartmenttree.py .........                               [ 30%]
../tests/test_greenstree.py ..                                           [ 32%]
../tests/test_ionchannels.py ....                                        [ 37%]
../tests/test_morphtree.py ..................                            [ 60%]
../tests/test_net.py .....                                               [ 66%]
../tests/test_neurontree.py .......                                      [ 75%]
../tests/test_phystree.py ....                                           [ 80%]
../tests/test_sovtree.py ...                                             [ 83%]
../tests/test_stree.py .............                                     [100%]
=================================== FAILURES ===================================
___________________________ TestCNET.testIOFunctions ___________________________
self = <test_cnet.TestCNET object at 0xe8a624d8>
    def testIOFunctions(self):
>       self.createTree()
../tests/test_cnet.py:188: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../tests/test_cnet.py:98: in createTree
    self.cnet = netsim.NETSim(net_py, v_eq=self.v_eq)
neat/tools/simtools/net/netsim.pyx:119: in netsim.NETSim.__cinit__
    ???
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
>   ???
E   ValueError: Buffer dtype mismatch, expected 'int64_t' but got 'long'
neat/tools/simtools/net/netsim.pyx:139: ValueError
_____________________________ TestCNET.testSolver ______________________________
self = <test_cnet.TestCNET object at 0xe8a623e8>
    def testSolver(self):
>       self.createTree()
../tests/test_cnet.py:233: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../tests/test_cnet.py:98: in createTree
    self.cnet = netsim.NETSim(net_py, v_eq=self.v_eq)
neat/tools/simtools/net/netsim.pyx:119: in netsim.NETSim.__cinit__
    ???
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
>   ???
E   ValueError: Buffer dtype mismatch, expected 'int64_t' but got 'long'
neat/tools/simtools/net/netsim.pyx:139: ValueError
___________________________ TestCNET.testIntegration ___________________________
self = <test_cnet.TestCNET object at 0xe8a625f8>
    def testIntegration(self):
        tmax = 1000.; dt = 0.025
>       self.createTree()
../tests/test_cnet.py:328: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../tests/test_cnet.py:98: in createTree
    self.cnet = netsim.NETSim(net_py, v_eq=self.v_eq)
neat/tools/simtools/net/netsim.pyx:119: in netsim.NETSim.__cinit__
    ???
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
>   ???
E   ValueError: Buffer dtype mismatch, expected 'int64_t' but got 'long'
neat/tools/simtools/net/netsim.pyx:139: ValueError
____________________________ TestCNET.testInversion ____________________________
self = <test_cnet.TestCNET object at 0xe8a62688>
    def testInversion(self):
        dt = 0.1

        # tests without linear terms
        # test with two non-leafs that integrate soma
>       self.createTree2(add_lin=False)
../tests/test_cnet.py:437: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../tests/test_cnet.py:138: in createTree2
    self.cnet = netsim.NETSim(net_py, lin_terms=self.lin_terms,
neat/tools/simtools/net/netsim.pyx:119: in netsim.NETSim.__cinit__
    ???
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
>   ???
E   ValueError: Buffer dtype mismatch, expected 'int64_t' but got 'long'
neat/tools/simtools/net/netsim.pyx:139: ValueError
_____________________________ TestCNET.testChannel _____________________________
self = <test_cnet.TestCNET object at 0xe8a62718>
    def testChannel(self):
>       self.createPointNeurons()
../tests/test_cnet.py:522: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../tests/test_cnet.py:53: in createPointNeurons
    self.cnet = netsim.NETSim(net_py, v_eq=self.v_eq)
neat/tools/simtools/net/netsim.pyx:119: in netsim.NETSim.__cinit__
    ???
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
>   ???
E   ValueError: Buffer dtype mismatch, expected 'int64_t' but got 'long'
neat/tools/simtools/net/netsim.pyx:139: ValueError
=============================== warnings summary ===============================
tests/test_compartmentfitter.py: 70 warnings
tests/test_neurontree.py: 240 warnings
  /builddir/build/BUILDROOT/python-NEAT-0.9.2-5.fc37.i386/usr/lib/python3.10/site-packages/neat/tools/simtools/neuron/neuronmodel.py:763: DeprecationWarning: neuron.run(tstop) is deprecated; use h.stdinit() and h.continuerun(tstop) instead
    neuron.run(t_max + self.t_calibrate)
tests/test_morphtree.py::TestMorphTree::testPlotting
tests/test_morphtree.py::TestMorphTree::testPlotting
  /builddir/build/BUILDROOT/python-NEAT-0.9.2-5.fc37.i386/usr/lib/python3.10/site-packages/neat/tools/plottools/scalebars.py:31: MatplotlibDeprecationWarning: 
  The 'minimumdescent' parameter of __init__() was deprecated in Matplotlib 3.4 and will be removed two minor releases later. If any parameter follows 'minimumdescent', they should be passed as keyword, not positionally.
    bars = VPacker(children=[bars, TextArea(labelx, minimumdescent=False)],
tests/test_net.py::TestNET::testKernels
  /builddir/build/BUILDROOT/python-NEAT-0.9.2-5.fc37.i386/usr/lib/python3.10/site-packages/neat/trees/netree.py:155: RuntimeWarning: invalid value encountered in true_divide
    return np.sum(self.c[:,None]*1e3 / (self.a[:,None]*1e3 + s_arr[None,:]), 0)
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
FAILED ../tests/test_cnet.py::TestCNET::testIOFunctions - ValueError: Buffer ...
FAILED ../tests/test_cnet.py::TestCNET::testSolver - ValueError: Buffer dtype...
FAILED ../tests/test_cnet.py::TestCNET::testIntegration - ValueError: Buffer ...
FAILED ../tests/test_cnet.py::TestCNET::testInversion - ValueError: Buffer dt...
FAILED ../tests/test_cnet.py::TestCNET::testChannel - ValueError: Buffer dtyp...
============ 5 failed, 75 passed, 313 warnings in 134.36s (0:02:14) ============

[WillemWybo]

Hi @sanjayankur31,

thanks so much for this suggestion. None of the core NEAT code should be platform-specific, so we can apply your suggested change. Could you create a PR?

As for the tests that fail, these specific tests have nothing to do core NEAT code for simplifying compartmental models. The issue seems to be that the function _addNode() in neat/tools/simtools/net/netsim.pyx expects numpy array with cython dtype np.int64_t but receives whatever dtype is create in lines 111, 113, 115 of the same file, and on i686 this gives an error. I've made a tentative fix, https://github.com/unibe-cns/NEAT/pull/143, would you mind checking in your build system whether it passes on all platforms?

[sanjayankur31]

Hi @WillemWybo ,

Thank you for your reply.

I've opened #144 for the simple arch fix. I'll test out your fix on our builders and report back ASAP too.

[WillemWybo]

Resolved by PR #144