search

lava-nc / lava

A Software Framework for Neuromorphic Computing
https://lava-nc.org
Other
554 stars 144 forks source link

PyBuilder becomes stuck on unit test execution (and other test failures) #136

Closed hthompson6 closed 2 years ago

hthompson6 commented 2 years ago

Description

I've been trying to go through the installation steps, but encountered the issues listed below. I didn't see a Python version specified, so I tried a couple of them. Though, I had seen that 3.6 was used for unit tests in another issue.

Note: I was able to install and execute the tutorial code successfully on python3.8. Only the final PyBuilder unit test step was problematic.

Environment

Manual Testing

Attempted Installation Using Python3.6

Fails due to future annotations not being available before python3.7

Steps

cd lava
virtualenv --python=/usr/bin/python3.6 ~/env_lava_36
source ~/env_lava_36/bin/activate
pip install -U pip
pip install -r build-requirements.txt
pip install -r requirements.txt
pip install .
export PYTHONPATH=$(pwd)/src
pyb -E unit

Output

PyBuilder version 0.13.3
Build started at 2021-11-30 17:47:44
------------------------------------------------------------
[INFO]  Installing or updating plugin "pypi:pybuilder_bandit, module name 'pybuilder_bandit'"
[INFO]  Processing plugin packages 'pybuilder_bandit' to be installed with {}
[INFO]  Activated environments: unit
[INFO]  Building lava-nc version 0.2.0
[INFO]  Executing build in /home/flynnto/self_study/lava
[INFO]  Going to execute tasks: analyze, publish
[INFO]  Processing plugin packages 'coverage~=5.2' to be installed with {'upgrade': True}
[INFO]  Processing plugin packages 'flake8~=3.7' to be installed with {'upgrade': True}
[INFO]  Processing plugin packages 'pypandoc~=1.4' to be installed with {'upgrade': True}
[INFO]  Processing plugin packages 'setuptools>=38.6.0' to be installed with {'upgrade': True}
[INFO]  Processing plugin packages 'sphinx_rtd_theme' to be installed with {}
[INFO]  Processing plugin packages 'sphinx_tabs' to be installed with {}
[INFO]  Processing plugin packages 'twine>=1.15.0' to be installed with {'upgrade': True}
[INFO]  Processing plugin packages 'unittest-xml-reporting~=3.0.4' to be installed with {'upgrade': True}
[INFO]  Processing plugin packages 'wheel>=0.34.0' to be installed with {'upgrade': True}
[INFO]  Creating target 'build' VEnv in '/home/flynnto/self_study/lava/target/venv/build/cpython-3.6.15.final.0'
[INFO]  Processing dependency packages 'requirements.txt' to be installed with {}
[INFO]  Creating target 'test' VEnv in '/home/flynnto/self_study/lava/target/venv/test/cpython-3.6.15.final.0'
[INFO]  Processing dependency packages 'requirements.txt' to be installed with {}
[INFO]  Requested coverage for tasks: pybuilder.plugins.python.unittest_plugin:run_unit_tests
[INFO]  Running unit tests
[INFO]  Executing unit tests from Python modules in /home/flynnto/self_study/lava/tests/lava
------------------------------------------------------------
BUILD FAILED - SyntaxError: future feature annotations is not defined (lava/magma/core/process/process.py:9)
------------------------------------------------------------
Build finished at 2021-11-30 17:47:54
Build took 10 seconds (10169 ms)

Attempted Installation Using Python3.7

Fails due to f-string formatting

Steps

cd lava
virtualenv --python=/usr/bin/python3.7 ~/env_lava_37
source ~/env_lava_37/bin/activate
pip install -U pip
pip install -r build-requirements.txt
pip install -r requirements.txt
pip install .
export PYTHONPATH=$(pwd)/src
pyb -E unit

Output

PyBuilder version 0.13.3
Build started at 2021-11-30 17:50:00
------------------------------------------------------------
[INFO]  Installing or updating plugin "pypi:pybuilder_bandit, module name 'pybuilder_bandit'"
[INFO]  Processing plugin packages 'pybuilder_bandit' to be installed with {}
[INFO]  Activated environments: unit
[INFO]  Building lava-nc version 0.2.0
[INFO]  Executing build in /home/flynnto/self_study/lava
[INFO]  Going to execute tasks: analyze, publish
[INFO]  Processing plugin packages 'coverage~=5.2' to be installed with {'upgrade': True}
[INFO]  Processing plugin packages 'flake8~=3.7' to be installed with {'upgrade': True}
[INFO]  Processing plugin packages 'pypandoc~=1.4' to be installed with {'upgrade': True}
[INFO]  Processing plugin packages 'setuptools>=38.6.0' to be installed with {'upgrade': True}
[INFO]  Processing plugin packages 'sphinx_rtd_theme' to be installed with {}
[INFO]  Processing plugin packages 'sphinx_tabs' to be installed with {}
[INFO]  Processing plugin packages 'twine>=1.15.0' to be installed with {'upgrade': True}
[INFO]  Processing plugin packages 'unittest-xml-reporting~=3.0.4' to be installed with {'upgrade': True}
[INFO]  Processing plugin packages 'wheel>=0.34.0' to be installed with {'upgrade': True}
[INFO]  Creating target 'build' VEnv in '/home/flynnto/self_study/lava/target/venv/build/cpython-3.7.12.final.0'
[INFO]  Processing dependency packages 'requirements.txt' to be installed with {}
[INFO]  Creating target 'test' VEnv in '/home/flynnto/self_study/lava/target/venv/test/cpython-3.7.12.final.0'
[INFO]  Processing dependency packages 'requirements.txt' to be installed with {}
[INFO]  Requested coverage for tasks: pybuilder.plugins.python.unittest_plugin:run_unit_tests
[INFO]  Running unit tests
[INFO]  Executing unit tests from Python modules in /home/flynnto/self_study/lava/tests/lava
------------------------------------------------------------
BUILD FAILED - SyntaxError: invalid syntax (<fstring>, line 1) (lava/magma/runtime/runtime.py:10)
------------------------------------------------------------
Build finished at 2021-11-30 17:50:08
Build took 8 seconds (8513 ms)

Attempted Installation Using Python3.8

Stuck for >1 hr

Steps

cd lava
virtualenv --python=/usr/bin/python3.8 ~/env_lava_38
source ~/env_lava_38/bin/activate
pip install -U pip
pip install -r build-requirements.txt
pip install -r requirements.txt
pip install .
export PYTHONPATH=$(pwd)/src
pyb -E unit

Output

PyBuilder version 0.13.3
Build started at 2021-11-30 17:52:57
------------------------------------------------------------
[INFO]  Installing or updating plugin "pypi:pybuilder_bandit, module name 'pybuilder_bandit'"
[INFO]  Processing plugin packages 'pybuilder_bandit' to be installed with {}
[INFO]  Activated environments: unit
[INFO]  Building lava-nc version 0.2.0
[INFO]  Executing build in /home/flynnto/self_study/lava
[INFO]  Going to execute tasks: analyze, publish
[INFO]  Processing plugin packages 'coverage~=5.2' to be installed with {'upgrade': True}
[INFO]  Processing plugin packages 'flake8~=3.7' to be installed with {'upgrade': True}
[INFO]  Processing plugin packages 'pypandoc~=1.4' to be installed with {'upgrade': True}
[INFO]  Processing plugin packages 'setuptools>=38.6.0' to be installed with {'upgrade': True}
[INFO]  Processing plugin packages 'sphinx_rtd_theme' to be installed with {}
[INFO]  Processing plugin packages 'sphinx_tabs' to be installed with {}
[INFO]  Processing plugin packages 'twine>=1.15.0' to be installed with {'upgrade': True}
[INFO]  Processing plugin packages 'unittest-xml-reporting~=3.0.4' to be installed with {'upgrade': True}
[INFO]  Processing plugin packages 'wheel>=0.34.0' to be installed with {'upgrade': True}
[INFO]  Creating target 'build' VEnv in '/home/flynnto/self_study/lava/target/venv/build/cpython-3.8.10.final.0'
[INFO]  Processing dependency packages 'requirements.txt' to be installed with {}
[INFO]  Creating target 'test' VEnv in '/home/flynnto/self_study/lava/target/venv/test/cpython-3.8.10.final.0'
[INFO]  Processing dependency packages 'requirements.txt' to be installed with {}
[INFO]  Requested coverage for tasks: pybuilder.plugins.python.unittest_plugin:run_unit_tests
[INFO]  Running unit tests
[INFO]  Executing unit tests from Python modules in /home/flynnto/self_study/lava/tests/lava
Runtime not started yet.
[Loihi1SimCfg]: Using the first PyProcessModel PyLifModelFloat available for Process Process_144::LIF.
[Loihi1SimCfg]: Using the first PyProcessModel PyLifModelFloat available for Process Process_147::LIF.

Attempted Installation Using Python3.9

Stuck for >1 hr

Steps

cd lava
virtualenv --python=/usr/bin/python3.9 ~/env_lava_39
source ~/env_lava_39/bin/activate
pip install -U pip
pip install -r build-requirements.txt
pip install -r requirements.txt
pip install .
export PYTHONPATH=$(pwd)/src
pyb -E unit

Output

PyBuilder version 0.13.3
Build started at 2021-11-30 17:55:40
------------------------------------------------------------
[INFO]  Installing or updating plugin "pypi:pybuilder_bandit, module name 'pybuilder_bandit'"
[INFO]  Processing plugin packages 'pybuilder_bandit' to be installed with {}
[INFO]  Activated environments: unit
[INFO]  Building lava-nc version 0.2.0
[INFO]  Executing build in /home/flynnto/self_study/lava
[INFO]  Going to execute tasks: analyze, publish
[INFO]  Processing plugin packages 'coverage~=5.2' to be installed with {'upgrade': True}
[INFO]  Processing plugin packages 'flake8~=3.7' to be installed with {'upgrade': True}
[INFO]  Processing plugin packages 'pypandoc~=1.4' to be installed with {'upgrade': True}
[INFO]  Processing plugin packages 'setuptools>=38.6.0' to be installed with {'upgrade': True}
[INFO]  Processing plugin packages 'sphinx_rtd_theme' to be installed with {}
[INFO]  Processing plugin packages 'sphinx_tabs' to be installed with {}
[INFO]  Processing plugin packages 'twine>=1.15.0' to be installed with {'upgrade': True}
[INFO]  Processing plugin packages 'unittest-xml-reporting~=3.0.4' to be installed with {'upgrade': True}
[INFO]  Processing plugin packages 'wheel>=0.34.0' to be installed with {'upgrade': True}
[INFO]  Creating target 'build' VEnv in '/home/flynnto/self_study/lava/target/venv/build/cpython-3.9.5.final.0'
[INFO]  Processing dependency packages 'requirements.txt' to be installed with {}
[INFO]  Creating target 'test' VEnv in '/home/flynnto/self_study/lava/target/venv/test/cpython-3.9.5.final.0'
[INFO]  Processing dependency packages 'requirements.txt' to be installed with {}
[INFO]  Requested coverage for tasks: pybuilder.plugins.python.unittest_plugin:run_unit_tests
[INFO]  Running unit tests
[INFO]  Executing unit tests from Python modules in /home/flynnto/self_study/lava/tests/lava
Runtime not started yet.
[Loihi1SimCfg]: Using the first PyProcessModel PyLifModelFloat available for Process Process_144::LIF.
[Loihi1SimCfg]: Using the first PyProcessModel PyLifModelFloat available for Process Process_147::LIF.
harryliu-intel commented 2 years ago

The minimum Python version is 3.8, and we will document it more prominently per issue #91. I was able the replicate the hanging behavior and I think it has the same root cause as issue #71, with the same workaround: could you try to increase the limit on the number of file descriptors (ulimit -n 2048) before running the unit tests? That allowed "pyb -E unit" to complete successfully for me.

hthompson6 commented 2 years ago

Increasing the file descriptor limit resolved the issue. Thank you!