Open muffgaga opened 2 years ago
Hey Eric, is the aim to setup GeNN for building models in Python or C++?
I started simple and ignored the pygenn stuff :grimacing:… I can try to have a look at the Python library :).
It seems that it's a modified standard Python library install flow: I need to run some make-based thing first, only then setup.py
— and I'm also not 100% sure if I can just run install
instead of develop
.
Ok, so for C++ all you should need to do is:
CUDA_PATH
(potentially OPENCL_PATH
) environment variablesmake install PREFIX=XXXX
Then genn-buildmodel.sh
should work correctly and users can build C++ models. As you say the python install is indeed a slightly modified setuptools process. install
does work (give or take annoying bugs)
Finally, for now, on Linux, compiler requirement is GCC 4.9.1 (when they fixed std::regex
)
Thanks for the details — I just tried to follow http://genn-team.github.io/genn/documentation/4/html/d8/d99/Installation.html (the Linux part) and missed, e.g., swig ;))… now I'm iterating spack install genn
until it looks okay ;)
Yeah, the top-level make suggestion was based on my vague understanding of what you might want to be doing in a spack install
BTW. spack's cuda package provides CUDA_HOME
instead of CUDA_PATH
… So we probably need to workaround and have a variant
setting in genn → downside: the switch between cuda and non-cuda would have to happen at genn's installation time…
Ahh that is annoying - CUDA_PATH
is NVIDIA's standard I think. Which backend GeNN actually uses is controlled via genn-buildmodel
command line arguments so what is present when you build GeNN dictates what is available (the idea of the top-level install is that you only need read access to the install location). You can also just checkout GeNN and stick bin in the path and let GeNN build stuff when it's required (this is typically how I use GeNN)
I added:
variant('python', default=True, description='Enable PyGeNN')
extends('python', when='+python')
depends_on('swig', when='+python')
depends_on('python@3.8.0:', when='+python')
depends_on('py-numpy@1.17.0:', when='+python')
depends_on('py-six', when='+python')
depends_on('py-deprecated', when='+python')
depends_on('py-psutil', when='+python')
depends_on('py-importlib-metadata@1.0.0:', when='+python')
# in case of building with PyGeNN we need to build a C++ library
@when('+python')
def build(self, spec, prefix):
make('DYNAMIC=1', 'LIBRARY_DIRECTORY={}/pygenn/genn_wrapper/'.format(self.stage.source_path))
super(Genn, self).build(spec, prefix)
→ seems to build :).
You can also just checkout GeNN and stick bin in the path and let GeNN build stuff when it's required (this is typically how I use GeNN)
In my initial tests I just installed (copied via install_tree
) everything from the source directory to some target folder (spack defines a prefix folder for each package instance (there's a hash computed based on package version, dependencies and variant settings) — that might be a bit excessive. Do you have a suggestion here? Mmaybe skip userproject
and don't copy pygenn
but only use setup.py install
for this?
def install(self, spec, prefix):
# python setup.py install
super(Genn, self).install(spec, prefix)
# the non-python things
install_tree('bin', prefix.bin)
install_tree('include', prefix.include)
# TODO: what else?
@neworderofjamie Does the Python library require built-time selection of CUDA/non-CUDA or is it somehow still possible to switch at runtime? (Sorry, I only used GeNN at some hands-on sessions and basically forgot everything :p…)
So PyGeNN has an inbuilt priority list (CUDA->CPU->OpenCL as OpenCL is still a bit experimental) which, by default, it uses to select from amongst the backends it was build with but this can be override with the backend
kwarg to the GeNNModel
constructor
userproject
is a bit of a mess as it also includes some helper header files you can use in your own C++ simulation loops so, honestly, copying the whole tree is probably the simplest and best idea
So PyGeNN has an inbuilt priority list (CUDA->CPU->OpenCL as OpenCL is still a bit experimental) which, by default, it uses to select from amongst the backends it was build with but this can be override with the
backend
kwarg to theGeNNModel
constructor
Nice! Is this is still possible if we didn't have CUDA at build-time of the dynamic library?
userproject
is a bit of a mess as it also includes some helper header files you can use in your own C++ simulation loops so, honestly, copying the whole tree is probably the simplest and best idea
I now installed like this:
def install(self, spec, prefix):
super(Genn, self).install(spec, prefix) # python setup.py install
install_tree('bin', prefix.bin)
install_tree('include', prefix.include)
and after replacing the relative-include I can build some stuff in userproject
:
diff --git a/userproject/include/generateRun.h b/userproject/include/generateRun.h
index 84fd3525a..14994563c 100644
--- a/userproject/include/generateRun.h
+++ b/userproject/include/generateRun.h
@@ -17,7 +17,7 @@
#endif
// CLI11 includes
-#include "../../include/genn/third_party/CLI11.hpp"
+#include "genn/third_party/CLI11.hpp"
//------------------------------------------------------------------------
// GenerateRunBase
(I believe providing include paths to the compiler via CPLUS_INCLUDE_PATH
is reasonable ;).)
However, generator
seems to be missing for genn-buildmodel.sh
, so I tried
mkdirp(prefix.src.genn)
install_tree('src/genn/generator', prefix.src.genn)
However,
generator
wasn't build in my build — did I miss something?Hmm, I didn't add
Seems I need make('PREFIX={}'.format(prefix), 'install')
yet… so maybe this one :)… testing.src/genn
completely… testing.
Thx
This seems to work:
from spack import *
class Genn(PythonPackage):
"""GeNN is a GPU-enhanced Neuronal Network simulation environment based on
code generation for Nvidia CUDA."""
homepage = "https://genn-team.github.io/genn/"
url = "https://github.com/genn-team/genn/archive/refs/tags/4.6.0.tar.gz"
version('4.6.0', sha256='5e5ca94fd3a56b5b963a4911ea1b2130df6fa7dcdde3b025bd8cb85d4c2d3236')
depends_on('gmake')
conflicts('%gcc@:4.9.3')
# TODO: maybe build-time select of cuda?
variant('python', default=True, description='Enable PyGeNN')
extends('python', when='+python')
depends_on('swig', when='+python')
depends_on('python@3.8.0:', when='+python')
depends_on('py-numpy@1.17.0:', when='+python')
depends_on('py-six', when='+python')
depends_on('py-deprecated', when='+python')
depends_on('py-psutil', when='+python')
depends_on('py-importlib-metadata@1.0.0:', when='+python')
# in case of building with PyGeNN we need to build a C++ library
@when('+python')
def build(self, spec, prefix):
make('DYNAMIC=1', 'LIBRARY_DIRECTORY={}/pygenn/genn_wrapper/'.format(self.stage.source_path))
make('PREFIX={}'.format(prefix), 'install')
super(Genn, self).build(spec, prefix)
def install(self, spec, prefix):
super(Genn, self).install(spec, prefix)
install_tree('bin', prefix.bin)
install_tree('include', prefix.include)
mkdirp(prefix.src.genn)
install_tree('src/genn', prefix.src.genn)
def setup_run_environment(self, env):
env.append_path('CPLUS_INCLUDE_PATH', self.prefix.include)
Do you want to maintain your spack package file in-repo or should I just try to get this into EBRAINS (and upstream spack)?
I think including it in the GeNN repro makes sense - do you want to make a PR with it named whatever is standard? I guess it would work as is if CUDA_PATH
is already set, it just doesn't do anything smart if CUDA_HOME is set by spack?
I think including it in the GeNN repro makes sense - do you want to make a PR with it named whatever is standard? I guess it would work as is if
CUDA_PATH
is already set, it just doesn't do anything smart if CUDA_HOME is set by spack?
To enable a smooth cuda usage we could add:
# …
variant('cuda', default=True, description='Enable CUDA support')
depends_on('cuda', when='+cuda')
# …
def setup_run_environment(self, env):
# …
env.append_path('CUDA_PATH', self.spec['cuda'].prefix)
For arbor @schmitts integrated the spack build into the CI flow of arbor — here it's Jenkins… so probably not as easy for contributors to perform :)?
Haha, nothing is easy with Jenkins. Also, unlike @schmitts, I don't really understand how spack works so it's not 100% clear to me what the integration would achieve. All it does is copy some files right?
[Jenkins rant]
:p (and yes, we also use Jenkins a lot… and there are many problems, but we still don't see a reasonable alternative — at least for situations where you need on-prem. CI with access to local clusters or custom hardware).
I don't really understand how spack works so it's not 100% clear to me what the integration would achieve. All it does is copy some files right?
It also performs the make calls and and setup.py install
. A spack build CI job could detect problems on newer cuda or compiler versions — in spack this is merely a modifier for the install call:
$ spack install genn ^gcc@11.2.0 ^cuda@11.4.2
# will install using gcc@11.2.0 and cuda@11.4.2
and you could still test-install in a different variant using an older compiler:
$ spack install genn~cuda ^gcc@10.3.0
# will install
Additionaly, spack also supports testing… so we could also run tests on this build. So it's just a structured way of building (and testing) software in multiple variants/versions/settings.
Unless someone suddenly decides to donate us lots of cloud-time to run workers, we're in the same situation.
That is interesting....Testing against more compiler and CUDA variants is something we want to integrate into our testing system but currently my plan had not got further than using NVIDIA Container Toolkit and docker. Isn't installing gcc from source going to be horribly slow though?
Building gcc takes some time, yes — but spack supports build caches… so it's just initial "cost" until the cache has been filled.
Worst case is probably what we do for BrainScaleS: We use spack to provide a containerized software environment comprising ~900 software packages (everything from gcc, to neovim incl. typical simulators and software development tools — ok, excluding texlive because we want to stay below 10GiB for the image file :p) and that typically takes 24h on a 16-core / 64GiB machine to build from scratch.
Created PR #486… close this one?
Hey *,
during EBRAINS CodeJam 12 I had some time to work on getting GeNN into the EBRAINS software environment. We use
spack
as a build-from-source package manager. (This was triggered via @schmitts and @bcumming.)Here's my first shot at a package file for GeNN:
This basically just mentions the latest release and its dependency on make (
Makefile
s are generated by the genn create project thingies) (but ignores the dependency on a recent C++ compiler and the optional dependency on cuda). It installs everything from the release tar-ball and providesCPLUS_INCLUDE_PATH
to the user (e.g. viaspack load genn
or when using spack-generated module files; when another spack package depends ongenn
we should probably also add adef setup_build_environment(…)
).Now one question regarding GeNN's headers: I saw that you use relative paths in the
#include
statements for the GeNN headers in the examples. Is there a reason for not settingCPLUS_INCLUDE_PATH
? You do mention extendingPATH
to include GeNN'sbin/
folder though…Cheers, Eric