team-robot vision project improvements

nengo / nengo-loihi

Run Nengo models on Intel's Loihi chip

https://www.nengo.ai/nengo-loihi/

Other

35 stars 12 forks source link

team-robot vision project improvements #309

Closed hunse closed 2 years ago

hunse commented 3 years ago

This is a culmination of a number of improvements made for a team-robot vision project:

Speed ups for block splitting code
Properly pass dt for preset DecodeNeurons, which allows dt to be changed for them.
Save info from discretize_block so that users can make use of this if need be.
Speed up build process.
Reduce memory required for build, and use the nengo float_dtype to allow users to further reduce memory by setting this to e.g. float32 instead of the default float64.
Add GreedyComms and PartitionComms allocators, which both aim to reduce inter-chip communication.

Based off #300.

hunse commented 2 years ago

One thing to figure out is there is a PartitionComms test, but it won't run on TravisCI because I can't figure out how to install networkx-metis. It can be installed from source, e.g. pip install git+https://github.com/networkx/networkx-metis.git, but it requires Cython and for some reason won't install it itself. So we might have to customize the TravisCI script to install Cython first, and then we can add networkx-metis to setup.py.

tbekolay commented 2 years ago

I made several changes in fixup commits, so take a look at those @hunse and let me know if you dislike any of those changes. I also had a few questions, which are in the inline comments, that would be good to address before merging.

All else looks good though, there's about a 50% reduction in interchip activity in that test, which makes a huge difference, I imagine.

tbekolay commented 2 years ago

With the extra waiting for the client socket, everything passed! ... Except for the cleanup task, as we had a bunch of failed builds in there. I made 0f8ea10d1c4d0a832ed359e3386c3003fc1b5cef that does a few things:

Uses rsync -a --delete emptydir/ targetdir/ to delete, as it is apparently faster than rm -rf (source 1, source 2).
Uses tqdm to take the verbose output of that rsync command and print out progress over time. This means that even if deleting the old directories takes a long time, it won't stall the build. It may overwhelm the TravisCI log, since you can't control how often it prints out, so we'll see how that goes.

After we use this for a bit and it seems to work okay, we should copy this change to NengoBones.

I'll make those last few changes from @hunse's comments, then I think this is good to go :+1: