This is a horribly huge PR, sadly everything kind of needed to happen at once; I will rewrite history before merging. :japanese_ogre:
New toolchain
Todo:
[ ] Modify the build process to call the new place/partition and route tools
[ ] Modify the simulator to use the new IO to load all data and executables to the SpiNNaker machine (mostly just iterate and call), and control the simulation.
[ ] Merge in @mossblaser's work on keyspaces
New Vertex model
A new vertex model is built for this purpose, the following objects require new vertices constructing:
[x] LIF Ensembles (needs testing)
[x] Filter vertices (filter)
[ ] Function of time nodes (value_source)
[ ] Probes (value_sink)
[x] Ethernet TX/RX elements (sdp_tx and sdp_rx)
The Connection "Tree"
A big part of this rewrite is some changes toward formalising the on-host model of connections between Nengo objects. The intent is to reduce the transmission of "duplicated" packets on the SpiNNaker machine as this incurs higher than necessary memory and network utilisation. This reduction is achieved through combining compatible connections. Compatible connections originate from the same object, live in the same keyspace and transmit the same data (that is, have the same transform and function*).
For example, the code below will result in the "tree":
The code in connections/connection_tree.py represents the high-level class for building up trees of connections. Correct aliasing of connection types is based on breaking each Connection object into a ReducedOutgoingConnection and a ReducedIncomingConnection; each type stores the minimum of information required when transmitting or receiving packets respectively.
The connection "tree" is generated by taking each connection in turn, breaking it into an outgoing and an incoming form, and indexing these such that: originating object -> [outgoing connection -> [incoming connections]]. Aliasing connections relies on correct definition of the __hash__ and __eq__ methods.
*Function equivalence is tricky. For connections from ensembles we can check that functions are equivalent by checking they have the same id, or by evaluating them at all evaluation points and checking for the equivalence of the outputs. For connections from other objects we can only check id (or the bytecode :skull:). Some interesting results follow:
with nengo.Network(label='Connection Sharing') as model:
a = nengo.Ensemble(100, 1)
b = nengo.Node(lambda t: t**2, size_in=0, size_out=1)
c = nengo.Ensemble(100, 1)
d = nengo.Ensemble(100, 1)
# These two connections will be considered equivalent (shared)
conn1 = nengo.Connection(a, c, function=lambda x: x**2)
conn2 = nengo.Connection(a, d, function=lambda x: x**2)
# Whereas these will not
conn3 = nengo.Connection(b, c, function=lambda x: x**2)
conn4 = nengo.Connection(b, d, function=lambda x: x**2)
# But these will
squared = lambda x: x**2
conn5 = nengo.Connection(b, c, function=squared)
conn6 = nengo.Connection(b, d, function=squared)
Upstream changes
I've been trying to keep up with changes in Nengo, thorough testing will be necessary... I'm a little concerned about learning rules.
Other TODO
[ ] Write proper tests for the IO builders, probably refactor
[ ] Ethernet
[ ] UART (anyone plan on using this?)
[ ] Write better tests for the simulator
For a different PR
Correct behaviour of filters in C. Each filter routing entry should allow a single packet to be included in the input to multiple filters (or even the same one multiple times).
Provide some mapping between the dimension number from the key of a packet to the dimension it should be included with; this will allow greater sharing of connections at the cost of some on-chip compute time and memory.
Add "functional" tests that run through a model and compare to expected results.
This is a horribly huge PR, sadly everything kind of needed to happen at once; I will rewrite history before merging. :japanese_ogre:
New toolchain
Todo:
New Vertex model
A new vertex model is built for this purpose, the following objects require new vertices constructing:
filter)value_source)value_sink)sdp_txandsdp_rx)The Connection "Tree"
A big part of this rewrite is some changes toward formalising the on-host model of connections between Nengo objects. The intent is to reduce the transmission of "duplicated" packets on the SpiNNaker machine as this incurs higher than necessary memory and network utilisation. This reduction is achieved through combining compatible connections. Compatible connections originate from the same object, live in the same keyspace and transmit the same data (that is, have the same transform and function*).
For example, the code below will result in the "tree":
The code in
connections/connection_tree.pyrepresents the high-level class for building up trees of connections. Correct aliasing of connection types is based on breaking eachConnectionobject into aReducedOutgoingConnectionand aReducedIncomingConnection; each type stores the minimum of information required when transmitting or receiving packets respectively.The connection "tree" is generated by taking each connection in turn, breaking it into an outgoing and an incoming form, and indexing these such that:
originating object -> [outgoing connection -> [incoming connections]]. Aliasing connections relies on correct definition of the__hash__and__eq__methods.*Function equivalence is tricky. For connections from ensembles we can check that functions are equivalent by checking they have the same
id, or by evaluating them at all evaluation points and checking for the equivalence of the outputs. For connections from other objects we can only checkid(or the bytecode :skull:). Some interesting results follow:Upstream changes
I've been trying to keep up with changes in Nengo, thorough testing will be necessary... I'm a little concerned about learning rules.
Other TODO
For a different PR