Generalised IO

[mundya]

When we don't specify a spinnaker_build function for a Node, the Builder currently looks through some flags before constructing either the connectivity required for SpiNNlink/UART IO or SDP/Ethernet IO. It would be nice if providing the "default" action for a Node were pluggable in the same way that we provide pluggable Connection building. I'm thinking then that when the Builder encounters a Node without a spinnaker_build function it passes the building of this Node off to some external function which may be one registered by a different module.

This different module/code/... can also provide code for interfacing with Nodes being computed on the host. This code needs to provide complete inputs from the network on SpiNNaker and accept input from Nodes for the network. The Simulator then cares just about this interface.

From the user side this could be as simple as:

# Design my model

import nengo_spinnaker
sim = nengo_spinnaker.Simulator(model)  # Will use SDP/Ethernet

# Design my model

import nengo_spinnaker
import nengo_spinnaker.io.spinnlink  # Or similar
sim = nengo_spinnaker.Simulator(model)  # Will use SpiNNlink/UART

# Design my model

import nengo_spinnaker
import nengo_spinnaker.io.spinn2pixiedust
sim = nengo_spinnaker.Simulator(model)  # Will use SpiNNaker2 Pixiedust IO

The Builder will need changing to have pluggable handles for building Nodes, x -> Node and Node -> x connections - but to an extent this is already done. The Simulator will need some interface for receiving/querying inputs from Nodes (these may be cached by the IO interface) and some interface for sending from Nodes.

The nengo_spinnaker.io.xyz modules can call a class method on the Builder contain decorated functions which register themselves as the default functions for constructing Nodes and connecting to/from Nodes. The SpiNNlink/UART module might look like:

from .. import builder  # or `from nengo_spinnaker import builder` if this were a separate package

@builder.default_node_builder
def build_spinnlink_node(builder, node):
    pass  # Shocking but true

@builder.default_node_in_builder
def build_spinnlink_node_in(builder, c):
    # ...

@builder.default_node_out_builder
def build_spinnlink_node_out(builder, c):
    # ...

This isn't too far from the kind of stuff we already do... We should "require" that any IO method defines all the required functions because that would mean that

import nengo_spinnaker
import nengo_spinnaker.io.entangled
import nengo_spinnaker.io.spinnlink
sim = nengo_spinnaker.Simulator(model)  # Will use SpiNNlink/UART

cleanly uses SpiNNlink/UART rather than entangled pairs or some mix-match of both.

For the Simulator, IO modules need to provide a class which can be instantiated to generate key/Node/whatever mappings (all IO will likely need this), and provides functions for getting the input and setting the output of a Node. We may also need a start/stop function which sets up serial links/starts IO handling threads/... Continuing with the SpiNNlink IO module:

from .. import simulator

@simulator.node_handler
class SpiNNlinkNodeHandler(object):
    def __init__(self, builder):
        # Build node map
        # ...

    def __enter__(self):
        # ... Set up serial link, etc ...
        return self

    def __exit__(self):
        # ... Teardown serial link, join threads, etc ...

    def __getitem__(self, node):
        """Get the input value for a Node.
        Return None if the Node is valid but the input is not available,
        raise KeyError if the Node is unknown.
        """
        # ...

    def __setitem__(self, node, value):
        """Set the input value for a Node.
        Raise KeyError if the Node is unknown.
        """
        # ...

The Simulator can then do something like:

    def simulate(self, ...):
        with self.node_handler as nodes:
            while True:  # Maybe set this off as a Timer instead?
                for node in self.nodes:
                    in_val = []
                    if node.size_in > 0:
                        in_val = nodes[node]

                    t = ...  # Get time from somewhere

                    if in_val is not None:
                        out_val = []
                        if len(in_val) > 0:
                            out_val = node(t, in_val)
                        else:
                            out_val = node(t)

                        if len(out_val) > 0:  # Assuming Nodes always return an iterable
                            nodes[node] = out_val

This doesn't include anything for Node <-> Node connections, but that should be relatively trivial to do.

Is this over engineered? Thoughts?

[mundya]

(Note, apparently PySerial is thread safe: http://stackoverflow.com/questions/8796800/pyserial-possible-to-write-to-serial-port-from-thread-a-do-blocking-reads-fro)

[mundya]

Using __getitem__ and __setitem__ to get and set Node input/output seemed nice when I thought of it, but now I'm not so sure. It seems weird that

a = nodes[node]
nodes[node] = b

# a != b
nodes[node] == a

Maybe explicit get_node_input(node) and set_node_output(node, value) would be better?

[tcstewar]

Hmm... I'm a bit worried about this being an implicit parameter based on imports. That makes it difficult to do things like:

import nengo
import nengo_spinnaker
...
sim1 = nengo_spinnaker(model, use_serial=True)
sim2 = nengo_spinnaker(model, use_serial=False)

But the approach for the internals of the code seems pretty reasonable to me... my first instinct is still to implement the Ethernet in/out and then figure out what the refactored nice version would be, but we are pretty close to having the Ethernet version done, so I guess we've already got a pretty good sense of how it'll end up looking.

[mundya]

That makes it difficult to do things like:

import nengo
import nengo_spinnaker
...
sim1 = nengo_spinnaker(model, use_serial=True)
sim2 = nengo_spinnaker(model, use_serial=False)

An excellent point... how would something like:

sim1 = nengo_spinnaker.Simulator(model, io=nengo_spinnaker.io.Serial(...))
sim2 = nengo_spinnaker.Simulator(model, io=nengo_spinnaker.io.Pixiedust(...))

Suit you?

I'm torn between "there'll likely only ever be two IO forms, so we should/could just stick everything in and have booleans to enable/disable" and "we should factorise nicely"... I can see the pragmatism of the former, but would sincerely prefer the latter...

[mundya]

I made a start with 2104674395e853a55ef89197dc48dc04c1451eb6.