pass commands through ticks to get data out of distributed [sub]ensembles

[hudon]

problem: so in distribute-proto/subensembles, you cannot print an ensemble's data from the client by simply accessing its data members. This is because the ensembles are separate processes, and they don't share their data with the main process. We'd have to do some basic RPC or even just some more message-passing to allow clients to get data out of ensembles (since right now, data only flows from one ensemble to the next, and from ensemble to ticker).

why is this a problem:

• debugging: Right now, we need to actually put print statements in the ensemble code to see the data. It would be much smoother if we can compare the data with the original code by accessing it from the client (matrix_multiply).
• actual research use: if we want our shit to be at all useful for Terry et co., they need to be able to query the state of the network components without having to modify the libraries...

initial solution: right now, the "tick" messages from administrator to ensemble is empty. By this I mean that either party (client and ensemble) do not need to know about the content of the message; they just know that they received a "tick" and then they react to that alone. What we can do is use this "tick" as a rudimentary RPC mechanism. So when the administrator sends a "tick", he puts a message with it (e.g. { "print": "origin-value" }), and the ensemble not only reacts to the tick, but then takes any necessary action if content was found in the message (e.g. print the origin's value before replying that he's done).

[hudon]

some work on this has been done in f3ff836 , however it is not like the "initial solution". Instead, we just set a flag when we create the ensemble, and the ensemble process will check the flag and print if it's on... like so:

net.make_array('D',50,D1*D3,radius=radius, is_printing=True)

In the future, perhaps we can have an enum so the client can go: network.make_array('B', dimensions=....., print=ORIGIN_VALUE) or network.make_array('B', dimensions=......, print=ACCUMULATORS)

or something.... What do you all think? @gretac @artemip @RobertElder

[gretac]

yes, having a single is_printing flag is not enough since typically when debugging you might need to find out either the origin or the accumulator or the neurons. An enum could be good. The only problem would be that you cannot print more than one value at a time (so like if I want to see an input and then a transformed input at the same time). We could use a dictionary where the keys can be the possible things that can be printed and the value would be a boolean indicating whether you want that printed.

I was wondering, are we going to have 2 versions of the library then, debug and clean? I imagine that having a bunch of if statements throughout the code would decrease performance.

[hudon]

or a list of enums [ORIGIN, ACCUMULATORS, etc.] . Yea ideally there would be a macro or something that can decide whether or not the ifs will be there... I'll see if we can do some reflection/macro type stuff in python

[gretac]

yeah list of enums would be good as well, could be easier for the user to specify. Internally, I think it would be easier for us to use a diict to make the if checks though.

[gretac]

Collecting and printing origin data is no longer done through the is_printing flag. Instead probes are being used, ref #17. Probes are enabled for ensembles. Enabling probes for sub-ensembles is deferred to issue #36.

[gretac]

closed in pull request #32, commit ab6211d