Auto-detect precompute

[tbekolay]

Right now we have users pass in precompute=True or precompute=False to change how the model is organized under the hood. There's not much guidance as to why they would choose one vs the other, so it feels odd having this be a user choice.

Ideally, whether the model is to be precomputed or not should be automatically determined, to the point that it is not user configurable.

The next level of this would be to allow the user to specify a preference, but consider that a suggestion and switch when we have reason to believe that a model can be precomputed (or not).

The smallest change to the current situation that would be helpful would be to detect the situation that results in the Cannot precompute input, as it is dependent on output exception and automatically switch to precompute=False in that situation.

[tcstewar]

One complication that comes to mind for this is that, for some Nodes, we have no way of automatically detecting whether it's safe to precompute them. In particular, any Node that reads data from a sensor, or a UDPNode, or from the GUI, or anything like that. While this may seem like kinda a rare use case, it came up a lot with nengo_spinnaker, where we also initially tried for an automatic detection. This caused a lot of headaches and annoying bugs, and we eventually switched back to the manual approach (although there you explicitly marked a Node as being purely a function of time (and thus safe to precompute) by setting model.config[node].function_of_time = True).

I expect the same sort of thing to be true for a lot of the nengo_loihi use cases, for similar reasons, so I think we should keep it as a manually selected option. If we are finding that the cannot precompute thing comes up a lot and annoys people, then instead of defaulting to precompute=True, we should default to precompute=False, which always works for all cases (and was the original default when the precompute stuff was added).

[tbekolay]

I think nodes that can be precomputed are the rare cases, and in general would be pretty easy to whitelist and otherwise assume it can't be precomputed. So things like the processes in nengo/processes.py and a few other known ones we can whitelist, otherwise assume it can't be precomputed.

The annoying (and obviously very common) case are functions. I think that's the case where manually specifying through the config system makes sense, and assuming that they're not precomputable otherwise. But yeah, I just definitely see manually specifying precompute as a tricky thing to teach users, so we should try to help as much as possible.

[tcstewar]

I think nodes that can be precomputed are the rare cases, and in general would be pretty easy to whitelist and otherwise assume it can't be precomputed. So things like the processes in nengo/processes.py and a few other known ones we can whitelist, otherwise assume it can't be precomputed.

I completely agree that that's the assumption we should make (any Node that's an unknown function and is not on a whitelist should be assumed to not be precomputable). So are you envisioning an eventual system where there's a precompute=None, which is the default and the builder chooses, and it only chooses precompute=True if all of the Nodes are on a whitelist? If so, then I think the "smallest change to the current situation that would be helpful" would just be switching the default from precompute=True to precompute=False. Or is there a reason that we don't want to change that default back to what it was?

[tbekolay]

I'm envisioning what I said in the original post, that

1. ideally there's no precompute arg and we just figure it out; if that's not possible
2. allow the precompute arg as a suggestion that we can override (i.e., if they say don't precompute but we determine that we definitely without a shadow of a doubt can, we go ahead and precompute it); and if that's not possible then
3. do what we do now, but instead of raising an exception when we determine precompute=True won't work, we warn and switch it to precompute=False.

The main new thing from this thread for me is that instead of using a precompute argument to Simulator we could instead use the config system to mark nodes as being precomputable. So that would be a variant of 2 and 3 in the list above.

[tbekolay]

We had a brief video chat about this issue. We didn't reach a consensus about the best way to handle this long-term, as automatically detecting precomputable function is very difficult. The SpiNNaker way of marking individual nodes are precomputable is preferable to some as it's more explicit, but the precompute argument is preferable to others as it's easier to remember and faster to change once the model doesn't run. So, we'll have to think about this more and see if there are other more preferable options.

We did, however, come to a consensus about a short-term issue, which is switching to precompute=False as the default. Some reasons why we might do this are that it's better to simulate a model slowly than to not simulate it at all, and users should be familiar with turning on optimization flags to speed something up when it's slow, as precomputing nodes is a specific optimization. I'll make a PR to make this change and fix up any testing issues that crop up as a result.

[tbekolay]

Dear Github: "does not close #53" should not close #53

[arvoelke]

Just adding a discussion point here so that it's not lost: it may be interesting/useful to explore a hybrid combation of True and False, where the maximal subraph(s) that can be precomputed are automatically identified, and the rest is provided online. This might help for real-time applications where some portion of the input is expensive to compute (relative to IO) but can be pre-computed separately using Nengo objects. Considerations might include: how common are such use-cases, when do such use-cases benefit from precomputing some portion of the model, and the trade-offs associated with the overhead / per-byte costs of IO channels. On the last points, it would be good to have systematic profiling benchmarks in place to quantify the cost of real-time IO. This might also depend on multi-chip support, with SNIPS distributed across chips.