search

nengo / nengo-gui

Nengo interactive visualizer
Other
99 stars 38 forks source link

Adding plots can change seeds and thus model behaviour #855

Closed jgosmann closed 7 years ago

jgosmann commented 7 years ago

When adding a plot in Nengo GUI (or a slider?), additional objects like nodes can be inserted into the model. That might modify seeds in the model (if only a top-level seed is set) because seeds for the additional nodes might be generated inbetween other seeds for other objects, thus changing all seeds generated afterwards. This is especially a problem when trying to debug the behaviour of a single seed because adding and removing plots to investigate the behaviour can change the behaviour itself.

tcstewar commented 7 years ago

This is definitely a good idea, but I'm not sure how to do it. I'm pretty sure it'd require some changes to nengo itself. Perhaps this is something that could be kept in mind when we're re-organizing some of the builder process such that they could be used by multiple backends? In particular, the seed generation process really should be something that's shared across backends....

jgosmann commented 7 years ago

Maybe we could write a helper function that takes a model and fixes all seeds by setting them on the Nengo objects? Then Nengo GUI could call that function before adding anything to the model.

tcstewar commented 7 years ago

Interesting.... that should definitely work. I can't see any problems with that... Lemme see what I can put together... :)

jgosmann commented 7 years ago

I still think that this requires a bit of Nengo builder refactoring to extract the seed generation. This helper function should use the same algorithm to ensure that a model run in the GUI gets the same seeds as a model run outside of the GUI.

Potentially my workaround code can be helpful. But it goes through sim.model which I think is discouraged.

tcstewar commented 7 years ago

I still think that this requires a bit of Nengo builder refactoring to extract the seed generation. This helper function should use the same algorithm to ensure that a model run in the GUI gets the same seeds as a model run outside of the GUI.

I think you're right that in the long run, that's what we want. But since we don't have that guarantee right now, I think it'd still be useful to have the seeds fixed by nengo_gui so that adding and removing graphs don't affect anything. And that shouldn't require nengo builder refactoring.

jgosmann commented 7 years ago

Sure. :)