Cannot call run(time) several times

[felix-schneider]

I am integrating Spinnaker into our current system, where the simulation should be run 100ms at a time, then stopped and resumed. However, attempting to call spinnaker.pyNN.run twice (with the same time interval) results in the error

spinn_front_end_common.utilities.exceptions.ConfigurationException: cannot set the number of machine time steps of a given model once it has already been set

Is this purely a frontend problem? Or is there a technical reason why the simulation cannot be stopped and resumed?

[rowleya]

Hi,

You can run something a second time, but you have to call p.end() and p.setup() again i.e. you have to set the simulation up again. We are investigating the pausing and continuing of simulations, but I don’t have a definite date for implementation yet. The issues involved with this are in how we currently initialize the simulation.

Can I ask what data you are retrieving in the gap? If it is spike data, this can be sent live during the execution of the simulation.

Andrew :)

From: Felix Schneider [mailto:notifications@github.com] Sent: 25 September 2015 13:39 To: SpiNNakerManchester/sPyNNaker sPyNNaker@noreply.github.com Subject: [sPyNNaker] Cannot call run(time) several times (#144)

I am integrating Spinnaker into our current system, where the simulation should be run 100ms at a time, then stopped and resumed. However, attempting to call spinnaker.pyNN.run twice (with the same time interval) results in the error

spinn_front_end_common.utilities.exceptions.ConfigurationException: cannot set the number of machine time steps of a given model once it has already been set

Is this purely a frontend problem? Or is there a technical reason why the simulation cannot be stopped and resumed?

— Reply to this email directly or view it on GitHubhttps://github.com/SpiNNakerManchester/sPyNNaker/issues/144.

[felix-schneider]

Thank you for your answer.

I am aware that data can be fed into and retrieved during the simulation. However, SpiNNaker would be part of a larger simulation in this case. The input data might not become available in realtime (e.g. because of a slow physics simulation). There are also other circumstances where it would be useful to be able to stop and resume the simulation without resetting.

Can you estimate how complex the changes to the code would be? I'm considering forking the project and attempting the modification myself, but I've not looked too deep into the inner workings of Spinnaker PyNN and Pacman.

[alan-stokes]

Hi Scaatis,

One of the issues with stopping and resuming a simulation is to do with how we allocate memory for recording data (such as spikes, v and gsyn). During normal pynn operations, these are stored on chip memory, and need to be extracted at the end of simulation. This means we need to know in advance how much space we need to do these recordings. (there is some work that's ongoing that will remove this requirement, but it is estimated to not be complete till end of Oct). These memory requirements are built into how we break apart a population onto core sized chunks, and therefore rerunning a script for longer than was originally setup for will cause issues for us, as then cores will write over each others memory slots, or just crease to record data.

There is 2 more fundamental issues for completeness sake which we currently have not got around to rectifying, these are that:

1: time is modelled implicitly during run-time, so when you pause a simulation (which will not be done simultaneously throughout the machine (as this command would need to travel to each core) what do you do with the spikes that are flying though the system since the first core paused? are these lost? counted? processed? etc. This is more of a completeness issue, but needs to be considered and addressed before we can truly support this functionality.

2: the way the binaries are currently programmed, they run for a number of cycles, and then complete and go into a exit state. At this point, rerunning the binary requires us to reload it, and start it executing again. Any internal state at this point (aka voltages, membrane potential, input spikes etc) have all been lost. We could manipulate the binary to rewrite this state into sdram, and thus allow it to be reloaded, but this requires more memory calculations, and software support and could be a rather complicated process to get right (when does the core read this state data, over just its original setup params.

Overall, it is quite a bit of work to make it a stable piece of functionality, but your welcome to try to implement it yourself. I ask that you keep us updated on your progress as we will be interested in merging this functionality into the tool chain at large (you'll need to change spynnaker, spinnman, and frontendCommon). Alternatively, you could wait till we move to PyNN 0.8 where we need to do this functionality ourselves anyhow.

I hope this clears up the restrictions and the potential road blocks.

On a side note, if your system is not real time, and thus the other application is becoming a blocking process, you could exploit the notification protocol discussed in the live output functionality, to initiate a reset during the time the other application is generating the data, and not send the start command till your ready.

regards Alan

[AlexRast]

On 25/09/15 15:40, Alan Stokes wrote:

Hi Scaatis,

One of the issues with stopping and resuming a simulation is to do with how we allocate memory for recording data (such as spikes, v and gsyn). During normal pynn operations, these are stored on chip memory, and need to be extracted at the end of simulation. This means we need to know in advance how much space we need to do these recordings. (there is some work that's ongoing that will remove this requirement, but it is estimated to not be complete till end of Oct). These memory requirements are built into how we break apart a population onto core sized chunks, and therefore rerunning a script for longer than was originally setup for will cause issues for us, as then cores will write over each others memory slots, or just crease to record data.

There is 2 more fundamental issues for completeness sake which we currently have not got around to rectifying, these are that:

1: time is modelled implicitly during run-time, so when you pause a simulation (which will not be done simultaneously throughout the machine (as this command would need to travel to each core) what do you do with the spikes that are flying though the system since the first core paused? are these lost? counted? processed? etc. This is more of a completeness issue, but needs to be considered and addressed before we can truly support this functionality.

It occurs to me that one simple way we could support this is simply to disable the Timer interrupt as soon as the core receives a "pause" message. That stops the mechanism that drives the simulation forward without disabling any of the spike processing, interrupt servicing etc. The simulation won't advance and so is effectively paused without having to do anything special with reload or packet processing. Eventually, assuming the pause duration is sufficient that all cores will receive the message long before any restart, the traffic will die of its own accord, because the cores won't generate new spikes since they're not advancing.

Restart then simply reenables the Timer interrupt. Everyone's received whatever spikes they're going to have received in the intervening time so they just resume from the point they left off at.

One would have to be aware that in this scenario there is no guarantee that cores will stop on the same timer tick (i.e. the same millisecond in the "typical" scenario) so there is some chance spikes will arrive in the "wrong" millisecond - because the issuing core issued a spike at a time when the receiver had already stopped, but had the receiver been running, it would have already advances to the next Timer tick. But you can't prevent that anyway, even when the simulation is running perfectly normally (the best you can do is mitigate the possibility of it happening)

This is also a solution very much for the neural simulation case and assuming the standard execution model. It's not a general-purpose solution - not for simulations that aren't using a timer-driven update of state variables.

In any case, the implementation of this requires only 2 new (SDP) command messages: pause and resume. Pause could take if desired a tick value at which to pause, so that there would be a form of quasi-barrier-synchronisation.

2: the way the binaries are currently programmed, they run for a number of cycles, and then complete and go into a exit state. At this point, rerunning the binary requires us to reload it, and start it executing again. Any internal state at this point (aka voltages, membrane potential, input spikes etc) have all been lost. We could manipulate the binary to rewrite this state into sdram, and thus allow it to be reloaded, but this requires more memory calculations, and software support and could be a rather complicated process to get right (when does the core read this state data, over just its original setup params.

Overall, it is quite a bit of work to make it a stable piece of functionality, but your welcome to try to implement it yourself. I ask that you keep us updated on your progress as we will be interested in merging this functionality into the tool chain at large (you'll need to change spynnaker, spinnman, and frontendCommon). Alternatively, you could wait till we move to PyNN 0.8 where we need to do this functionality ourselves anyhow.

I hope this clears up the restrictions and the potential road blocks.

On a side note, if your system is not real time, and thus the other application is becoming a blocking process, you could exploit the notification protocol discussed in the live output functionality, to initiate a reset during the time the other application is generating the data, and not send the start command till your ready.

regards Alan

— Reply to this email directly or view it on GitHub https://github.com/SpiNNakerManchester/sPyNNaker/issues/144#issuecomment-143240926.

[rowleya]

Hi,

We are always happy for contributions.

An idea for an implementation is along the lines Alex is suggesting:

1) When the time equals the simulation time, call spin1_callback_off to turn the timer interrupt off and then call event_wait() (from sark.h). After this call, returns, reset the recorded data and the time to 0, and turn the interrupt back on. The event_wait sets the cpu into SYNC_1 or SYNC_0 alternately each time it is called (it is called once at startup, so the first time you call it, it will be SYNC_1), and doesn't return until a signal is sent from host.

2) Instead of waiting for EXIT for the end of simulation, wait for SYNC_1 and SYNC_0 alternately.

3) When the SYNC_1 or 0 is detected, read all recorded data.

4) When run is called, call SYNC_0 and SYNC_1 alternately, instead of just SYNC_0.

5) Only allow run to be called with the same time repeatedly. This would be a restriction of this simple implementation.

This should get you what you want, although there may be things that I have missed!

Andrew :)

Sent from my android device.

-----Original Message----- From: AlexRast notifications@github.com To: SpiNNakerManchester/sPyNNaker sPyNNaker@noreply.github.com Cc: Andrew Rowley Andrew.Rowley@manchester.ac.uk Sent: Fri, 25 Sep 2015 16:14 Subject: Re: [sPyNNaker] Cannot call run(time) several times (#144)

On 25/09/15 15:40, Alan Stokes wrote:

Hi Scaatis,

One of the issues with stopping and resuming a simulation is to do with how we allocate memory for recording data (such as spikes, v and gsyn). During normal pynn operations, these are stored on chip memory, and need to be extracted at the end of simulation. This means we need to know in advance how much space we need to do these recordings. (there is some work that's ongoing that will remove this requirement, but it is estimated to not be complete till end of Oct). These memory requirements are built into how we break apart a population onto core sized chunks, and therefore rerunning a script for longer than was originally setup for will cause issues for us, as then cores will write over each others memory slots, or just crease to record data.

There is 2 more fundamental issues for completeness sake which we currently have not got around to rectifying, these are that:

1: time is modelled implicitly during run-time, so when you pause a simulation (which will not be done simultaneously throughout the machine (as this command would need to travel to each core) what do you do with the spikes that are flying though the system since the first core paused? are these lost? counted? processed? etc. This is more of a completeness issue, but needs to be considered and addressed before we can truly support this functionality.

It occurs to me that one simple way we could support this is simply to disable the Timer interrupt as soon as the core receives a "pause" message. That stops the mechanism that drives the simulation forward without disabling any of the spike processing, interrupt servicing etc. The simulation won't advance and so is effectively paused without having to do anything special with reload or packet processing. Eventually, assuming the pause duration is sufficient that all cores will receive the message long before any restart, the traffic will die of its own accord, because the cores won't generate new spikes since they're not advancing.

Restart then simply reenables the Timer interrupt. Everyone's received whatever spikes they're going to have received in the intervening time so they just resume from the point they left off at.

One would have to be aware that in this scenario there is no guarantee that cores will stop on the same timer tick (i.e. the same millisecond in the "typical" scenario) so there is some chance spikes will arrive in the "wrong" millisecond - because the issuing core issued a spike at a time when the receiver had already stopped, but had the receiver been running, it would have already advances to the next Timer tick. But you can't prevent that anyway, even when the simulation is running perfectly normally (the best you can do is mitigate the possibility of it happening)

This is also a solution very much for the neural simulation case and assuming the standard execution model. It's not a general-purpose solution - not for simulations that aren't using a timer-driven update of state variables.

In any case, the implementation of this requires only 2 new (SDP) command messages: pause and resume. Pause could take if desired a tick value at which to pause, so that there would be a form of quasi-barrier-synchronisation.

2: the way the binaries are currently programmed, they run for a number of cycles, and then complete and go into a exit state. At this point, rerunning the binary requires us to reload it, and start it executing again. Any internal state at this point (aka voltages, membrane potential, input spikes etc) have all been lost. We could manipulate the binary to rewrite this state into sdram, and thus allow it to be reloaded, but this requires more memory calculations, and software support and could be a rather complicated process to get right (when does the core read this state data, over just its original setup params.

Overall, it is quite a bit of work to make it a stable piece of functionality, but your welcome to try to implement it yourself. I ask that you keep us updated on your progress as we will be interested in merging this functionality into the tool chain at large (you'll need to change spynnaker, spinnman, and frontendCommon). Alternatively, you could wait till we move to PyNN 0.8 where we need to do this functionality ourselves anyhow.

I hope this clears up the restrictions and the potential road blocks.

On a side note, if your system is not real time, and thus the other application is becoming a blocking process, you could exploit the notification protocol discussed in the live output functionality, to initiate a reset during the time the other application is generating the data, and not send the start command till your ready.

regards Alan

— Reply to this email directly or view it on GitHub https://github.com/SpiNNakerManchester/sPyNNaker/issues/144#issuecomment-143240926.

— Reply to this email directly or view it on GitHubhttps://github.com/SpiNNakerManchester/sPyNNaker/issues/144#issuecomment-143249168.

[felix-schneider]

So I'm still wrapping my head around what actually happens when I call run(tstop) in the PyNN script. Here is what I understand so far, I would appreciate if you could tell me whether that is at all correct: sPyNNaker calls PACMAN, which "compiles" the neural net that I described with PyNN into an application for SpiNNaker and uploads it to the board. That application is then started on SpiNNaker. The application gets to spin1_start and then waits for the call to event_wait to return. When it receives the SYNC_0 signal, it starts the simulation. This is where I get somewhat confused.

• How does the application get to spin1_start? Where can I find its entry point?
• What exactly is sark? Is it part of SpiNNaker's "OS"? Or is it part of the application that gets uploaded to it?
• You said the program waits for EXIT to end the simulation but I can't seem to find where. Could you point me in the right direction?

[alan-stokes]

Hi Scaatis,

So I'm at the HBP summit hackerthon right now, and numerous people have brought this to my attention. Therefore I'm going to try to built it now. Ill keep you updated on how we go.

Its worth noting that you need to tie all the capabilities of pynn for after run (allowing multiple changes to graph and still running, and this needs then to go though mapping).

for spin1_start there's a simulation_run(); which calls simulation.c in front end common/c_common/front_end_common_lib/src/simulation.c line 49 to 51

for spin1_exit spynnaker/neural_mdoelling/src/neuron/c_main.c line 192.

sark and scamp are low level software and most of the c code is via the spin1 api, which means you shouldn’t need to worry about the sark stuff.

Anyhow, ill let you know by thursday on how we go. Itll be in a brnach called multi-run once ive finished.

regards Alan

[felix-schneider]

Hi Alan

I would be happy to assist you with developing this functionality in the Python frontend, though much of the C code is still quite obscure to me :P

Could you fill me in on what areas need work, and what you have already done? There are a lot of layers so SpiNNaker and I find it difficult to anticipate which parts of the software need changing as a result of this feature. For now, I've gone to work on SpyNNaker/pynn/spinnaker.py as it seems like the most high-level and easiest to understand part of the process.

[alan-stokes]

ok, so becuase it seems adding multiple people to a assignment is not doable. Im reocrding it here. This is currently been assigned to both myself, Andrew Rowley and Felix Schneider.

[felix-schneider]

The changes are looking very good. I think I'll leave a few nitpicks in the commit, but nothing major. I expect I'll be back working with SpiNNaker by next week, so I'll be able to give more detailed feedback then.

I do have one question: getSpikes in PynnPopulation always returns all spikes since the start of the simulation, so as I run more and more simulations, the performance of both this method and whatever method I use to filter only the spikes from the last run is going to get worse. Can you suggest a workaround?

[alan-stokes]

Hi Felix,

So having had a bit of a gander, pynn0.7 does not support anything that would help you out here. Interestingly pynn0.8 does supports what you want via the get_data('spikes' clear = true). So I think the anwser to this is to wait till we support pynn0.8 to rectify this issue.

[felix-schneider]

I'll manage somehow. Which branches of which repo should I use if I want to use/test these changes?

[felix-schneider]

I figured it out. Did you test the way to set spike times between runs? You previously suggested this pseudocode:

runtime=1000 total_runtime = 0 ssa = p.pop(spike_source_array, spike_times = [0]) p.run(runtime) total_runtime += runtime sa.tset(spikes_times=[1001]) p.run(runtime) total_runtime += runtime time = 1 sa.tset(spike_times=[total_runtime + time]) p.run(runtime)

1) What is the meaning of tset here? spike_times used to be a list of lists, one for each neuron, containing the spike times. Tset seems to duplicate this? Has something changed there? 2) I'm not sure why, but pop.get("spike_times")[0] = <list> did not result in me getting any spikes. Since spike times is a list, it should be mutable. Is the send buffer of SpikeSourceArray updated before every run? (This was my problem before) 3) While SpikeSourceArray sees if you're trying to set spike_times and recognizes that this does not require mapping again, the method call first passes through set or tset in Population, which always sets self._change_needs_mapping, regeardless of what value you set. So if you do change the spike times that way, you get an error message (and a crash, rather than run just running stop, reset, run)

Edit: here is some of what I did:

injector = Population(10, SpikeSourceArray, {"spike_times":[[]]*10}, label="injector") # spikes to be set later
injector.tset("spike_times", [spikes_for_first_neuron] + [[]] * 9) # does not result in spikes being fired, causes remapping if done after first run
injector.set("spike_times", [spikes_for_first_neuron] + [[]] * 9) # same
injector.get("spike_times")[0] = spikes_for_first_neuron # does not result in spikes
injector._vertex.set_value("spike_times", [spikes_for_first_neuron] + [[]] * 9) # does not result in spikes
injector._vertex.spike_times[0] = spikes_for_first_neuron # does not result in spikes
injector._vertex.spike_times = [spikes_for_first_neuron] + [[]] * 9 # does not result in spikes

[rowleya]

To answer your questions:

1) tset sets a new set of spikes for the SpikeSourceArray to play the next time run is called. You can also use "set" in sPyNNaker, as these calls our identical, but PyNN 0.7 says that you can use tset to set an array of arrays. In our implementation, you can either pass a single array (in which case all neurons send the same spikes) or an array of arrays (one set of spikes per neuron).

2) In the branch currently, there is an issue with the call to "reset". If you are using reset, the SpikeSourceArray doesn't get updated correctly. If you just call run setting the spike times in between, it should work as it is. This reset bug is being investigated.

3) This sounds like a bug. We did have this working, but like reset above, it might have been broken again.

Hope this helps...

[felix-schneider]

I still don't understand the difference between set and tset. tset just calls set right now.

However, tset also always sets change_requires_mapping, even if set does not (because the set parameter was spike_times). This seems like a bug to me.

I am not using Spinnaker.reset().

[apdavison]

In PyNN 0.7, set is supposed to accept only "scalar" values*, and apply the value to all neurons in the population. tset is supposed to accept vector values of the size of the population, and map the values in the vector to the population. In PyNN 0.8, set accepts either scalar or vector values, and does the right thing depending on the argument type. So in a way, the SpyNNaker implementation is anticipating PyNN 0.8, even if the behaviour is incorrect for 0.7.

(* I put "scalar" in quotes, because for SpikeSourceArray the parameter value for an individual neuron is a list/array, while the "vector" value is a list of lists.)

[felix-schneider]

I have investigated some more. There does not appear to be a problem with reset (or rather, reset_for_run). It is called and the SpikeSourceArray does generate its SendBuffers, apparently with the correct spike times. But the spikes are not executed.

On a different note: I found the code in spinnaker.py much harder to follow this time around, mostly because the various calls to pacman always say do_mapping or some such, where it is actually doing much more than that. Perhaps these functions should be renamed.

[felix-schneider]

More investigation, I think I may know the source of the problem: My simulation works like this: I simulate 10ms at 0.1ms timestep. My spike source fires for the first time at 18.8ms, then regularly every 24ms, so it does not fire a spike for every run of the simulation, most importantly, it does not fire for the first run. The send_buffer generated for the first run therefor is empty. Because of this, the size of the memory segment allocated for spike times is 0. I guess no spikes are written to SpiNNaker memory even when there are some to send.

Curiously, this does not appear to be the only problem. I increased the simulation time to 100ms (at 1ms timestep) and got spikes. But even though I filled the SpikeSourceArray with spikes for ~2400ms, I only got spikes on the first and second run of the simulation. I suspect the Buffer problem here as well. After ~2 runs SpiNNaker runs out of buffer space.

[alan-stokes]

Howdi all,

so just got setup in Baltimore now, and am reading the notifications and got caught up.

Ill leave Felix's comment 1 off as Andrew Davidson explained it.

2. pop.get("spike_times")[0] = will return whatever array you put in, so i'd recommend you use debug to see what your actually getting / putting into the SSA
3. This one has perplexed me, I've just ran my test suite which does this for a SSA (namely https://github.com/SpiNNakerManchester/PyNNExamples/blob/work_flow_multi_run/examples/multi_call_examples/synfire_2_run_no_extraction__spike_array_changed_if_curr_exp.py )

This runs without errors and proves that adding spike times to a SSA works with the current branch. So I ask that you look at that example and see if there’s something going wrong, as currently I cannot see any issues (but i am now feeling rather sleepy).

To note about the population requires_mapping_changes and how we detect that the change from the SSA doesn’t require mapping is due to the two prong approach. The pops state if something has changed, the vertex is then queried to see if the change actually requires mapping, this allowed us to state that spike times did not require changing, but say setting it to record did. (Rowley, I added the record check this afternoon and will push up momentarily).

On the code in spinnaker.py being harder to follow. This is the one really annoying thing with the new system, I couldn’t figure a clean and scalable way to deal with this different behaviours for pause, resume, reset etc. Your welcome to try to clean it up, what id settled on is what i believe to be suboptimal, but just getting it to work was more my target than perfect code at the moment. agile to the win.

You have landed on what is very much a issue, maybe we should add a min_size_buffer param to the ssa to allow this to work successfully, i vaguely remember a discussion between myself and rowley on this issue, but cant remember any decision.

Tomorrow ill look at it when ive got a few mins. Alan

[alan-stokes]

Just a update. The tests I've built and ran seem to show that reset, rerun multiple times, and different sizes of spike times over multiple runs all work now. So please update and try again. If you discover issues, please get back in touch.

regards Alan

[felix-schneider]

I tried it just now and I get:

Traceback (most recent call last):
  File "dummy_net_multi_run.py", line 109, in <module>
    fast_injector.set("spike_times", [fast_spikes] + [[]] * 9)
  File "/home/felix/Documents/SpiNNaker/sPyNNaker/spynnaker/pyNN/models/pynn_population.py", line 673, in set
    self._vertex.set_value(parameter, value)
  File "/home/felix/Documents/SpiNNaker/sPyNNaker/spynnaker/pyNN/models/common/simple_population_settable.py", line 28, in set_value
    setattr(self, key, value)
  File "/home/felix/Documents/SpiNNaker/sPyNNaker/spynnaker/pyNN/models/spike_source/spike_source_array.py", line 171, in spike_times
    self._check_buffer_sizes()
AttributeError: 'SpikeSourceArray' object has no attribute '_check_buffer_sizes

You added this function call in your most recent commit to work_flow_multi_run, but I can't see where you defined it.

[alan-stokes]

mmmm..... interesting....... i wonder if I've left out a commit somewhere, as they worked when I last left it.

I'm finally being able to get time to revisit this, so tomorrow ill try to figure out whats missing and try to commit it (or more likely, rebuild it [as I'm not at the machine I did that work on]) and update you when the fix is in place.

Alan

[alan-stokes]

Howdi,

so it was due to me losing a commit/push during the work. that line is actually completely incorrect and just needs to be deleted. I've done the change on this version and pushed up.

The reasoning I had had was that during sequent runs, I'd have to check that the memory usage wasn’t bigger than the memory region size i gave it. Then i had one of those "WTF am i thinking" moments, as too big never exists, as that’s what buffered in actually deals with, and too little issue was rectify with a min size allocation of a 1MG. This meant the check was obsolete and i deleted the method, pushed and then forgot i needed to delete the call, and caught it in the next test, fixed it, might have committed and must have forgotten to push.

but try pulling off now and if you get any issues, shout back.

[felix-schneider]

I can't confirm that, I get an error saying:

Traceback (most recent call last):
  File "dummy_net_multi_run.py", line 116, in <module>
    sim.run(time_to_run)
  File "/home/felix/Documents/spinnaker/sPyNNaker/spynnaker/pyNN/__init__.py", line 194, in run
    _spinnaker.run(run_time)
  File "/home/felix/Documents/spinnaker/sPyNNaker/spynnaker/pyNN/spinnaker.py", line 307, in run
    config.getboolean("Reports", "outputTimesForSections"))
  File "/home/felix/Documents/spinnaker/SpiNNFrontEndCommon/spinn_front_end_common/utilities/helpful_functions.py", line 266, in do_mapping
    pacman_executor.execute_mapping()
  File "/home/felix/Documents/spinnaker/PACMAN/pacman/operations/pacman_algorithm_executor.py", line 264, in execute_mapping
    self._handle_internal_algorithm(algorithm)
  File "/home/felix/Documents/spinnaker/PACMAN/pacman/operations/pacman_algorithm_executor.py", line 275, in _handle_internal_algorithm
    python_algorithm = self._create_python_object(algorithm)
  File "/home/felix/Documents/spinnaker/PACMAN/pacman/operations/pacman_algorithm_executor.py", line 422, in _create_python_object
    importlib.import_module(algorithm.python_module_import),
  File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
  File "/home/felix/Documents/spinnaker/SpiNNFrontEndCommon/spinn_front_end_common/interface/interface_functions/front_end_common_machine_interfacer.py", line 2, in <module>
    from spinnman.connections.socket_address_with_chip import SocketAddressWithChip
ImportError: No module named socket_address_with_chip

This error appears before even running once. Please confirm what branches you are working on, here are mine: DataSpecification=master PACMAN=work_flow_multi_run spinnaker_tools=master spinn_common=master SpiNNFrontEndCommon=work_flow_multiple_runs SpiNNMachine=master SpiNNMan=work_flow_multi_runs sPyNNaker=work_flow_multi_run

P.S.: Have you considered putting the SpiNNaker software into a single repository? It would remove an irritation like this one, get rid of a variety of inconsistent naming schemes. I had to walk my supervisor through setting up the software and cloning nine repositories and choosing the correct branch for each one really cost more time than it should have.

[felix-schneider]

Ah. My mistake, it was a fresh install (different workstation) and I didn't run setup.py for SpiNNMan.

The error from before does indeed seem to be fixed, I am getting spikes throughout the whole simulation.

However, changing the rate of a SpikeSourcePoisson between runs currently triggers mapping. I changed it to inheriting from SimplePopulationSettable but that seems like a bit of a hack.

[rowleya]

I don’t think changing the rate of a poisson source was in the spec for this work. This is actually much harder to do, as it might even move the source between two data structures!

From: Felix Schneider [mailto:notifications@github.com] Sent: 04 December 2015 17:03 To: SpiNNakerManchester/sPyNNaker sPyNNaker@noreply.github.com Cc: Andrew Rowley Andrew.Rowley@manchester.ac.uk Subject: Re: [sPyNNaker] Cannot call run(time) several times (#144)

Ah. My mistake, it was a fresh install (different workstation) and I didn't run setup.py for SpiNNMan.

The error from before does indeed seem to be fixed, I am getting spikes throughout the whole simulation.

However, changing the rate of a SpikeSourcePoisson between runs currently triggers mapping. I changed it to inheriting from SimplePopulationSettable but that seems like a bit of a hack.

— Reply to this email directly or view it on GitHubhttps://github.com/SpiNNakerManchester/sPyNNaker/issues/144#issuecomment-162022489.

[alan-stokes]

Hi Felix,

So changing the spike rate of a possion is not supported by the changes to reset and resume. The only things that you can change and not set off mapping are as follows:

1. spikes times of spike source array
2. the runtime, (as long as its smaller than the initial run).

Changing a spike source poisson, as Rowley points out, needs more work to be done with it. But I do note that somewhere there is a possion code that supports changing its rate via a sdp packet (rowley, where did that stuff go in the end?)

On a side note, making it inherit from SimplePopulationSettable wont allow you to change the state and get away from mapping, it only supports setting params, but the change_requires_mapping should still get set to True when you change the spike rates, which should force it to go into mapping again.

ON the topic of merging all the repositories into one module, we decided against doing this long ago, as these modules are meant to be self contained bits of functionality, and therefore allows you to filter changes by this scope. I understand your pain, as making the branches and keeping track of them all can be a pain, but keeping everything in one module causes its own pains as well.

Your experience is, to a degree, because your using cutting edge stuff that's not even merged into master yet. If this is merged into master by the December freeze date, you would then find installing the tools to support yourself much easier. Hopefully the last bit of work I'm doing on the branch will get us to a state where its stable enough to be merged into master, which will make your life much simpler.

[alan-stokes]

given all this, i think that this issue is closable now? would you agree felix?

[felix-schneider]

I wouldn't say that the multi-run functionality is complete, but I suppose the specific problem in the title of this ticket is resolved, so maybe it makes more sense to open new tickets for the missing features and close this one.

[alan-stokes]

I concur. thank you felix, i look forward to the next bunch of issues to work with you on.