Use only unused cores

[mossblaser]

Introduction

In the beginning there was read and write and it was good. Except when multiple things wanted to share memory. Then it was very, very bad. And so on the 1.33rd day, S-t created SDRAM alloc and life was good. Multiple applications and even the monitor could share the chip-wide memory resources, there was much rejoicing about entering the 21st century and all was well.

...but all was not well in the world. All this time ReserveResourceConstraints were all that kept Rig applications from trampling the monitor; and nothing at all protected the poor, innocent applications already running on the machine!

Ahem... Anyway... It seems odd that we're so careful to allocate SDRAM in some sane way and yet do not take the same approach to application cores. As well as being a little unsightly on a purely perfectionist perspective, here are a few examples of where this behaviour makes life hugely unpleasant:

• If the on-chip monitor process ever grows to multiple cores, every Rig application will need patching -- yuck! (This is a real possibility that ST/Luis are considering for the future.)
• If you want packet reinjection you have to integrate it into your application. In a sane world you'd start the packet reinjector on every core (using ybug, some standalone script, or whatever) and then just run your application as normal. Doing that right now will result in an unreported massacre of random instances of the packet-reinjector.
• If you want to do some detailed logging of network resources (e.g. to spot burstiness) the current best solution is to hack your application to reserve core 1 and then hope your logging tool gets loaded onto core 1... If anything goes wrong the result is a diagnostics-free mess.
• Insert many other instrumentation/debug-like scenarios here...
• Seriously, why does every application developer need to specify -- by hand -- that core 0 should not be used?!

Either way, the current way of doing things is a bit short-sighted and I think fixing this warrants proper consideration. This issue describes a suggested fix for this.

Proposed fix

I've been thinking about this for a few days now and so far the nicest solution to this seems to be to introduce a get_resource_constraints() method to MachineController (maybe needs better name...) which produces a whole load of ReserveResourceConstraints which rope-off any cores which aren't idle. ST would be amenable to adding a bit-mask of idle cores in SC&MP to make this process more efficient than reading the process table.

While we're at it we should really try and do something similar for SDRAM availability (e.g. if an instrumentation app has used 32 MByte of SDRAM per chip you probably can't afford to ignore this when placing memory-bound applications). Obviously this will require some changes to SC&MP, most likely with assistance or at least guidance from ST.

Pros of this method

• Minimal, fully-backward compatible API changes (add get_core_constraints method somewhere)
• Minimal changes required to existing applications (call the aforementioned method and stuff the result into the place-and-route tool chain, remove any manual ReserveResourceConstraints for skipping the monitor).
• Makes use of existing P&R primitives in (pretty-much) the way they were intended to be used.
• Removes the "oh hey, don't forget to reserve the monitor yourself in every application, despite the fact that every application will have to do this..." thing.

  Cons of this method

• Requires an additional SCP query sent to every chip to get the set of available cores.
• In the event that non-contiguous ranges of cores being in use you get a packing problem during placement/allocation and nobody wants that. With this said, with the use-cases outlined in the introduction this would never occur (e.g. you'd always start the instrumentation/debug apps first and quit them afterwards).
• The P&R wrapper should really have its "reserve_monitor" option removed (a very minor breaking change!)

  Proposed implementation plan

A preemptive implementation of my proposal has been started in the only-use-idle-cores branch.

• [x] Implement proof-of-concept get_idle_cores() method for MachineController
• [x] Implement get_resource_constraints() method for MachineController using the above.
• [x] Sanity check!
• [ ] Add another patch for SC&MP which adds cheap-to-read idle-cores info and free SDRAM indicator. get_idle_cores() should be updated accordingly; get_free_sdram() should also be introduced and this should be used by get_resource_constraints to reserve any SDRAM.
• Following release, submit PRs with code/docs changes to all known applications which would benefit from this fix:
  • [ ] nengo_spinnaker
  • [ ] pinus_rigida
  • [ ] network_tester
  • [ ] rig_examples
• Possibly package up the following as their own stand-alone tool (i.e. in project-rig but not as part of rig)
  • [ ] The network burstiness analysis script built on Network Tester I wrote some time ago
  • [ ] The packet-reinjector

    Alternative ideas

While thinking about all this I came up with the following rather yuck ideas. I think they're all significantly inferior to the proposed solution and are here mostly to allow people to laugh at me...

1. get_machine's "Cores" vertex resource could be updated to report idle cores rather than working cores. Users would then proceed to do p&r as usual but without the ReserveResourceConstraint. Some post processing step would then be needed to translate core numbers to the "correct" set of available cores. Obviously a number of these steps would be required for all the different data structures produced.... Yuck...
2. As above, but with post-processing applied after the allocation step to shift allocations into the correct core numbers. This fixes everything else but relies on nothing caring that allocations may not be within the range of the Machine's resource list, or the Machine object being patched up. Yuck...
3. SC&MP could present a virtual core numbering scheme on an appid-by-appid basis (e.g. you always got core 0, 1, 2... for your app). Lots of "yuck" in there both in terms of potential for confusion, implementation effort, major (SC&MP) API changes and potential for bugs.

[mossblaser]

@mundya and @neworderofjamie, your feedback would be very welcome as always!

[mundya]

I think this is an excellent idea. I'm happy to do/prototype bits of the SC&MP work if ST is overloaded?

[mossblaser]

I think this is an excellent idea. I'm happy to do/prototype bits of the SC&MP work if ST is overloaded?

Sorry I missed your message!

ST's plan for this is to introduce a new chip-global struct which includes this information in a form which can be SCP'd off quickly and efficiently, ideally also including such things as the total number of working cores and number of links. To truly exploit this get_machine and get_resource_constraints should really be combined in some way... ideally without being messy...

I'm not sure whether the current (slow) implementation is release-worthy or not. If you're willing to do some implementation you should speak to ST; I'm sure he'd be delighted and besides it would make the situation with releasing this a lot easier! 140 isn't making stunningly rapid progress...

[neworderofjamie]

I'm sure either @mossblaser or ST has considered this, but get_free_sdram() is probably not enough -- If you have applications with different lifetimes, your SDRAM heap is going to get fragmented - get_largest_free_sdram_block() would help a bit, but still would not be a guarantee

[mossblaser]

your SDRAM heap is going to get fragmented

Yeah; the question of "how much free ram is there?" is not a straight question. The use-case for this is not for general purpose core-sharing in the machine (which would cause many more serious issues, e.g. routing table/key sharing etc.). Rather it is intended for easing the use of diagnostic software alongside your application. For example, the traffic burstiness measuring script I wrote requires one core on each chip in the machine but is designed specifically to safely coexist with other apps by not using routing table resources etc. In these cases you'd load the diagnostic app then your app, and unload in the reverse order. This means no fragmentation and thus the two answers are kind-of the same.

Clearly having some foreknowledge of SDRAM availability is important (e.g. if you diagnostic app uses lots of RAM) but the answer being nothing more than a not-insane estimate is probably all that is needed. The "how much is available" question can be answered by a counter (which is updated by malloc and free) while the largest free block potentially means walking through the blocks which maybe/maybe-not is more effort... We'll see what ST does anyway!

[mundya]

(there is already a function for getting the largest free block in the heap... Sadly I can't remember what it's called...)

[mossblaser]

A function but sadly not an SCP command.

Chatting with ST; his plan for supporting this was just to make a new SCP command for SC&MP which gathered all this stuff (not actually creating a new struct like SV etc.). Should be fairly easy to add!

On 25 November 2015 at 13:34, Andrew Mundy notifications@github.com wrote:

(there is already a function for getting the largest free block in the heap... Sadly I can't remember what it's called...)

— Reply to this email directly or view it on GitHub https://github.com/project-rig/rig/issues/202#issuecomment-159609369.

[mundya]

Chatting with ST; his plan for supporting this was just to make a new SCP command for SC&MP which gathered all this stuff (not actually creating a new struct like SV etc.). Should be fairly easy to add!

I think this sounds like the sanest approach. The more host <-> SpiNNaker interactions that can be made into either a short command followed by a block read or a block write followed by a short command the better...

[mossblaser]

As a break between presentation run-throughs I'm going to attempt to put together an SCP command which returns everything you might want to know about a chip (e.g. num cores, largest SDRAM block, which links are working, which cores are idle.)

On 25 November 2015 at 14:46, Andrew Mundy notifications@github.com wrote:

Chatting with ST; his plan for supporting this was just to make a new SCP command for SC&MP which gathered all this stuff (not actually creating a new struct like SV etc.). Should be fairly easy to add!

I think this sounds like the sanest approach. The more host <-> SpiNNaker interactions that can be made into either a short command followed by a block read or a block write followed by a short command the better...

— Reply to this email directly or view it on GitHub https://github.com/project-rig/rig/issues/202#issuecomment-159629388.

[mossblaser]

Fixed in #206