Softswitch: Implement Hardware Idle

[heliosfa]

This will be needed for Synchronisation steps in DPD as it is implemented.

[heliosfa]

Implementing hardware idle is not going to be as easy as it first seems.

Underlying support in the Softswitch (and a couple of needed tweaks in the composer to exploit the Softswitch's function-pointer design) has been implemented on the feature branch for this issue and seems to work as far as can be tested currently.

Unfortunately, the hardware idle detection requires all threads in the system, whether they are actually executing or not, to end up at a call to tinselIdle() for the idle detection to work. Ultimately, this means that we need to make some deeper changes to composition, deployment and startup logic.

My proposed way forward is to create a "dummy" softswitch that starts and enters a while(1) that just calls tinselIdle() ad infiniteum. This then needs to be loaded onto the relevant cores that are not part of the application.

The obvious option is to make use of Hostlink's Boot() method, as this loads one instruction binary and one data binary onto every core in the system, and then overwrite the instructions and data for the cores we actually care about. The problem with this is that Boot() calls startAll() at the end of its execution, knocking the core out of the bootloader. There is also the issue of what about unused cores and threads that share instuction space with active threads? they will try to execute the "normal" softswitch.

So, suggested chain of events to realise this:

• Modify Softswitch so that threads with no loaded devices enter a while(1) that just calls tinselIdle().
• Modify Composer to generate suitable 0-device ThreadContext initialisers for other threads on the same core.
• Modify Composer to generate and build "dummy" binaries for instruction and data space. The data binaries should contain suitable 0-device ThreadContext initialisers so that cores that share instruction space (and thus an active softswitch) with active cores block on tinselIdle().
• Modify Hostlink so that Boot() or an equivalent method can be called to load all of the cores without starting them.
• Modify deployment logic to deploy the dummy binaries to all cores.
• Modify deployment logic to use startAll() instead of startOne().
• Modify deployment logic to use go() instead of goOne().

This seems to have the minimal overall overhead in terms of implementation effort and setup time impact.

thoughts?

[mvousden]

Regarding Hostlink changes, see https://github.com/POETSII/tinsel/pull/111.

[mvousden]

So what I'm going to do is:

• Introduce a preprocessor define HWIDLE_SINGLE_APP_MODE to allow the Orchestrator to discriminate between single-application (for hardware idle, =true) and non-single-application behaviour (=false).
• Support an extra boolean argument in APP,SPEC messages (from Root to Mothership), which informs the Mothership whether to use startAll and go over startOne and goOne. This extra argument is stored in the AppInfo object in a bool soloApp field, and its value is defined from the aforementioned preprocessor define (in root) - the preprocessor define is not read by the Mothership (separation of concerns).
• Support an additional input message APP,EMPT (from Root to Mothership) that takes an application name and a path to a binary. This message causes the Mothership to blindly load all cores under its command with that binary, using Hostlink::boot(..., start=false). This "distribution" message is to be sent by Root if SINGLE_APP_MODE is true, and this message contributes to the deployed message count distCountExpected if AppInfo::soloApp == true.
• Change the deployment logic (in Root) to:
  • Send an APP,EMPT message when HWIDLE_SINGLE_APP_MODE == true and an application is being deployed, as part of the deployment process in CmDepl::DeployGraph.
  • Set the new APP::SPEC boolean field with the contents of HWIDLE_SINGLE_APP_MODE.
• Update the documentation with these changes.

[mvousden]

See #282 for my progress on the above.

[m8pple]

Isn't this done now? I thought it was fixed long ago.

[mvousden]

If I recall correctly, @heliosfa is still in documentation hell over this one.