Load initial machine configuration

[dkfellows]

For #325, how should we load the initial machine configuration? Should this be a feature of the server, or should it be a separate little application?

What I've been doing for testing (mocking stuff up manually in sqlitebrowser!) absolutely won't scale.

[rowleya]

The initial machine configuration could be file-based. It is possible to generate some config files with "spinner" (https://github.com/SpiNNakerManchester/spinner), which could be adapted to generate files in a new format if desired. The question then is how to remember things like down boards when the server is stopped and started again.

[Christian-B]

comment without fully understanding the use case.

I would optimise the system for general use over optimising for initialisation as we dont do that very often so it does not matter is init is slower.

We may also want to split two kinds of init - When the Spalloc server goes down or when the whole system goes down for example after a power outage. Where the server goes down you could assume the actual boards are unchanged and use a file based system as suggested. Be this from a previous init or some kind of backup.

Whole machine going down is luckily rare so even if that takes a few hours to recover not an issue.

[Christian-B]

The question then is how to remember things like down boards when the server is stopped and started again.

There could be a hybrid solution where the file is used for normal machine and the database used to fill in the latest info such as down boards or changes noted since the file was generated.

[dkfellows]

Since state is in the database, restarting the server should be very easy. It will not need to load machine definitions each time. (The machine definition used in the DB is a lot more long-winded than that in the spinner config file, since it spells everything out by design. By doing that, the allocation algorithm itself becomes much easier and simultaneously more flexible as it is based primarily on relational algebra.)

[dkfellows]

However, there will still need to be a step to bring a machine into service, i.e., to load into the database the description in the spinner file and turn it into entries in the relevant tables. This will not be done every time the application boots. It's an import operation, not a startup operation; this is totally by design and at the core of why I went to all the effort to rewrite spalloc from scratch.

[rowleya]

Could there be an "add machine" API call which could take a file-based machine description in that case?

[dkfellows]

That's what I'm thinking of doing, yes. I'll need to look at the spinner format; it'd be nice to reuse those.

[dkfellows]

A graphical view of the current schema is this:

(I plan to put a version of this in the Javadoc once I've stopped tinkering with the schema.)

[dkfellows]

The interesting tables really are jobs and boards; pending_changes and job_request are both transient, and most of the others are either effectively static (machines, bmp, tags) or entirely static (most other tables as they represent what's effectively enumerations in Java).

The diagram is missing a computed column from boards. Can't be helped. It's just a feature of how the database editor I'm using works.

[dkfellows]

There should be a link from machines.board_model to the board_model_coordinates table. Not sure why that's missing; probably a minor schema error.

[dkfellows]

This is now done except for whatever UI surfacing we put on the admin interface. Which is really its own thing.