openflighthpc / flight-control

0 stars 0 forks source link

Replace project compute group config files with database records #68

Open timalces opened 1 year ago

timalces commented 1 year ago

Currently details of each project's compute groups (group names, chart colours, regions, group priorities, node types, node priorities and node counts) are stored in project specific yaml files. These are generated using a project's instance logs and a default.yaml template file.

In the future Control will be hosted using Dokku, which normally involves losing locally generated files whenever a deployment is made. Although this can be prevented by configuring some persistent storage, this is not ideal for data recovery, or if we wanted to scale Control. Editing the yaml files within Dokku is also slightly less straightforward, and would likely require giving more people access to the primary and secondary hosting instances.

It would therefore be better if this compute group configuration was persisted in the database, and options to configure or regenerate the groups/nodes details added to the policies page.

We should also add a step to the new project workflow (in the projects:manage rake task) to record initial instance logs and generate this config. We should also improve error handling for if the config has not yet been set, as currently this raises an error and makes a project unreachable in the web browser interface.