Initial web application with costs breakdown page

[timalces]

Adds initial web application with visualiser functionality, with one page /costs-breakdown showing costs chart and current node counts.

• Includes a daily chart, showing costs (in a bar chart) and remaining budget (as a line)
• Includes a cumulative costs chart, showing cumulative costs for the billing cycle, overall budget and total costs

• These charts show both recorded costs and forecasts, based on cost logs and instance logs
• By default this page shows details for the first active project, but this can be changed with the URL parameter project=projectname (this will be updated once users and logins implemented)
• By default this page shows the latest billing cycle. This can be changed to any date range using the filter form at the top of the page

• Charts will display the last day of a cycle, last day of the project, date of going over budget and the day the project's balance runs out (if within date range)
• Datasets visible in the charts can be filtered using the filter form at the top of the page. They can also be toggled visible or hidden by clicking the dataset(s) in the charts' legends
• If any compute groups are selected in this form, only those groups will be shown in the current states table. If none selected, the table will include all groups
• Supports all current spend_profile types (though may be edge cases not yet considered)
• When a project reaches its end date, budgets become 0 and all are treated as continuous

• Every 30 seconds a check is made if any new data. If there is, an alert is shown, which on closing refreshes the page (to obtain latest data)
• Costs logic is handled primarily by a new service class CostsPlotter
• This page & backend logic will require updates once action logs and scheduled requests implemented

Other updates

• Instead of adding them to config files, application credentials must now be set using EDITOR=vim rails credentials:edit, adding values for database_username, database_password and slack_token
• The project attribute archived is now archived_date, so we can track when it occurs

Must run:

• rails:db:migrate
• bundle
• yarn
• bundle exec rake assets:precompile
• Ensure all required instance mappings are created and then run rake projects:create_config:all[true] The colours for compute groups & their related storage datasets must be set.

[ColonelPanicks]

Only issue I've managed to hit was when I was changing the budget policy, selected Dynamic, it asked for an End Date and I exited the modification as this was not the spend profile I wanted:

Must be one of: fixed, rolling, continuous, dynamic
Spend profile(fixed/rolling/continuous/dynamic): dynamic
Effective at (YYYY-MM-DD): 2022-01-01
Budget policy created
Warning: dynamic spend profile requires the project to have an end date.
Please specify an end date for the project:^Crake aborted!
Interrupt:
/root/flight-control/lib/project_manager.rb:518:in `gets'
/root/flight-control/lib/project_manager.rb:518:in `add_budget_policy'
/root/flight-control/lib/project_manager.rb:63:in `update_attributes'
/root/flight-control/lib/project_manager.rb:25:in `add_or_update_project'
/root/flight-control/lib/tasks/projects.rake:7:in `block (2 levels) in <top (required)>'
/usr/local/rvm/gems/ruby-2.5.1/gems/rake-13.0.6/exe/rake:27:in `<top (required)>'
/usr/local/rvm/gems/ruby-2.5.1/bin/ruby_executable_hooks:24:in `eval'
/usr/local/rvm/gems/ruby-2.5.1/bin/ruby_executable_hooks:24:in `<main>'
Tasks: TOP => projects:manage
(See full trace by running task with --trace)

It ended up making the front-end funky as the project gets set to dynamic with no end date. This isn't a huge issue, setting it back to fixed removed the problems. I know the answer is "you probably shouldn't do that" so just a heads up!

One small improvement might be to add a Project Start indicator to the graph.

Other than above, LGTM!

[timalces]

This isn't a huge issue, setting it back to fixed removed the problems. I know the answer is "you probably shouldn't do that" so just a heads up

Setting an end date would also have fixed it, but yeah there isn't really a way of handling that situation. If it's dynamic and there's no end date, the costs are effectively spread across an infinite number of cycles (though there is some logic in there to prevent looking for infinite cycles, as that would break accessing the project's pages).

Other than not doing it/ fixing it manually, the only thing I can think of is setting some sort of default end date (e.g. 1 year after start date). But maybe that would confuse things further.

One small improvement might be to add a Project Start indicator to the graph.

Make sense - have added it:

[ColonelPanicks]

Makes sense, any assumptions would add unnecessary complications. It was more from a data validation perspective but I'm well aware that was a silly thing I did to break input!

Thanks for adding that, looks good. Happy for this to be merged 👍