alan-turing-institute / CROP

CROP is a Research Observation Platform
MIT License
25 stars 5 forks source link

Join `model` and `test_model` tables #285

Closed mhauru closed 2 years ago

mhauru commented 2 years ago

There's a set of tables, model, model_measure, model_parameter, model_product, model_run, and model_value, for which there also exist versions with names prepended with test_. The GES model uses the test_ ones, Arima uses the plain ones. These should be unified, as was clearly the original intent.

mhauru commented 2 years ago

test_model and test_model_measure can be dropped, all the information is already in model and model_measure. test_model_product, test_model_run, and test_model_value need to be merged (data copied). Why is there no test_model_parameter, corresponding to model_parameter?

Also, GES doesn't keep the historical runs. The database only keeps the latest. This should be changed, both because it would be good to have the old runs, and because I don't know why/how it does the overwrite, but I fear it might overwrite the ARIMA results too if it used the same table.

mhauru commented 2 years ago

The old GES results aren't kept because there's an explicit call to a deleteResults function in GES, which does what it says. Will remove that call, and hopefully everything goes smoothly. It seems like some left over thing from development.

mhauru commented 2 years ago

A Slack conversation with @rmward61 and @myyong confirms that the model_parameter table is some old vestige from back when GES was being developed, an alternate scheme for keeping track of GES parameters that wasn't used in the end, or somesuch. It can, and will, be dropped.

mhauru commented 2 years ago

This is now done on test/dev. Once the code change is merged to main, we still need to drop the test_model* tables (all of them) in the prod database. Then we can close.

mhauru commented 2 years ago

The test_model_* tables have now been dropped on prod too.