Differentiating "approved" blueprints - Githubissues

CWorthy-ocean / C-Star

C-Star is a python package for setting up and running ocean model simulations, with a particular focus on marine carbon dioxide removal (mCDR) applications.

https://c-star.readthedocs.io

Apache License 2.0

10 stars 4 forks source link

Differentiating "approved" blueprints #8

Open TomNicholas opened 3 months ago

TomNicholas commented 3 months ago

Downstream, there should be some indication of whether a Case is an "approved" blueprint-sourced one or something the user has cooked up. While this isn't strictly that, I think it's good to bake something similar in earlier and keep it in mind.

_Originally posted by @dafyddstephenson in https://github.com/CWorthy-ocean/C-Star/pull/24#discussion_r1681996691_

TomNicholas commented 3 months ago

The context for this is the presence of a property in CWorthy-ocean/C-Star#24 which stores whether or not a case was created from a blueprint or not.

Whilst I agree that we should have some easy way to differentiate a Case that matches a validated blueprint from one that doesn't, I'm not sure that this should be a static property of the Case object.

Really whether or not a particular Case (i.e. model configuration) is "validated" is a subjective decision made by us at CWorthy - it could for example change over time. It's not really a property of the actual set of configuration parameters, because it cannot be derived from information in the Case instance alone. It just tells you whether or not that set matches some set we at CWorthy have separately decided is "validated". I therefore wonder whether a .validated property should actually dynamically go and compare against an online database that we at CWorthy manage...

TomNicholas commented 3 months ago

Perhaps also returning the validation status as a boolean True/False is simplistic. Users might want to know when and how it was validated, or be linked to further information about the validation procedure.

TomNicholas commented 3 months ago

It just tells you whether or not that set matches some set we at CWorthy have separately decided is "validated".

Two examples of when this might matter:

If CWorthy validates a configuration then later realizes there is a problem - we would want to be able to control the fact that we would now consider this case not validated.
Similarly if someone is running some configuration before it has been formally validated, but then it passes our validation checks, it would now have become validated. We would want to be able to remotely now change the fact that this setup is now considered validated, even though nothing in the Case object has changed.

TomNicholas commented 3 months ago

Perhaps also returning the validation status as a boolean True/False is simplistic.

Validation might also be contextual - i.e. a Case is validated for some use cases but not for others.

TomNicholas commented 3 months ago

cc @matt-long