hubverse-org / hubValidations

Testing framework for hubverse hub validations
https://hubverse-org.github.io/hubValidations/
Other
1 stars 3 forks source link

Output error/warning/message when model output updated #65

Closed LucieContamin closed 6 months ago

LucieContamin commented 7 months ago

Following the issue #44 and the PR #63, I wonder if the test to check for modifications/deletion of a previously submitted model output files should have an additional parameter with the possibility to have the check either:

I am thinking for the Scenario projection hubs, it happen sometimes that a team wants to update their submission during an ongoing run and in this case we will allow the team to modify(or even remove) their submission.

We can technically bypass the failed validation and still merge the PR manually but I think it might be more clear to the team submitting to have a message/warning than an error (in the case we allow modification)

elray1 commented 7 months ago

Another similar idea could be to (provide an option to) allow changes as long as the submission window has not expired... Though maybe in that case this check would just be turned off as its pass/fail results would be equivalent to the submission window check?

LucieContamin commented 7 months ago

It might still be interesting to know that it's an update of a submission in an ongoing round.