hubverse-org / schemas

JSON schemas for modeling hubs
Creative Commons Zero v1.0 Universal
4 stars 3 forks source link

hubAdmin::validate_hub_config - model metadata information #77

Closed LucieContamin closed 5 months ago

LucieContamin commented 5 months ago

As I was testing the new v3.0.0 schema version, I noticed an issue with the validate_hub_config() function. I created a hub for testing here: https://github.com/LucieContamin/hub_test

To validate my json files I used the hubAdmin::validate_hub_config() function and got some errors:

hub_path <- "." 
err_schema <- hubAdmin::validate_hub_config(hub_path = hub_path, 
                                            branch = "br-v3.0.0", 
                                            schema_version = "latest")

Some error were expected:

But I was confused by the other errors: In the admin.json, I have three hub_models:

"hub_models": [
        {
            "team_abbr": "",
            "model_abbr": "Ensemble",
            "model_type": "ensemble"
        },
        {
            "team_abbr": "",
            "model_abbr": "Ensemble_LOP",
            "model_type": "ensemble"
        },
        {
            "team_abbr": "",
            "model_abbr": "Ensemble_LOP_untrimmed",
            "model_type": "ensemble"
        }
    ]

They don't have a team_abbr and one is longer than 16 characters. So, as expected I got the error about that. However, I thought that by fixing the information in the model-metadata-schema.json it will fix the issue. So, I updated my model-metadata-schema.json to:

"team_name": {
            "description": "The name of the team submitting the model",
            "type": "string"
        },
        "team_abbr": {
            "description": "Abbreviated name of the team submitting the model",
            "type": "string",
            "pattern": "[a-zA-Z0-9_]*",
            "maxLength": 25
        },
        "model_name": {
            "description": "The name of the model",
            "type": "string"
        },
        "model_abbr": {
            "description": "Abbreviated name of the model",
            "type": "string",
            "pattern": "[a-zA-Z0-9_]*",
            "maxLength": 25
        }

I thought it might fix the issue, but the function hubAdmin::validate_hub_config() is not taking my update into account, is that expected? If so, should we allow the hubs to update that information?

annakrystalli commented 5 months ago

The checks for admin.json are determined by the admin-schema.json, this part in particular: https://github.com/Infectious-Disease-Modeling-Hubs/schemas/blob/2e79b13d309168824bcca7ef716ebeb9aa3cda44/v2.0.1/admin-schema.json#L54-L82 so not affected by changes in the model-metadata-schema.json

I guess we could just increase the maxLength in the central schema but it's an interesting question regarding potentially conflicting schemas for configurations in two places (admin-schema.json for hub models and model-metadata-schema.json for all models.

annakrystalli commented 5 months ago

Discussed today at the retreat and decided to fix by removing hub_models property from admin.json