hubverse-org / hubverse-actions

GitHub Actions for common hubverse CI tasks
MIT License
0 stars 0 forks source link

add an action to validate updates to hub config files #9

Closed elray1 closed 3 months ago

elray1 commented 9 months ago

this would make me feel better about things when i'm making updates to config files

zkamvar commented 3 months ago

I can give this a shot. I had initially thought this was completed in https://github.com/hubverse-org/hubverse-actions/commit/b59f6bc121540a99de2f2bfbeaefcdc9a14ec89f, but now I understand that doesn't validate the config files, so this action would be, in a sense, validating the validators.

I'm going to implement an action that will use hubValidations::check_config_hub_valid() to validate the config files because it looks like it wraps the validations from hubAdmin

As a side note: I believe a workflow like this would have prevented the config errors I was getting in https://github.com/hubverse-org/hubDocs/issues/152

  • πŸ€” Validating config files
    • πŸ‘πŸ½ installed hubAdmin package
    • πŸ€” validate_hub_config() fails for metadata schema because some elements need to be arrays
annakrystalli commented 3 months ago

Sorry I should have added more detail in the request!

Yes the config is checked during submission validation but that's only triggered when model output or metadata files are submitted.

It would be nice to have a workflow that is triggered by changes to the config.

The hubValidations check function, as you noted, actually only wraps a hubAdmin function specifically hubAdmin::validate_hub_config(). So we only need to run this function.

This returns TRUE or FALSE so we want the workflow to fail if it returns FALSE. If FALSE, an errors table is also attached as an attribute.

There's also function hubAdmin:: view_config_val_errors() that presents any validation errors in a nicely formated gt table. Would be amazing to post that as a comment but not sure how straight forward it will be. If not just printing the errors attribute would be useful instead.

zkamvar commented 3 months ago

currently testing things out in https://github.com/zkamvar/example-simple-forecast-hub/actions/runs/10201766462/job/28224400302?pr=1

zkamvar commented 3 months ago

Okay, I've got something working here:

https://github.com/zkamvar/example-complex-forecast-hub/pull/1

(because I'm going to delete my fork, it looks like this:)

:x: Invalid Configuration

The schema in hub-config/ appear to be invalid. Details about the exact locations of the errors can be found in the table below.

hubUtils config validation error report
Report for directory /home/runner/work/example-complex-forecast-hub/example-complex-forecast-hub/hub-config using schema version v3.0.0
Error location Schema details Config
fileName instancePath schemaPath keyword message schema data
admin.json contact β””name properties β””contact └─properties └──name └───type type ❌ must be string string Maintainer!!!?, name?>??
For more information, please consult the hubDocs documentation.