Evaluate the potential for creating functionality to support generating json config files

[annakrystalli]

One of the the most difficult and time consuming part of setting up and maintaining a hub is currently generating and updating the tasks.json config file.

It would be great if hubUtils could provide functionality for helping hub maintainers create and update their tasks.json files.

Initial inspiration can be sought from the EML package as well as functionality already being used by @LucieContamin & @shauntruelove .

[annakrystalli]

Link to megaround and potential starting code for proposed functionality: https://github.com/midas-network/megaround-scenariohub

[annakrystalli]

Useful functionality:

• [ ] Ability to create a modeling task items (task_ids, output_type and target_metadata) from scratch in R as lists and write to json
  • [x] create_task_id()
  • [x] create_output_type_mean()
  • [x] create_output_type_median()
  • [x] create_output_type_quantile()
  • [x] create_output_type_cdf()
  • [x] create_output_type_categorical()
  • [x] create_output_type_sample()
  • [x] create_modelling_task()
  • [x] create_round()
• [x] Ability to create target metadata object from task ids item
• [ ] Ability to read specific modeling task items from json config file and use as a template for creating task id object in R
• [x] Ability to bring all this together in a round object
• [ ] Ability to append new rounds. This should include being able to attend new values to variables which are used as round_id variables.
• [ ] Launch validate_config before writing anything out
• [ ] Create vignette to showcase functionality

[annakrystalli]

Hey @LucieContamin (and potentially @shauntruelove & @nickreich if you are interested in providing feedback)

I'm just starting to do some work on this and would be interested in your thoughts on the following.

When generating new config elements, if the user does not provide values for properties, we could potentially use either examples from the schema for a given property or make use of the default json schema keyword and add default values to our schema that can be used when creating elements in R from our schema.

Here's an example of what I'm thinking:

User provides values

create_task_id(task_id_name = "horizon",
               required =c(1L), optional =c(2:3))
#> $horizon
#> $horizon$required
#> [1] 1
#> 
#> $horizon$optional
#> [1] 2 3 4

When any property is NULL use examples/default from schema

create_task_id(task_id_name = "horizon",
               required = NULL, optional = NULL)
#> $horizon
#> $horizon$required
#> NULL
#> 
#> $horizon$optional
#> [1] 1 2 3 4

Do you think being able to create elements with default values would be useful? If so, do you think it might be a good idea to review current examples or think consider adding default values to our schema to better capture what reasonable defaults might look like?

[LucieContamin]

Hey @annakrystalli ,

I am not certain on how a default value would work for all hubs.

First the default value might be difficult to identify. For example, the default "horizon" might be difficult to find, as for example the US scenario and US forecast hubs have different default horizon values. And also, we might have some target without horizon so we should have the possibility to have NULL for both "required" and "optional" field, especially if both parameter are set to NULL as input of the function.

In this case, if we have a default, should the default behavior be NULL for all variables?

Second, if we have a default value, I see another potential issue: when someone create the tasks.json file, they might forget to set a variable, as a result it will be set to a default value and will not return an issue in the validation of the JSON file. And in this case, we might prefer to have the validation returning an error.

Does that make sense?

[nickreich]

I second the idea that default values could be confusing and lead to undesirable consequences. Maybe for now we just leave the behavior as simple and nothing filled in by default and then later if we see a need or a place where things could be streamlined we add it?

[LucieContamin]

It sounds like a good plan to me. Thanks

[annakrystalli]

Ok, thanks for the feedback @LucieContamin & @nickreich . I'll just make it required for the users to provide values then.

[annakrystalli]

Clarify required copying behaviour

Currently it's difficult to envisage what useful copying functionality would look like but will revisit once @LucieContamin has had a chance to work with the functions as this could help clarify requirements

[annakrystalli]

Currently stalled on https://github.com/ropensci/jsonvalidate/issues/65