hubverse-org / hubUtils

Utility functions for Infectious Disease Modeling Hubs
https://hubverse-org.github.io/hubUtils/
Other
6 stars 3 forks source link

Revisit and decide the output of `create_model_out_submit_tmpl` when `require_vals_only = TRUE` #90

Closed annakrystalli closed 1 year ago

annakrystalli commented 1 year ago

The main discussion was started in PR https://github.com/Infectious-Disease-Modeling-Hubs/hubUtils/pull/86

nickreich commented 1 year ago

Fleshing this out a bit more, the "current" functionality with template building can yield weird and illogical output when require_vals_only = TRUE. e.g.

library(hubUtils)
hub_con <- connect_hub(
      system.file("testhubs/flusight", package = "hubUtils")
    )
create_model_out_submit_tmpl(hub_con, round_id = "2022-12-12", required_vals_only = TRUE)
#> ! Column "target" whose values are all optional included as all `NA` column.
#> ! Round contains more than one modeling task (2)
#> ℹ See Hub's 'tasks.json' file or <hub_connection> attribute "config_tasks" for
#>   details of optional task ID/output_type/output_type ID value combinations.
#> # A tibble: 29 × 7
#>    forecast_date target horizon location output_type output_type_id value
#>    <chr>         <chr>    <int> <chr>    <chr>       <chr>          <dbl>
#>  1 2022-12-12    <NA>         2 US       pmf         large_decrease    NA
#>  2 2022-12-12    <NA>         2 US       pmf         decrease          NA
#>  3 2022-12-12    <NA>         2 US       pmf         stable            NA
#>  4 2022-12-12    <NA>         2 US       pmf         increase          NA
#>  5 2022-12-12    <NA>         2 US       pmf         large_increase    NA
#>  6 2022-12-12    <NA>         2 US       quantile    0.01              NA
#>  7 2022-12-12    <NA>         2 US       quantile    0.025             NA
#>  8 2022-12-12    <NA>         2 US       quantile    0.05              NA
#>  9 2022-12-12    <NA>         2 US       quantile    0.1               NA
#> 10 2022-12-12    <NA>         2 US       quantile    0.15              NA
#> # ℹ 19 more rows

Created on 2023-07-14 by the reprex package (v2.0.1)

Multiple people didn't like how this returned an output where output_type was clearly tied to a specific target, but yet target wasn't specified in the template generated.

In a conversation on July 12, 2023, we decided I think to keep the current slightly broken functionality in place, but would like to update the template creation as follows: