One suggestion in the workshop was that we should investigate whether it matters to have additional features included in your model.
I think the best way to implement this is to add an entry in the submission_details.yml whether other features than historical fatalities were included in the model, and possibly how many input features where used in the model (the last might be difficult to count, but will still give us some indication about the complexity of the input feature space).
When this is present, it would be a simple task to subset evaluation based on this information.
One suggestion in the workshop was that we should investigate whether it matters to have additional features included in your model.
I think the best way to implement this is to add an entry in the
submission_details.ymlwhether other features than historical fatalities were included in the model, and possibly how many input features where used in the model (the last might be difficult to count, but will still give us some indication about the complexity of the input feature space).When this is present, it would be a simple task to subset evaluation based on this information.