We write "Any aggregation function $C$ may be specified by the user. For example, a median ensemble may also be created by specifying "median" as the aggregation function, or a custom function may be passed to the agg_fun argument to create other ensemble types. Similarly, model weights can be specified to create a weighted ensemble."
This is true, but upon reading this I thought, "this is dangerous". A user could get themselves into trouble by providing an invalid aggregation function, e.g. $C(p) = mean(p) + 100$ where p is a vector of component model predictive cdf values. Similar issues come up with weights, where i think depending on the output type, complex weighting strategies (e.g., cdf weights that depend on x) could get you into trouble. I think we do some checks/restrictions on weights and have an open issue to deal with this.
Anyways I'm not sure what we would say or say differently given our current methods, but this text felt... like an invitation for criticism.
We write "Any aggregation function $C$ may be specified by the user. For example, a median ensemble may also be created by specifying "median" as the aggregation function, or a custom function may be passed to the
agg_funargument to create other ensemble types. Similarly, model weights can be specified to create a weighted ensemble."This is true, but upon reading this I thought, "this is dangerous". A user could get themselves into trouble by providing an invalid aggregation function, e.g. $C(p) = mean(p) + 100$ where p is a vector of component model predictive cdf values. Similar issues come up with weights, where i think depending on the output type, complex weighting strategies (e.g., cdf weights that depend on x) could get you into trouble. I think we do some checks/restrictions on weights and have an open issue to deal with this.
Anyways I'm not sure what we would say or say differently given our current methods, but this text felt... like an invitation for criticism.