Our estimators currently do not compute metrics. It should me implemented.
The interfaces of the fitting and training methods already have metrics support, so we just need to handle them during training and DE-init.
Ideally we should implement a BaseMetric class for the API and a couple of metrics we could use for testing.
Since metrics and loss functions behave similarly maybe one could inherit its API from the other, or both could inherit it from a common class (class BaseMetric(BaseEval) and class BaseLoss(BaseEval)).
During training after evaluating the loss all metrics in the metrics list should be evaluated and added to the history object.
Idea
Our estimators currently do not compute metrics. It should me implemented. The interfaces of the fitting and training methods already have metrics support, so we just need to handle them during training and DE-init. Ideally we should implement a
BaseMetricclass for the API and a couple of metrics we could use for testing. Since metrics and loss functions behave similarly maybe one could inherit its API from the other, or both could inherit it from a common class (class BaseMetric(BaseEval)andclass BaseLoss(BaseEval)). During training after evaluating the loss all metrics in the metrics list should be evaluated and added to the history object.Dependencies
Comments