intel-spark / automl

Apache License 2.0
0 stars 0 forks source link

add MC dropout to model uncertainty and add confidence level #18

Open shane-huang opened 5 years ago

shane-huang commented 5 years ago

Monte Carlo Dropout is a simple way of adding uncertainty for modeling extreme/unexpected events and for anomaly detection.

Refer to paper: https://arxiv.org/abs/1709.01907

shanyu-sys commented 5 years ago

use Monte Carlo dropout with tsp.fit(mc=True) add a new interface predict_with_uncertainty for pipeline:

y_pred, y_pred_uncertainty = pipeline.predict_with_uncertainty(input_df, n_iter=100)

y_pred is a dataframe with datetime and value column. value column contains the mean value of the prediction results for n_iter times. y_pred_uncertainty is a numpy array with a shape of [num samples, future_seq_len]