Final error calculation

[Niharikajo]

Hello Wenjie,

I have a doubt regarding the calculation of the final error metrics on the test data.

Suppose my sample data looks like this:

date          A       B
timestamp1    3       5
timestamp2    4       7
timestamp3    6       8
timestamp4    8       10

After introducing 50% missingness :

date          A       B
timestamp1    Nan     5
timestamp2    Nan     7
timestamp3    6       8
timestamp4    Nan    Nan

After imputation :

date          A       B
timestamp1    2       5
timestamp2    4       7
timestamp3    6       8
timestamp4    6       5

1. The MAE, RMSE, and MRE are calculated only on the imputed values or on the whole dataset?
2. Can you explain the MAE, RMSE, and MRE formulas/ equations used.

Thank you, Please let me know

Regards Niharika Joshi

[WenjieDu]

Hi Niharika,

Answers to your questions:

1. Only on the imputed values because they are imputation errors;
2. Regarding the equations, we have explained in the paper. I think the point making you confused is about the mask term in the input. The mask term is to indicate which values should be taken into the calculation of the error. In your first question, we only calculate the error between imputations and observations. Correspondingly, we make respective values in the mask term as 1, and the left as 0 (cause we don't calculate error on them). The mask term in your first question for calculating imputation errors is as below:

date          A       B
timestamp1    1       0
timestamp2    1       0
timestamp3    0       0
timestamp4    1       1

[Niharikajo]

Thank you, Your answers are very helpful.