search

talhanai / redbud-tree-depression

scripts to model depression in speech and text
70 stars 30 forks source link

How to calculate the mae, rmse metric? #3

Closed zhouzyhfut closed 4 years ago

zhouzyhfut commented 4 years ago

Hi, i have read your paper. I am doubt that why the metrics (mae,rmse) value in your experiment >1? I am using the mae metric in Keras to train my model, but its value is between 0 and 1? Could you tell me how do you calculate the metrics in your paper?

talhanai commented 4 years ago

Hi,

The outcome in the dataset is a score on the PHQ8-scale (I think it is phq8_labels = [0 to 24]).

  1. The labels in which the classifier is trained with are normalized by the max(phq8_labels). So during training your ground truth (true) and predictions (pred) will be between 0 and 1.

    true = [1, 0, 0, 1, 0, 1]
pred = [0.8, 0.2, 0.1, 0.7, 0.3, 0.9]

  2. To calculate MAE and RMSE, you will need to rescale your true and pred values back to the PHQ-8 scale.

    true = max(phq8_labels) * true
pred = max(phq8_labels) * pred

  3. Now you can calculate MAE (mean absolute error) and RMSE (root mean square error), as follows.

    MAE   = np.mean(np.abs(true - pred))
RMSE = np.sqrt(np.mean(np.abs(true - pred) * np.abs(true - pred)))

    I hope that clarifies it.

zhouzyhfut commented 4 years ago

Ok, i understand your explanation. Thank you for your help.