search

facebookexperimental / Robyn

Robyn is an experimental, AI/ML-powered and open sourced Marketing Mix Modeling (MMM) package from Meta Marketing Science. Our mission is to democratise modeling knowledge, inspire the industry through innovation, reduce human bias in the modeling process & build a strong open source marketing science community.
https://facebookexperimental.github.io/Robyn/
MIT License
1.13k stars 336 forks source link

Error in NRMSE calculation #801

Closed Jakobus51 closed 1 year ago

Jakobus51 commented 1 year ago

In Robyn/R/R/Model.R line 1097 and 1098, there is a mistake in the calculation of the NRMSE.

nrmse_val <- sqrt(mean(sum((y_val - y_val_pred)^2))) / (max(y_val) - min(y_val)) nrmse_test <- sqrt(mean(sum((y_test - y_test_pred)^2))) / (max(y_test) - min(y_test))

It should be without the sum, just like you did for the nrmse_train calculation nrmse_val <- sqrt(mean((y_val - y_val_pred)^2)) / (max(y_val) - min(y_val)) nrmse_test <- sqrt(mean((y_test - y_test_pred)^2)) / (max(y_test) - min(y_test))

gufengzhou commented 1 year ago

you're right. Will fix that soon. To assure things: it doesn't affect the optimisation process, because either mean or sum will be just minimised. So theoretically it doesn't really matter. But sure we'll Correct it.