search

PIFSCstockassessments / ss3diags

R package with advanced diagnostics to evaluate a Stock Synthesis model. Diagnostics include residual analyses, hindcasting and cross-validation techniques, and retrospective analyses.
http://pifscstockassessments.github.io/ss3diags/
2 stars 4 forks source link

Adjusted MASE for Hindcast Cross-Validation #37

Open MOshima-PIFSC opened 2 years ago

MOshima-PIFSC commented 2 years ago

As per Henning's email to Nicholas, a potential update for the adjusted MASE could be implemented that only adjusts the naïve prediction residuals with minimal contrast instead of the MAE (current method). To do this the following changes would be made to SSmase.R (lines 231-234):

scaler <- mean(abs(naive.eval))
scaler <- mean(pmax(abs(naive.eval),MAE.base.adj)
mase <- maepr / scaler
mase.adj <- maepr /scaler.adj

This method would be more targeted at adjusting specific years with very low contrast vs. the entire timeseries. Additionally, it would be helpful to update the documentation for this function, in the R script and the user handbook to improve understanding of the application and interpretation.

@N-DucharmeBarth-NOAA would you have time and be willing to help us work on this issue to improve the MASE function?

N-DucharmeBarth-NOAA commented 2 years ago

@MOshima-PIFSC let me finish up this blue shark assessment work first and then I can make time to help out with this issue.

MOshima-PIFSC commented 2 years ago

Thanks @N-DucharmeBarth-NOAA no rush!