nmfs-fish-tools / SSMSE

Management Strategy Evaluation (MSE) using Stock Synthesis (SS3)
https://nmfs-fish-tools.github.io/SSMSE
MIT License
18 stars 11 forks source link

Mismatch between AgeSel_P_X_...(X) parm_dev values specified in custom list and values input to .par file #188

Closed charliehinchliffe closed 6 months ago

charliehinchliffe commented 7 months ago

I am trying to implement future changes in AgeSel_PX...(X) parameters in the OM using the "custom" pattern and the create_future_om_list() function.

I've noticed that the list of values I provide do not match the values that SSMSE inputs for the .par file parm_dev[X]'s. The value in the .par file is instead equal to the custom list value minus the base parameter value. For example, if the base parameter for AgeSel_PX...(X) is 1, and the custom list input value for the deviation is -0.2, the parm_dev[X] in the .par file will be -1.2. If AgeSel_PX...(X) is 0.7, parm_dev[X] will be -0.9. I am not sure what is causing this, and it is not the case for other parameters I modify in the projection.

k-doering-NOAA commented 7 months ago

Thanks @charliehinchliffe ! I think for this one, a reproducible example that I could try running to duplicate the issue would be helpful. Feel free to share it via GitHub or email me! My best guess is there is a small typo somewhere doing deviation - AgeSel_PX...(X) instead of AgeSel_PX...(X) + deviation

charliehinchliffe commented 7 months ago

Thanks @k-doering-NOAA I've emailed you a reproducible example. To my mind, the values supplied to the par file should just be equal to the deviation values in the custom list as SS3 computes the new selectivity value for each year as the base + parm_dev.

k-doering-NOAA commented 7 months ago

Thanks @charliehinchliffe I plan to work on this tomorrow.

k-doering-NOAA commented 7 months ago

@charliehinchliffe the custom values are meant to be what you would like the actual value of the parameter to be, not deviations. SSMSE then behind the scens calculates what the deviation should be and puts that value into the par file. So for instance, if you would like the selectivity parameter to be 1.1 in year 2026, you would put 1.1 into the custom data frame.

You can see the calculations done by SSMSE by adding browser() after this line and looking at the list components of future_om_dat (base is the parameter value in the model; dev is the deviation that will be put in the par file, abs should match what you put in the future_om_list: https://github.com/nmfs-fish-tools/SSMSE/blob/484939bf3d4f4bb499fd91db7e47fe351d31ccd4/R/runSSMSE.R#L589

I did this for scenario 1, and the calculations seemed correct to me for the selectivity values. Could you check that they do to you? If not, maybe we can chat when I'm in on Thursday to try to figure out the discrepancy.

Two changes that I think should be made, even if the calculations are right: 1) Add better documentation on specifying custom values. Upon reading the user manual custom example, it's not spelled out that the user should put in the actual parameter values desired 2) Pass back the future_om_dat object as a return from run_SSMSE so the user can double-check their inputs are being processed as expected.

k-doering-NOAA commented 7 months ago

@charliehinchliffe I should also say thanks for providing the reproducible example! It was super helpful to understand your scenario better and I was able to get it running pretty quickly.

k-doering-NOAA commented 6 months ago

Closing this after checking in with @charliehinchliffe that this was a lack of documentation issue rather than bug! Opened separate issues for improving documentation and passing back objects to the user.