pfmc-assessments / lingcod

2 stars 2 forks source link

re-run diagnostics for yet another north model #111

Closed iantaylor-NOAA closed 3 years ago

iantaylor-NOAA commented 3 years ago

model 2021.n.021.001 added in cd27d3983ef96a35ee6490aa20d2ef76e6d50720 (with forecast file added as an afterthought in 4594a2da1e3255443066029d722b8546f7a880e3 to avoid a separate place for forecasts) has a more comprehensive solution to the descending slope parameters that were near the upper bound, but not triggering the warning about parameters on bounds.

Stuff to add to the repo:

iantaylor-NOAA commented 3 years ago

Sorry, @kellijohnson-NOAA, hold the horses. A further change will be need in the initial values or phasing because I just confirmed that the likelihood is not as good as jitter 0.05 models. I can't sort that out until after lunch. I think the improve model configuration will work fine once it can find the proper solution reliably. I don't see using the .par file as a solution, but using the control.ss_new file to start from the estimated parameters (except for recdevs) from the previous run would be a worse case backup plan if I can't find a better solution. I'll post something more after eating lunch.

This model configuration was developed after issues in the jitter for 2021.n.020.001, but for future reference, a faster diagnostic that I SHOULD have used is the parameter distribution plots like the one below where some of the descending slope parameters are obviously near the upper bound and have really wide uncertainty. I've confirmed that none of the parameters in the new model 21 show this pattern. image

iantaylor-NOAA commented 3 years ago

Best likelihood yet found in model 2021.n.022.001 (inputs added in ed945a0).

I'm not quite sure why the model is so sensitive. My best guess is that there's too much data and the search among 132 years of recdevs for the best values is cumbersome. But for now I'm happy to call this good enough and run the diagnostics and finish writing the report. Thank you for your patience, @kellijohnson-NOAA, and for running all the diagnostics.

Note that the differences among these models are really trivial as shown in the figure below (where the green is a 22.002 which is just 22.001 with -nohess so I could examine it faster). compare_2021 n 020 001_2021 n 020 020_2021 n 022 002

kellijohnson-NOAA commented 3 years ago

No problemo, just glad I wasn't the one tasked with doing the investigation. Have fun with Rosie this afternoon :)

iantaylor-NOAA commented 3 years ago

Clarification, difference in likelihood between the 2021.n.022.001 I just posted and the better jitter 0.05 solution is essentially all due to the softbounds likelihood for parameters that were previously close to bounds and are now fixed at those bounds so that likelihood component is no longer present. So really this is just the same better jitter solution represented as a new model and not requiring jitter to get there.

iantaylor-NOAA commented 3 years ago

Thank you. Additional sensitivities are now running and I'll be checked out for the afternoon.

kellijohnson-NOAA commented 3 years ago

pre-checked the jitter and so far nothing lower.

iantaylor-NOAA commented 3 years ago

model output and custom plots (maybe created during build anyway) added in 6cd6aaa