pfmc-assessments / lingcod

https://pfmc-assessments.github.io/lingcod/
2 stars 2 forks source link

model to test for profiles and regularizations #83

Closed iantaylor-NOAA closed 3 years ago

iantaylor-NOAA commented 3 years ago

Model 2021.s.012.004, added in 43750ddf2ace6a37afe56a9bfd4b8e8fa79dcf18, uses ages from WCGBTS to estimate growth but excludes other age data as discussed at the end of #80. One iteration of Francis weighting has been applied which applies weights between 7% (WCGBTS lengths) and 93% for the remaining composition data.

The scale of this model is now in the ballpark with the previous assessment (yellow and red lines in figure below), in spite of numerous other model changes, indicating that the limited age data for the south model were playing a larger-than-expected role and some conflict between the age and length composition data may have been causing the implausible high R0 estimates in the previous models. This could be due to time-varying growth which we aren't able to model with the limited ages we have in this area.

There are some key alternative setups to explore next as suggested by @kellijohnson-NOAA, including exploration of recruitment settings, estimating h with prior, and adding marginal ages, but I'm done exploring models for the day so it's time to kick off some additional diagnostics. Also exploration of the impact of similar changes to the north model are warranted.

@kellijohnson-NOAA, if you want to try a version of this model with estimated h, feel free, or I can set that up tomorrow after we learn more about this one.

compare_2019 n 001 001_2021 n 011 008_2019 s 001 001_2021 s 012 004

iantaylor-NOAA commented 3 years ago

Forgot to add that to speed up the Francis-tuning in 2021.s.012.004 I changed the starter file to use the .par file for initial values. I'm not sure if that will be an issue for the diagnostics or not.

The most recent north model (just committed in 559bd0a543b8f882a8324816f9e5105e490ba6b3 is similar enough to previously explored model that I don't think it's worth re-running diagnostics).

kellijohnson-NOAA commented 3 years ago

Retros and regularization are running now.

iantaylor-NOAA commented 3 years ago

Profiles running now with a finer scale.

iantaylor-NOAA commented 3 years ago

The r4ss figures and profile results are uploaded to https://drive.google.com/drive/folders/1E3TM6PK9V8ov6jDx4Ii7ABRQRnzzdh7J. Profiles show some convergence issues with a few of the profiles, but I'm in the SSC Groundfish Subcommittee meeting until maybe 10am so haven't looked in more detail. Also the M estimates are lower than expected so the chosen range of M values should be shifted to lower values, as should the range of R0 explored.

kellijohnson-NOAA commented 3 years ago

In the North ... compare1_spawnbio image

kellijohnson-NOAA commented 3 years ago

I uploaded retros for S to that folder, and they look MUCH improved.

kellijohnson-NOAA commented 3 years ago

I haven't looked at all of the diagnostic plots, but using the hake prior for h I get image

iantaylor-NOAA commented 3 years ago

Removing ages or converting from CAAL to marginal in the north model caused a similar jump upwards in scale similar to the retro-3 case shown in the comment above. These are models 2021.s.014.002 and 2021.s.014.003.

I would like to explore two more changes to the north model right now: leaving out the WA rec index and changing the marginal rec ages to CAAL. We can then pick whatever model among those we've run to go forward for the first draft document.

iantaylor-NOAA commented 3 years ago

North model with rec age data as CAAL rather than marginal has relatively little change. I'm running again with 1 iteration of Francis tuning as the change in data necessitated the change. The fit to the WA rec age data is actually quite good and consistent across years, so I'm now less concerned about the influence of that data set as shown in earlier profiles.

Removing the Rec_WA index also had little impact. The end point was higher without the index in spite of the index having a high 2020 value. I'm inclined to go forward with the index included because we don't have a specific reason to exclude it and it doesn't have as much impact as I expected anyway.

I'll post more in a bit after the tuning is finished.

kellijohnson-NOAA commented 3 years ago

In the North, the Triennial survey has quite a bit of influence on R0 because it catches such small fish. There are a few years in which the data are not fitting this survey. Do you think that adding an sd parameter to Triennial would give the model more flexibility and it wouldn't be forced to have such a high R0? index2_cpuefit_6_Surv_TRI

melissahaltuch-NOAA commented 3 years ago

I think the fit is pretty good with the exception of 2004, which we know is different for some stocks due to the NWFSC running the survey with AFSC protocol resulting in higher catch rates compared to the rest of the time series. You could model this as a time change in q for 2004 (although I don't think that this has been done before.

On Tue, Jun 22, 2021 at 8:33 PM Kelli Johnson @.***> wrote:

In the North, the Triennial survey has quite a bit of influence on R0 because it catches such small fish. There are a few years in which the data are not fitting this survey. Do you think that adding an sd parameter to Triennial would give the model more flexibility and it wouldn't be forced to have such a high R0? [image: index2_cpuefit_6_Surv_TRI] https://user-images.githubusercontent.com/4108564/123031381-13c3fb00-d399-11eb-8c69-0f0fc8f74f50.png

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/iantaylor-NOAA/Lingcod_2021/issues/83#issuecomment-866495438, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFP5YEA7BSCVMWRCOTK4BATTUFIZZANCNFSM47C2YQ3Q .

-- Melissa A. Haltuch, Ph.D Pronouns: she/her/hers Acting Fish Ecology Division Director, NWFSC, NOAA Fisheries Research Fishery Biologist, NOAA Fisheries

*University of Washington, School of Aquatic and Fishery Science, Associate Affiliate @. @.> 206.860.3480

iantaylor-NOAA commented 3 years ago

Thank you for catching that @kellijohnson-NOAA. That's a good idea.

Looking at the index fits in the north, I now see for the first time that the extraSD for the CA Rec is huge, which I previously attributed to the index being noisy, but I now see that when I added a block to the CA Rec catchability for the south model to account for differences among the two time periods with different sampling programs (I think around model 9), I turned off "float" to estimate Q as a parameter and then succesfully turned on both LnQ_base_5_Rec_CA(5) and LnQ_base_5_Rec_CA(5)_BLK9repl_1999 as estimated parameters in the south model (along with the extraSD in both models). But for the north I accidentally also turned off float but didn't touch the parameter line (which has negative phase), thus the Q is fixed for the north CA Rec index and the fit looks funny (requires log scale to see--fig below).

I though that model 2021.n.015.003_Francis was going to be good enough for now but these two issues make me think we need to run a new 2021.n.015.004_extraSD in which the extraSD is turned on for the Triennial and the float is turned on for the CA Rec CPUE.

I'll upload the input files for that model shortly and we can divide up some further diagnostics.

image

iantaylor-NOAA commented 3 years ago

Model 2021.n.015.004 with the index fixes looks good. Suggested tunings show little change from previous run as there were not changes to the comp data, just index assumptions. The fit to the triennial is pretty similar except for 2004 and the scale is a bit lower as expected based on the profile. The fit to the CA Rec index now looks fine (lower plot).

So this seems good enough for today and for running new profiles and retrospectives. imageimage