2021.00.41_removehistoricalcomps sensitivity run

[kellijohnson-NOAA]

What would I do with weight-at-age data from that year that is being removed? Weight-at-age is a combination of fishery and survey. So, in reality I would really need to remake the dat file leaving only survey ages for those years that have age compositions removed, which is a pain. Currently, I am leaving the weight-at-age the same. At least we will have that to look at on Tuesday.

[kellijohnson-NOAA]

Some initial results ... wondering if anyone has thoughts on next steps? Note, if we compare the mcmc output from above to their corresponding MLE runs the current status is very different for just a few runs, making me think that the bias adjustment ramp is more influential here than what we would hope

[cgrandin]

That's great, removing the old age data doesn't change the current status. That is interesting how much different those few MLE runs are. What leads you to say it is the bias adjustment causing it?

[aaronmberger-nwfsc]

That is quite interesting. First off, is the scale of SSB here not using the bioscale version of r4ss as it seems 2x larger (though that's beside the main point). Yes, it seems like the bias ramp, which in MLE world would be iteratively evaluated and changed depending on the data, plays a role here. Because the scales are different it is somewhat hard to see but the variability from beginning to 1990 among runs is fairly similar between the MLE and MCMC, but clearly there are some key differences post 2010 for some runs as you say (and those weird post 2010 runs in MLE are quite different colors suggesting they are not just from one time period, correct?).

What does this tell us though? That historical age comps have little to no influence on current estimates of SSB (and it appears also on current estimates of relative stock status, Bratio, given the equilibrium SSB is mostly stable among runs). So given that we wouldn't expect to "up weight" effective sample sizes during that historical period, this shows that severely downweighting them (effectively to 0 weight by removing them) has no real impact in terms of current management. So would this then also imply that getting the effective sample size wrong during this time would also have little/no influence on current management. So my take home is that this shows we may not need to be so concerned about digging up (if even available) all the sampling techniques done during this time to ensure we get the annual initial sample sizes for age composition data set exactly correctly (e.g., the tow versus trip aggregation; the # of age samples by year; the study design to collect samples; etc.). These questions still remain for more recent periods, and if a solution was found or determined unnecessary for the recent period, that would then seemingly apply going backwards in time to the more historical period were age collection/sample design is not so easily recalled.

We also have to remember that this doesn't include changing wt-at-ages, so changing those would influence our overall mean wt-at-age equilibrium condition vector and thus could change relative stock status.

I'm not sure what else to do here for this year.

[kellijohnson-NOAA]

@aaronmberger-nwfsc , yeah sorry I didn't turn bioscale on and correct, one would hope that if you were iteratively tuning the bias adjustment ramp that the run would match the mcmc more closely, but who really does that for sensitivity runs / retrospectives?

[aaronmberger-nwfsc]

We could take this pre-1990 age comp evaluation to another step which we've talked about in the past and simply add in a time block for the Dirichlet parameters such that we have a DM parameter for fishery comps pre-1990 and post-1990 and then one for survey (as we always do). This is a bit ad-hoc but might be interesting to see.

Yes, no one does that for sensitivity/retros.

[kellijohnson-NOAA]

I checked and the DM parameters were relatively similar no matter what data was included, though that doesn't mean that they would be if you allowed for a time block. I don't really think that the time block gets at what we want though given the SRG complaint. I believe the crux of the issue is that a trip doesn't necessarily equate to a tow and yet we treat it as being that way.

[aaronmberger-nwfsc]

One other thought I had, and I believe we've been over this before so just 'thumb down' this comment as way to quickly remind me that its been done or not possible. I know there is not haul/tow level information for shoreside fleet (only trip), and at-sea sector has information by hauls. If we assume that the amount of catch in a haul or trip is indicative of the information content related to age compositions (and assume that sampling/subsampling schemes have adequately captured age compositions despite being a bit variable across all years), then would it be feasible as a first cut towards the tow/trip questions of sample size aggregation to look at the distribution of total catch across at-sea hauls or tows versus the distribution of total catch across shoreside trips? This could be looked at by year and across years to see if there are substantial swings or differences (i.e., does the inter-annual variance considerably increase compared to the intra-annual variance and how do they compare between sectors?). My guess is that it will be a bit messy, but it might show that on average the catch is x-times higher in a shoreside trip then in an at-sea haul/tow. Again, this would just be a first cut type of analysis. And first cut because we then have to think about the variable age sampling schemes over time as an overlay to these results....

[kellijohnson-NOAA]

What exactly do you mean by "distribution of total catch across at-sea hauls or tows versus the distribution of total catch across shoreside trips"? Do you mean the landed weight of a tow vs the landed weight of a trip?

[aaronmberger-nwfsc]

Yes, just thinking about starting with characterizing differences between a tow versus a trip. I thought I've heard from fishermen that they think they are pretty close (i.e., an at-sea haul is nearly the same as a shoreside trip because shore side trips are usually one or two hauls), though I'm not sure.

[aaronmberger-nwfsc]

And its all about variability through time for the initial sample size issue right, so if there wasn't much difference over the last 10 to 15 years say then when combined with your reverse-retro we might be getting closer to convincing ourselves that we are good enough, especially given all the other uncertain aspects in a stock assessment.

[kellijohnson-NOAA]

At-sea hauls, on average, are 2x as large as a trip in the U.S. fisheries.

[aaronmberger-nwfsc]

Just as a set of exploratory MLE runs I reduced the US at-sea sector hauls to 15% of what they were because 0.15 is roughly the difference between the average number of fish samples per haul (~3 fish) and the average number of fish samples per trip (~20 fish); so 3/20ths. I did this for all CP and MS years on the US only; and then I did it by also doing the reduction additionally to the Canadian FT fleet (not sure about the sampling differences between Canadian FT and shoreside fleets though, so just assuming similar relationship as US for now). In this figure the base is our 2021 base model and the Adjust Age Comp US only is reducing at-sea sample sizes by 85% and the Adjust Age Comp US and CAN is reducing at-sea and FT by 85%. I know this is only a time invariant reduction so its not too surprising that the results are insensitive (the DM parameters for the fishery just increased to compensate for the decrease in overall resulting input sample size; goes from [theta/(1+theta)] of 0.45 to 0.73).

On the US side of things the sampling effort for shoreside has pretty consistently been 20 fish per trip (at least since 2008) and the at-sea has pretty consistently been about 3 fish per haul (again, at least since 2008). So from 2008 onwards there is little evidence to suggest a time-varying component to these sample size adjustments. Prior to 2008, there is some evidence that at-sea sampling may have sampled closer to 5 fish per haul going back to 1999. Sampling may have been even higher (perhaps closer to 20 fish per haul) for at-sea from 1991 to 1998. So I could go further and adjust the ratio across three periods: 1) 1991-1998 would be no adjustment because 20 fish/haul = 20 fish/trip; 2) 1999-2008 the adjustment would be 5/20ths = 0.25; and 3) 2009 to present would be 3/20ths = 0.15. This only addresses the fish/haul or fish/trip discrepency; it doesn't get into the differences in catch weight between a haul and a trip (2x difference as Kelli notes).

[kellijohnson-NOAA]

are we done with this? do we need a figure, sentence, or something else in the document or will it go into a figure or nothing?

[aaronmberger-nwfsc]

Let's keep this open for now as I may reference it when looking at SRG request section and compiling the sampling history stuff. I'll close it when I'm done.

[aaronmberger-nwfsc]

I used this to help write a few more sentences in the response to SRG, so I think we can close now...