Pacific-salmon-assess / samSim

3 stars 0 forks source link

Question about returns age composition error in loop 3 #18

Closed catarinawor closed 2 years ago

catarinawor commented 2 years ago

This is something that may become clearer as I progress in the code, but making a note here so I remember to come back to it. If you know the answer right away, please chime in! In the projection loop, obs spawner age composition error is added to the last five years of data (here). I have three questions about this code:

  1. the error is added inside a if statement, error is only added if obsREc != true rec. I think this statement is here to prevent adding error for years before nPrime. Is that correct?
  2. It seems that the age composition error is added again at each round of the loop, i.e. for each y, obsS[y-5:y-1,] is overwritten. If that is true, the observed returns for the past five years change every year? Am I am misunderstanding/missing something? (help!)
  3. Seems like both study case assume that no error is added to the observed age structure, i.e., obsAgeErr =0. What was the rationale for that?
CamFreshwater commented 2 years ago
  1. I believe you're correct about this. I remember spending a bit of time crafting this because the "priming" period was giving me some trouble, but I don't remember the precise rationale and there may be a more elegant approach.
  2. Ohh that could be the case, but I'd want to run the code to confirm. It's possible that this bug was never caught because we didn't heavily test different scenarios here (see next question).
  3. Yes I don't believe we used this in any of the case studies because none of the management procedures we developed actually used observed status to set harvest rates or assess performance. The Fraser River case studies relied upon a HCR that attempted to replicate the relatively convoluted in-season assessment process that is independent of post-season counts, while the Nass chum focused on fixed exploitation rates because of fairly ad hoc management rules in that fishery. We also evaluated MP performance using "true" rather than observed status to simplify interpretation.

Hope this helps and if anyone else remembers something differently please chime in. It's been a while since I worked with this code...

krHolt commented 2 years ago

To chime in on a few points:

  1. I agree with you Catarina - it seems that the age composition for a given cohort is over-written at each y loop. If Carrie agrees, this should be fixed. However, as with Cam's work, it will not affect our LRP case studies because we don't use age error.
  2. For the LRP case studies, we include error in the true age structure, but do not add additional error to the observed age structure because the observed age data never gets used. This is because we aren't simulating any assessment process or adding any management feedback - we are only looking at how the 'true' CUs abundances respond to exploitation when calculating LRPs.
carrieholt commented 2 years ago

For point 2. Do you mean that the same random age composition should be used for ppnObsSRet5, ppnObsSRet4, ppnObsSRet3, ppnObsSRet2, and ppnObsSRet1 between lines 1409-1451, link above? If so, I think that makes sense. If not, please let me know how you would revise code so I can better understand.

catarinawor commented 2 years ago

No, I actually think they should be different as RetX indicates the year in which the population returns (e.g., Ret1 returned a year ago, Ret2 returned two years ago). I checked the code and the values are not overwritten (I was wrong) as the ppnObsSRet[1-5] are given in 5 separate arrays. My questioning now moved down to the lines 1491-1497, when the ppnObsSRet arrays get used to generate obsRecBY.

I think that we see the observation error on the recruitment years, i.e., at the spawning grounds or when sampling the catches, and that the proportions of obsRecRY must sum to one. -- The issue is that I am not sure this is happening given the current format at the code, here is a numerical example: for y= 10: ppnObsSRet1[10-1,k,2:6] = one vector of error for y=11: ppnObsSRet2[11-2,k,2:6] = another vector of error

I think these two need to be the same, because 10-1 = 11-2, i.e., it is the same return year.

This might be a bit too convoluted to explain here. I'll implement this change on another branch and post a link here, hopefully that will make things a bit clearer. But let me know if you have more questions.

carrieholt commented 2 years ago

Don't ppnObsSRet5, ppnObsSRet4, ppnObsSRet3, etc all sum to one? (as in the code starting on line 1401).

I wonder if there is an issue that each time the code runs through the annual loop, it creates a new vector: ppnObsSRet5[y - 5, k, ] , and ppnObsSRet4[y - 4, k, ] and ppnObsSRet3[y - 3, k, ], etc. but the ppns are only observed once, not 5 times. So, this might cause the mis-alignment you mentioned?

catarinawor commented 2 years ago

Just confirmed that this is a non-issue. yes, ppnObsSRet5 (...) all some to one and they are the same across arrays for the same RY. They get generated five times, but with the same random seed, so they produce the same numbers. The code is a bit redundant but all is working correctly! Thank you for jumping into this with me Carrie, it really helps to bounce ideas to figure this out.

catarinawor commented 2 years ago

I propose this small change to the code code that generates the age ppn obs error, see here This suggestion does not change how the code operates but may have two advantages: 1) make it explicit that the age structure error is the same for the same RY across all the ppnObsSRetX arrays 2) avoid calling the ppnAgeErr() function 5x per loop, and rely instead on the similarities across the ppnObsSRetX arrays. This will probably lead to small improvements in efficiency of the genericRecoverySim() function.

CamFreshwater commented 2 years ago

Yeah I definitely see how this is redundant and think your proposed change makes sense. Thrilled that someone is improving this!

carrieholt commented 2 years ago

Yes, makes sense to me. Thanks Catarina!