[Question/help] Why do we parameterize fishing mortality and recruitment using mean and deviations?

[JaneSullivan-NOAA]

The population and fleet subgroup (@Andrea-Havron-NOAA , @k-doering-NOAA, @nathanvaughan-NOAA) are working on implementing fishing mortality. We are trying to decide if we should estimate annual log_Fmort or use the log_mean_Fmort + log_Fmort_devs approach.

This conversation spurred several questions:

1. Why is the mean + devs approach more commonly used in existing assessment models than annual values? What are the tradeoffs, other than one has an extra parameter?
2. Which one should FIMS use for Fmort (and recruitment)?
3. Will there be issues with random effects estimation of recruitment if the values are not centered around a mean? We could think of examples where random effects are not centered that seem to work fine but thought we'd poll the group.

@Rick-Methot-NOAA @timjmiller @cmlegault

[Rick-Methot-NOAA]

I think the mean+dev approach should only be used when you expect no trend in the devs. More generally, one uses fxn+dev, where the simplest form of fxn is a scalar (e.g. mean), a reasonable compromise might be fxn = linear trend, and in the case of recruitment the fxn is the spawner-recruitment relationship. Any estimate of the variance of the devs is conditioned on the degree of variance explained by fxn. If you choose too simple of a fxn and impose any penalty on the devs, then to some degree the resultant time series of fxn+dev will be flatter than might be indicated by the data. In SS3, the F parameters are all freely estimated with no penalty because each F is highly informed by the total catch for that fleet in that year, and total catch is a datum known with high certainty. Also, F routinely has large trends over time so any constraint on the devs could create a bias. In SS3, recruitment is done using the fxn+dev approach. This is partly because informative data are not routinely available and for some eras of the time series informative data may be absent. For recruitment deviations, SS3 includes some alternatives to fxn+dev that seem to perform well, but a fuller test is needed. For example, one does R = R0 + dev, then imposes penalty using R-spawn_recruit_curve.

[cmlegault]

ASAP uses the mean plus devs approach because it makes it easy to use phases to estimate the mean early and the devs later. However, with TMB and no need for phases, this does not seem like a good reason anymore.

For state-space models, we have seen some realizations blowing up (going to unbelievably high values) when using a random walk to simulate data including the process errors. When the simulations do not include the process errors, then this does not happen. An advantage of using the random walk process is that it lends itself well to including correlation in the process errors.

For FIMS v1, I think it makes sense to keep the mean plus devs approach to make it easier to compare to current models. In later versions, it will be "easy" to change to random walk if we want.

Cheers, Chris

[timjmiller]

I agree with @cmlegault that the modular programming in FIMS should allow for any number of alternative models for F and recruitment so any decision now hopefully doesn't disallow additional options later. Just for keeping parameters organized though, it may make sense to have a set of parameters that would always be fixed effects (e.g., first year value/mean, variance, correlation parameters) and a set of "deviation" parameters that may be treated as fixed or random effects. The subsets of each that are used would depend on the chosen option.

[Cole-Monnahan-NOAA]

Maybe I misunderstand but it seems like the issue is whether to put a penalty on the devs, not whether to use them. If you constrain one of the devs so they sum to zero for instance, then the mean+devs approach will give the same values. You're still freely estimating N parameters.

The bias @Rick-Methot-NOAA mentioned comes about when you penalize the devs with a hyperdistribution. Agreed that if you have a trend then a simple iid normal penalty is violated (they're not exchangeable) and so it doesn't make sense to use that formulation.

I suppose I'm a fan of the fnx+dev approach too, at least initially. But if we want it to match existing software, e.g. F in SS3, then we'll have to find a way to enforce the sum-to-zero constraint. I think? Or we map off the mean and estimate the Fdevs separately with no penalty. Either way we have to drop a parameter and have no hierarchical penalty.

[Cole-Monnahan-NOAA]

I forgot to respond to the actual questions

1. Probably so people can do random effects? SAM does that for Fs too I think.
2. I vote to use fnx+devs approach
3. I don't think so? E.g. WHAM models log_naa as random effects rather than deviations centered at zero. Just have to parameterize it correctly internally to be statistically consistent I think.

[Rick-Methot-NOAA]

@Cole-Monnahan-NOAA correction. F in SS3 is just a vector of free parameters. Dev approach not used. For recruitment, the fxn + sum-to-zero-dev approach was used in SS3 so that the resultant mean could be used in the equilibrium calcs for MSY, etc. But subsequent development and testing leads me to see more merit in simpler fxn+devs

[Cole-Monnahan-NOAA]

@Rick-Methot-NOAA Yes I meant that we can parameterize F as mean+devs and then mimic SS3 by fixing the mean at 0 and having no penalty on the devs (they become freely estimated fixed effects). And we can mimic SAM by turning the mean back on and integrating out the devs as random effects (I think, I know very little about SAM). So I think that approach gives us sufficient flexibility for now.

[nathanvaughan-NOAA]

Great point @Cole-Monnahan-NOAA. I had been leaning towards independent parameters but didn't think of just fixing the mean at zero to replicate independent parameters. That is a great idea! If we have that flexibility and non-zero-centered devs in general then including internal derived mean and variance parameters would allow us/users to run the model/optimizer however we/they prefer and run comparisons easily to see if one approach has significantly improved performance.

[JonBrodziak]

Here are some comments on the setup if using ADMB.

If one uses a "log_mean_Fmort + log_Fmort_devs" approach for estimating Y years of annual log-scale fishing mortalities in which the log_Fmort_devs is defined as an init_bounded_dev_vector, then there is a numerical constraint applied in the ADMB optimization to force the sum of the elements of the log_Fmort_devs() vector to be zero. This constraint implies the effective degrees of freedom for log_Fmort_devs() is (Y-1) because one degree of freedom is lost using the sum to zero requirement. Adding in the one degree of freedom for estimating the log_mean_Fmort scalar parameter, then the number of freely estimated parameters is Y using the log_mean_Fmort + log_Fmort_devs" approach.

This is the same parameter dimension as required to estimate a Y-Dimensional annual log_Fmort vector. And so there is no benefit to reducing the number of freely estimated parameters if the log-scale annual F deviations were set up as an init_bounded_dev_vector in ADMB.

There may be a difference in the parameter spaces being defined under both approaches, however. Whether a difference exists depends on how one sets up any bounds on the log_Fmort() vector and the log_mean_Fmort scalar parameters using the ADMB init_bounded_number and init_bounded_vector syntaxes.

On Thu, Aug 25, 2022 at 2:56 AM Chris Legault @.***> wrote:

ASAP uses the mean plus devs approach because it makes it easy to use phases to estimate the mean early and the devs later. However, with TMB and no need for phases, this does not seem like a good reason anymore.

For state-space models, we have seen some realizations blowing up (going to unbelievably high values) when using a random walk to simulate data including the process errors. When the simulations do not include the process errors, then this does not happen. An advantage of using the random walk process is that it lends itself well to including correlation in the process errors.

For FIMS v1, I think it makes sense to keep the mean plus devs approach to make it easier to compare to current models. In later versions, it will be "easy" to change to random walk if we want.

Cheers, Chris

— Reply to this email directly, view it on GitHub https://github.com/NOAA-FIMS/FIMS/issues/220#issuecomment-1227222061, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACVXIZQ4UKGLMOTIZD7NJMDV25UQZANCNFSM57Q3A77Q . You are receiving this because you are subscribed to this thread.Message ID: @.***>

-- Jon Brodziak, Ph.D. NOAA Inouye Regional Center Pacific Islands Fisheries Science Center 1845 Wasp Boulevard, Building 176, NMFS/PIFSC/FRMD Mail Room 2247 Honolulu, Hawaii 96818 USA Phone: 808-725-5617 Email: @.***

“Wherever my travels may lead, paradise is where I am.” ~ Voltaire

The views expressed in this message are my own and do not necessarily reflect any position of NOAA.

[ChristineStawitz-NOAA]

from @jimianelli :

• For population models where optimizations require getting the scale (magnitude) close, estimating the mean first can help avoid crazy non-plausible parameter space (i.e., using earlier phases for mean, latter phases for deviations)

• have devs centered on mean zero should help with regularization or Bayesian applications, including having a switch for treating them as random effects

• provides clarity in errors-in-variables applications (which I guess is similar to 2nd bullet)

• provides some flexibility that otherwise might be less general and require case-specific code

Had a quick look a Bard (1974) which has been the bible for estimation, maybe something in there on it (didn't try very hard to find)

[ChristineStawitz-NOAA]

Right now the approach is to have F be a vector of free parameters and recruitment to be parameterized with a mean recruitment function + devs which is similar to SS3 and should be fine for testing, but consider rearchitecting F as simple devs + mean for MQ

[timjmiller]

wham followed asap3 and uses a "random walk" parameterization of F (initial logF and subsequent deviation parameters) that are all unpenalized fixed effects. A couple of days ago while working on an extension of wham, I changed to annual F parameters and found when estimating at least one of my test models a lack of convergence when starting from the same initial parameters so I changed back to using the deviations approach.

[ChristineStawitz-NOAA]

is this a different issue than #186 or should we close one of them out?

[iantaylor-NOAA]

They could be merged, but currently each contains unique suggestions that aren't represented in the other.

[ChristineStawitz-NOAA]

Adding comments from #186 here to close that one out

Currently, we have implemented a vector of recruitment deviations and PrepareConstrainedDeviations method in FIMS M1 to add the sum-to-zero constraint to recruitment deviations without penalty. There are other recruitment dev options available.

Describe the solution you would like. Comment from @Rick-Methot-NOAA: Note that I hope FIMS considers SS3 recruitment dev options 3 and 4 also: 1=devvector (R=F(SSB)+dev); 2=deviations (R=F(SSB)+dev); 3=deviations (R=R0*dev; dev2=R-f(SSB)); [here penalty is based on dev2] 4=like 3 with sum(dev2) adding penalty see https://github.com/nmfs-ost/ss3-source-code/issues/29 for additional info

[ChristineStawitz-NOAA]

from @k-doering-NOAA I'm not sure this if this is the best place to put this, but Liz Brooks just authored a perspective piece on modeling recruitment that may be of interest.

[Rick-Methot-NOAA]

Thanks Kathryn. We are evaluating that in SS3 at: https://github.com/nmfs-ost/ss3-source-code/issues/191