pacific-hake / hake-assessment

:zap: :fish: Build the assessment document using latex and knitr
MIT License
13 stars 6 forks source link

catchability #162

Closed allanhicks closed 8 years ago

allanhicks commented 8 years ago

Do we want to report the mcmc results of catchability? I suppose that we should add a line in Table 14. I can rerun the "extraOutput" executable. I'm not sure how to automate this.

cgrandin commented 8 years ago

Where are the values located in the output?

allanhicks commented 8 years ago

Only I have them, because I have a "special" version of SS (that I coded to get the specific output).

Here are the q values from the updated model:

MLE | Median 2016 | Median 2015 1.14 1.03 0.92

cgrandin commented 8 years ago

What about getting them during the "model-partest" runs (see utilities.r)? For now though we could just hard-code them in but would like to have a better method for next year.

iantaylor-NOAA commented 8 years ago

I can work on replicating Allan's values using the model-partest. We can hardwire in the doc if needed but it will be useful to figure out if we can do it. Note this issue is redundant with #138.

On Mon, Feb 29, 2016 at 2:26 PM, cgrandin notifications@github.com wrote:

What about getting them during the "model-partest" runs (see utilities.r)? For now though we could just hard-code them in but would like to have a better method for next year.

— Reply to this email directly or view it on GitHub https://github.com/cgrandin/hake-assessment/issues/162#issuecomment-190427983 .

allan-hicks commented 8 years ago

Good point! But, I can't figure out what the object model.partest represents. You can get the MLE value for q from the following:

model.partest$index_variance_tuning_check

but I don't see the 999 posterior samples.

Sorry for the redundant issue.

On Mon, Feb 29, 2016 at 2:26 PM, cgrandin notifications@github.com wrote:

What about getting them during the "model-partest" runs (see utilities.r)? For now though we could just hard-code them in but would like to have a better method for next year.

— Reply to this email directly or view it on GitHub https://github.com/cgrandin/hake-assessment/issues/162#issuecomment-190427983 .

cgrandin commented 8 years ago

I noticed in the partest/reports directory, in the ss3_outputXXX.par files there is a value for

Q_parm[1]

for each of the posterior files XXX We could easily parse this grepping for the tag above, the get the value from the next line. Should be no problem but will have to run partest again

cgrandin commented 8 years ago

After doing what I said above I get a median of 0.3378 so clearly Q_param[1] is not the right thing

iantaylor-NOAA commented 8 years ago

Q_parm[1] is the extra SD added to the survey variance, which is an estimated parameter. The problem is that in the eyes of SS, the catchability is not a parameter.

As noted in my initial comment on issue #138, the catchability is available in the list created by SS_output from any Report file in the $cpue data.frame. For a model without time-varying or non-linear catchability like this one, we can grab $cpue$Calc_Q[1].

We could hack a purpose-built script to extract those values from the Report files in the partest folder but that's silly since Allan already got us the values. It would probably also be silly to rerun partest with code added to extract the values because ideally Rick will get SS fixed up by 2017 to report everything we need in the standard MCMC output so that we can skip the extra partest step.

So I've talked myself into thinking we should either hard-wire Allan's values or skip the catchability step and trust that nobody really cares that much. -Ian

On Mon, Feb 29, 2016 at 2:36 PM, cgrandin notifications@github.com wrote:

I noticed in the partest/reports directory, in the ss3_outputXXX.par files there is a value for Q_parm[1]

for each of the posterior files XXX We could easily parse this grepping for the tag above, the get the value from the next line. Should be no problem but will have to run partest again

— Reply to this email directly or view it on GitHub https://github.com/cgrandin/hake-assessment/issues/162#issuecomment-190430953 .

cgrandin commented 8 years ago

That makes too much sense! It will be nice if Rick makes that happen. I'll add a line for the values for this year, and we can keep an issue open so we don't forget to remove them.

cgrandin commented 8 years ago

Should it go in the parameters section of the tables, or derived quantities?

iantaylor-NOAA commented 8 years ago

In the 2015 assessment, the MLE vs MCMC vs previous year's MCMC table (our Table 13), it was shown right above the survey extra SD parameter (but only reported the MLE value, which wasn't super useful). I think we could do the same. Previous inquiries showed that if you make Q a parameter, you get the same result, just with one more thing to estimate.

On Mon, Feb 29, 2016 at 2:56 PM, cgrandin notifications@github.com wrote:

Should it go in the parameters section of the tables, or derived quantities?

— Reply to this email directly or view it on GitHub https://github.com/cgrandin/hake-assessment/issues/162#issuecomment-190437353 .

allan-hicks commented 8 years ago

Thanks for clearing this up. Sounds like a good solution. I suggest that it goes in the parameter section. It theoretically is a parameter, but it is analytically derived by taking the derivative of the likelihood. Which brings up another point that I think we discussed. If Rick does not provide a output in version 3.3, you could enter it as an estiamted parameter. It may mess with the MCMC, but I believe past tests showed that it worked the same.

On Mon, Feb 29, 2016 at 2:56 PM, cgrandin notifications@github.com wrote:

Should it go in the parameters section of the tables, or derived quantities?

— Reply to this email directly or view it on GitHub https://github.com/cgrandin/hake-assessment/issues/162#issuecomment-190437353 .

iantaylor-NOAA commented 8 years ago

I just assigned this to the 2017 milestone so that we can remember to go back through the thread and remember to either remove hardwired values or replace with new calculations.

cgrandin commented 8 years ago

Ok, I have to leave now, but will get this done later today or first thing tomorrow. It's straightforward, but involves a little table formatting.

iantaylor-NOAA commented 8 years ago

Thanks for all your work today, Chris. Like I said in the morning, we'll get this done in record time even if it takes a few days.

On Mon, Feb 29, 2016 at 3:07 PM, cgrandin notifications@github.com wrote:

Ok, I have to leave now, but will get this done later today or first thing tomorrow. It's straightforward, but involves a little table formatting.

— Reply to this email directly or view it on GitHub https://github.com/cgrandin/hake-assessment/issues/162#issuecomment-190440194 .

cgrandin commented 8 years ago

This is finished for this year. I'm closing it, although it will still appear on the 2017 milestones, so we can reopen next year, or once Rick fixes the output of SS3.