fleet-specific F method

[k-doering-NOAA]

Imported from redmine, Issue #53098 Opened by @RickMethot on 2018-07-25 Status when imported: In Progress

It seems advantageous to allow fleet-specific F method so that only fleets with high F will need to use F as parameters

[k-doering-NOAA]

comment from @RickMethot on 2018-12-18: it would be useful to have fleet-specific controls for F_method, starting F value, and starting phase.

The code seems amenable to allowing for some fleets to do F as parameters and other fleets as hybrid, but probably easiest to define all fleets to have F as parameters and then to just be fleet-specific for the phase in which estimation switched from hybrid to F_as_parameter

because this would require I/O change, do it as F_method #4. The I/O already is conditional on F_method, so this is easy

[k-doering-NOAA]

comment from @RickMethot on 2019-02-20: The internal method for conversion of a list of F_parms to Hrate(t,f) is quite flexible. So it seems quite feasible to create a new F_method that does F as parameters for only certain fleets. Or a F_method that only does F as parameter for (f,t) with input catch exceeding a user-specified threshold. In execution, hybrid F would be used for all in the early phases and then the selected Hrate(f,t) would switch to parms at a specified phase.

[k-doering-NOAA]

comment from @RickMethot on 2020-04-09: Fleet-specific F_Method Motivation:

1. bycatch fleets must use F_Method=2 (parameters), so use of any bycatch fleet currently precludes use of hybrid F for other fleets
2. F as parameter is most needed for high F situations, so good if could be restricted to high F fleets
3. Extending that logic, might be desirable to have F as parameter only after some fleet-specific start year

Plan is to allow for a new option #4 which will allow for fleet-specific switch for the phase in which switch from hybrid to parameter occurs

Aside from the I/O, there need to be changes in: SS_proced.tpl Here the change is for the population of the Hrate(f,t) matrix from the parameter vector - F_rate(g) Need a smart way to do this in a fleet and phase specific way that does not involve IF statements

SS_popdyn.tpl here Hrate(f,t) is always used as the basis for the calculations the values for Hrate are created here if hybrid is used, or have already been assigned in SS_proced so the decision to calc the value via hybrid now needs to be fleet-specific. it already is fleet-specific conditioned on fleet_type==1 because bycatch and survey fleets cannot do hybrid so can switch to conditioning on F_Method(f)==3 for fleet-specific use of hybrid

also need to deal with logic around F_Method_Use. This drives the CASE statement for F_Method, so probably need to retain this so that if any fleet uses hybrid in current phase it will uses this pathway, else do all fleets as parameters

[Rick-Methot-NOAA]

New thought: The identifier fleet_type(f) is already fleet-specific. fleet_type is closely associated with F_method as fleet_type=bycatch requires F_method=2, and fleet_type=3 (survey) requires no F_method. Also, there is desire for fleet_type=4=predators which would have its own unique F_method.

SO: rather than create fleet-specific I/O for F_method, instead elaborate fleet_type(f) to have these options: 3=survey 11=Pope's 12.XX=F parameter with hybrid for early phases (allows for hybrid to be invoked for all or none phases, with X being the phase) 13=predator

Because there is much switching already that uses fleet_type(f) it may be good to read the new approach into fleet_type_rd(f), then parse it into fleet_type(f) using the old 1,2,3 codes and a new F_method(f) @nschindler-noaa Neal - do you have ideas about this restructuring?

Note that there also is F_Method_use that implements the switch from hybrid to param for Fmethod 2. But in ss_popdyn, there are still some switches beginning at line 1363 below that are incorrectly referring to F_method itself: 897 // SS_Label_Info_24.3.3 #Do fishing mortality using switch(F_method) 902 switch (F_Method_use) 905 case 3: // hybrid F_method 1037 } // end hybrid F_Method 1039 case 2: // continuous F_method 1079 case 1: // F_Method is Pope's approximation 1081 // SS_Label_Info_24.3.3.1 #Use F_Method=1 for Pope's approximation 1159 } // end F_Method switch 1363 if(F_Method==1) 1603 if(F_Method==1) // Pope's approx 1740 if(F_Method==1) 1776 if(F_Method>=2) 1793 else // F_method=1 1863 if(F_Method==1)

Another point: need to deal with the I/O for input of of F_setup and F_setup2 which are detailed F inputs, etc.

[nschindler-noaa]

My thoughts aside from this being an input/output change To allow for greatest flexibility, use different digits for different purposes. First digit for the type of fleet and second for the method: e.g. 13 would be a fishing fleet using the hybrid method, 40 would be a predator with its unique method. Some combinations would not be allowed, so some checks would be needed. That way you can add fleet types and methods without needing to rewrite anything (except how to handle the new values). I think this would be simpler for the users as well.

On Mon, Jun 28, 2021 at 11:16 AM Richard Methot @.***> wrote:

New thought: The identifier fleet_type(f) is already fleet-specific. fleet_type is closely associated with F_method as fleet_type=bycatch requires F_method=2, and fleet_type=3 (survey) requires no F_method. Also, there is desire for fleet_type=4=predators which would have its own unique F_method.

SO: rather than create fleet-specific I/O for F_method, instead elaborate fleet_type(f) to have these options: 3=survey 11=Pope's 12.XX=F parameter with hybrid for early phases (allows for hybrid to be invoked for all or none phases, with X being the phase) 13=predator

Because there is much switching already that uses fleet_type(f) it may be good to read the new approach into fleet_type_rd(f), then parse it into fleet_type(f) using the old 1,2,3 codes and a new F_method(f) @nschindler-noaa https://github.com/nschindler-noaa Neal - do you have ideas about this restructuring?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/nmfs-stock-synthesis/stock-synthesis/issues/33#issuecomment-869909708, or unsubscribe https://github.com/notifications/unsubscribe-auth/ARUZACRSVTUWENT24D6JF23TVC36HANCNFSM4TLVK7QA .

--

Neal Schindler

Programmer Analyst, Caelum

NWFSC (206) 860-3407

cell (206) 909-0505

=(((< >=(((< >=(((`<

=(((`<

[Rick-Methot-NOAA]

Good suggestion Neal. Better than mine.

On further thought, I think it best to keep F_method consolidated in control file, so F_Method scalar input = 9 will then invoke fleet-specific F_method. Noting that F_Method=1 must be applied to all or no fishing fleets, so this new flexibility will just apply to the fleets with F_method=2, 3, or predator

[Rick-Methot-NOAA]

A substantial revision to everything about Fparms and their indexing has been pushed. Do not do a pull request yet. It works for an example that uses Fmethod=2 and has a bycatch fleet (using cobia example) lots of extra cout statements still need cleaning up. Next step is to finish implementation of Fmethod 4 which will use the revised innards to allow for fleet-specific Fmethod.

[Rick-Methot-NOAA]

Got it. Syntax looks like this to get F_Method 4, which I now consider a preferred option over 2 and 3.

4 # F_Method:  1=Pope; 2=instan. F; 3=hybrid; 4=fleet-specific parm/hybrid (#4 is recommended)
2.9 # max F or harvest rate, depends on F_Method
# for Fmethod 1; no additional F input needed
# for Fmethod=2; read overall start F value; overall phase; N detailed inputs to read
# for Fmethod=3; read N iterations for tuning for Fmethod 3
# for Fmethod=4; read list of fleets needing parameters; syntax is:  fleet, F_starting_value (if start_PH=1), first PH for parms (99 to stay in hybrid)
# for Fmethod=4; then read N tuning loops for hybrid fleets 2-3 normally enough
#Fleet start_F first_parm_phase
 1 0.05 2
 2 0.05 99
 3 0.05 1
 -9999 1 1
4 #_number of tuning loops for hybrid fleets; 4 good; 3 faster

It works well for a model (cobia) that once was Fmethod=2 for all fleets, one of which was a bycatch fleet.

[Rick-Methot-NOAA]

Realizing that the previous F_Detail feature should be retained for F_Method_4. It will be straightforward to allow it for F_Method 4. Also note that the F_Detail feature reads yr, seas and if yr is negative it fills forward for all remaining years. But it does so for just that season, not all seasons. A user can compensate by doing Nseas entries to cover all of the seasons. While a -season entry might be used to invoke filling all seasons, I think I will leave it as is and document the current situation.

[k-doering-NOAA]

@Rick-Methot-NOAA , I hope you don't mind I edited your comment! adding three backticks before and after the input marks it as code, which means github markdown formatting won't be automatically be applied (the pound sign indicates a heading, the same as an h1 html tag).

Also, do you expect estimated F values to change for models using F methods 1-3 ?

[Rick-Methot-NOAA]

Thanks. I know that but forgot again.

Existing models "should" not be affected. However, I did improve/tweak the hybrid algorithm so it is possible for slight differences to occur for F_Method 3. Please let me know. The internal code for F_Methods 2, 3, 4 is nearly all shared. All that really changed was the added indexing needed to allow fleet-specific F_Method decision within the popdyn calculations. In doing so, I made the hybrid adjustment based only on the fleets being adjusted, not the total catch across all fleets. That is the improvement I mentioned. I temporarily added some outputs so I could track each iteration of the hybrid tuning. In the high F, 3 fleet cobia example; 4 tuning loops gets match to input catch within 0.0001 and 3 gets close enough. 5 would slow the model with extra calculations for trivial gain.

On Tue, Aug 3, 2021 at 7:55 AM Kathryn Doering @.***> wrote:

@Rick-Methot-NOAA https://github.com/Rick-Methot-NOAA , I hope you don't mind I edited your comment! adding three backticks before and after the input marks it as code, which means github markdown formatting won't be automatically be applied (the pound sign indicates a heading, the same as an h1 html tag).

Also, do you expect estimated F values to change for models using F methods 1-3 ?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/nmfs-stock-synthesis/stock-synthesis/issues/33#issuecomment-891915298, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABPV4IFGKHN4WMUHZSDIQOTT277N5ANCNFSM4TLVK7QA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email .

[Rick-Methot-NOAA]

latest test (commit = fix errant break) only had minor differences for vermillion test set. So I conclude that the current code causes no problems for models that use F_Method 2 or 3.

Next step will be to convert some additional test cases over to F_Method 4 and compare to performance with original F_Method 2 or 3. Note that with F_Method 4, a fleet can be instructed to continue to use hybrid for all phases, so should be equivalent to F Method 3. Also note that F_Method 2 already starts all fleets with hybrid then converts all to parameters at specified phase. Only difference with F_Method 4 is that the change to parameters is fleet-specific. Note also that if a fleet is instructed to stay in hybrid for all phases, the F parameters are not even created for that fleet.

[k-doering-NOAA]

Just to make this more quantitative, I think vermillion total likelihood changed by ~1% when running with no estimation:

@Rick-Methot-NOAA are you sure this is ok, given this is running with no estimation? I only ask because this is the only F method 2 model in our test set, so the changes flagged here might be indicitive of changes for other F method 2 models.

Let me know if you would like me to help with performance testing! I think we could start with one-off tests, then figure out when F method 4 models we want to keep in the test set (I think you are right that it probably makes sense to convert at least 1 existing model to F method 4 rather than add a whole new model to the test set).

[Rick-Methot-NOAA]

All of my testing was with cobia, which is a Fmethod 2 model. I will double-check the no estimation with that one and also will take a look at vermillion. I thought that vermillion was already one of our problematic examples, but I will indeed look more closely.

[Rick-Methot-NOAA]

Kathryn was right; the code logic needed more work. Now nearly ready for a pull request. Max willing to be beta tester.

[k-doering-NOAA]

Glad to hear you figured it out! Great to hear Max is willing to help test; perhaps he could test out the branch before it gets merged into main?

[Rick-Methot-NOAA]

The input format for the F_Method 4 is: 4 F_Method: 1=Pope; 2=instan. F; 3=hybrid; 4=fleet-specific parm/hybrid (#4 is recommended) 3 max F or harvest rate, depends on F_Method for Fmethod 1; no additional F input needed for Fmethod=2; read overall start F value; overall phase; N detailed inputs to read for Fmethod=3; read N iterations for tuning for Fmethod 3 for Fmethod=4; read list of fleets needing parameters; syntax is: fleet, F_starting_value (if start_PH=1), first PH for parms (99 to stay in hybrid) for Fmethod=4; then read N tuning loops for hybrid fleets 2-3 normally enough Fleet start_F first_parm_phase 1 0.05 5 2 0.05 5 3 0.1 5 4 0.04 1 -999 1 1 2 #_number of tuning loops for hybrid fleets; 4 good; 3 faster

I did some performance testing using the vermilion example, which has 3 catch fleets and one bycatch fleet.

3.30.17 F_Method=2 starting Fparms in Phase=1 for all 4 fleets 1154 iterations, 42 sec, logL 359.90

3.30.18.beta; F_Method=2 starting in Phase=1 for all 4 fleets 1154 iterations, 44 sec, logL 359.90

3.30.18.beta; F_Method=4 starting in Phase=1 for all 4 fleets; 1176 iterations, 45 sec, logL 359.90 starting in Phase=2 for fleets 2-4 (so hybrid in phase 1); 781 iterations, 30 sec, logL 359.90 starting in Phase=99 for fleets 2-4 (so hybrid in all phases); 552 iterations, 23 sec, logL 363.6 note different logL starting in Phase=5 for fleets 2-4 (so hybrid in phase 1-4); 765 iterations, 31 sec, logL 359.90

above used 4 tuning loops for hybrid phases reducing to 3 or even 2 had negligible impact on runtime or number of total iterations. all end up with parameters in the final phase.

[k-doering-NOAA]

See PR #195 for the code

[Rick-Methot-NOAA]

In final testing, we found a situation where hybrid F through the final phase causes a worse fit than use of F as parameters. This was with red grouper where catch se is high (0.15) and some fleets have discard data to fit also. with hybrid for fleet3, the F's cause the expected catch to match the observed catch, which is much better than expected given se = 0.15 on the catch. These forced F's cause fit to other data, including fleet 3 discard, to be worse and the overall logL is worse With fleet 3 F as parameters, SS can allow some lack of fit to the catch in order to find better values for fitting all data. Let's add a cautionary note in the manual about hybrid usage when there is imprecise catch data and discard data exist.

[k-doering-NOAA]

@Rick-Methot-NOAA , thank you for putting this together, I was able to switch to using F method 4 easily.

I did have a question about one section of the input:

for Fmethod=4; read list of fleets needing parameters; syntax is: fleet, F_starting_value (if start_PH=1), first PH for parms (99 to stay in hybrid)
for Fmethod=4; then read N tuning loops for hybrid fleets 2-3 normally enough
Fleet start_F first_parm_phase
1 0.05 5
2 0.05 5
3 0.1 5
4 0.04 1
-999 1 1

When I removed the line for fleet 4, I got a fatal warning:

Fatal_input_error; not all bycatch fleets have been included in Fparm list

However, when I removed the line for fleet 3, the model ran through with no fatal error. Is this exected? Are inputs only required for bycatch fleets in this section, but optional for fishing fleets?

[Rick-Methot-NOAA]

good testing. Your interpretation is correct. bycatch fleets must be assigned to use parameters in phase 1 all other fishing fleets will be hybrid (essentially the 99 option) unless explicitly given a parameter conversion phase.

[k-doering-NOAA]

Great, thanks! I'll make sure that explanation makes it into the manual.

[k-doering-NOAA]

@Rick-Methot-NOAA , while I was making changes to r4ss read/write functions, I realized that in all other cases where data being read does not have a defined number of rows to read, the last row ends with -9999, in the first column not -999.

Could this be changed to make it more consistent?

[Rick-Methot-NOAA]

Yes! Good catch. Here are instances in readdata and then readcontrol that should be addressed. Let's settle on -999, change write_ss_new to output -999, not -9999, then change all the internal checks to look for <=-999 to catch -999 and -9999.

404 if(tempvec(1)==-9999.) ender=1; 434 if(g==-999) 440 if(y>=styr-1 && y<=endyr && (g==-999 || g>=styr)) // observation is in date range 566 if(tempvec(1)==-9999.) ender=1; 660 Svy_maxval=-999999999.; 787 disc_maxval=-999999999.; 810 if(tempvec(1)==-9999.) ender=1; 965 if(tempvec(1)==-9999.) ender=1; 1363 if(tempvec(1)==-9999.) ender=1; 1846 if(tempvec(1)==-9999.) ender=1; 2230 if(tempvec(1)==-9999.) ender=1; 2394 if(tempvec(1)==-9999.) ender=1; 2966 echoinput<<"codes: -999 means start year; >0 is an actual year; <=0 is relative to endyr"<<endl; 2975 if(Bmark_Yr_rd(i)==-999) 3085 if(Fcast_yr(i)==-999) 3207 "Terminate with -9999 for season"<<endl<<"Will be re-scaled to sum to 1.0"<<endl; 3213 if(tempvec(1)==-9999.) 3288 echoinput<<endl<<"# next read list of fleet ID and max annual catch; end with fleet=-9999"<<endl; 3296 if(tempvec(1)==-9999.) 3310 echoinput<<endl<<"Read list of area ID and max annual catch; end with area=-9999"<<endl; 3318 if(tempvec(1)==-9999.) 3329 echoinput<<endl<<"Read list of fleet ID and assignment to allocation group; end with fleet ID=-9999"<<endl; 3338 if(tempvec(1)==-9999.) 3362 if(tempvec(1)==-9999.) ender=1; 3413 if(tempvec(1)==-9999.) ender=1;

in readcontrol: 2411 if(j>0 && F_setup2(k,6)!=-999){ 2415 if(j>0 && F_setup2(k,5)!=-999) catch_se(t,f)=F_setup2(k,5); // reset the se for this observation 3373 {selparm_1(k,3)=-999.;} // hardwire to force to 0.0 3392 {selparm_1(k,4)=-999.;} // hardwire to force to 0.0 3426 selparm_1(parmcount+1,7)=-99; 3427 selparm_1(parmcount+2,7)=-99; 3451 selparm_1(parmcount+1,7)=-99; 3452 selparm_1(parmcount+2,7)=-99; 3471 selparm_1(parmcount+1,7)=-99; 3472 selparm_1(parmcount+2,7)=-99; 4070 echoinput<<" read var_adjust list until -9999"<<endl; 4077 if(tempvec(1)==-9999.) ender=1; 4164 echoinput<<" read lambda changes list until -9999"<<endl; 4170 if(tempvec(1)==-9999.) ender=1; 4688 if(MGparm_PH(k)==-9999) {MGparm_RD(k)=prof_var(prof_var_cnt); prof_var_cnt+=1;} 4701 if(SR_parm_PH(j)==-9999) {SR_parm_1(j,3)=prof_var(prof_var_cnt); SR_parm_RD(j,3)=SR_parm_1(j,3); prof_var_cnt+=1;} 4805 if(init_F_PH(j)==-9999) {init_F_parm_1(j,3)=prof_var(prof_var_cnt); init_F_RD(j)=init_F_parm_1(j,3); prof_var_cnt++;} 4834 if(Q_parm_PH(f)==-9999) {Q_parm_1(f,3)=prof_var(prof_var_cnt); Q_parm_RD(f,3)=prof_var(prof_var_cnt); prof_var_cnt++;} 4926 selparm_PH(z)=-99; 4950 selparm_PH(Ip+p)=-99; 4960 selparm_PH(p)=-99; 4986 selparm_PH(p)=-99; 5008 if(selparm_PH(k)==-9999) {selparm_RD(k)=prof_var(prof_var_cnt); selparm_1(k)=prof_var(prof_var_cnt); prof_var_cnt++;} 5600 if(tempvec(1)==-9999.) ender=1; 5636 {WTage_emp(t,g,f,a)=-9999.;} 5673 if(WTage_emp(t,g,f,a)==-9999.)

On Tue, Sep 28, 2021 at 7:40 AM Kathryn Doering @.***> wrote:

@Rick-Methot-NOAA https://github.com/Rick-Methot-NOAA , while I was making changes to r4ss read/write functions, I realized that in all other cases where data being read does not have a defined number of rows to read, the last row ends with -9999, in the first column not -999.

Could this be changed to make it more consistent?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/nmfs-stock-synthesis/stock-synthesis/issues/33#issuecomment-929296007, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABPV4IHUWMPATZZETGX5Z2TUEHHXJANCNFSM4TLVK7QA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

[k-doering-NOAA]

wait, I think this change is simpler if we just change the new fleet-specific F method ending from -999 to -9999. I think the only thing that would need to be done is change that one, because all the others already use -9999, to my knowledge.

[Rick-Methot-NOAA]

good. I will make that change.

[k-doering-NOAA]

Great!

[Rick-Methot-NOAA]

The readcontrol code for F_Method list was already generic and actually only checked for any negative number. So just update the examples and manual to say that ender is indicated by -9999 for consistency.

[nschindler-noaa]

Well, I set the SSI to check for -9999 out of habit, so I guess I came out ahead this time!

On Tue, Sep 28, 2021 at 11:32 AM Richard Methot @.***> wrote:

The readcontrol code for F_Method list was already generic and actually only checked for any negative number. So just update the examples and manual to say that ender is indicated by -9999 for consistency.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/nmfs-stock-synthesis/stock-synthesis/issues/33#issuecomment-929518231, or unsubscribe https://github.com/notifications/unsubscribe-auth/ARUZACX4GVGXGDREFTIFV4LUEICZXANCNFSM4TLVK7QA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

--

Neal Schindler

Programmer Analyst, Caelum

NWFSC (206) 860-3407

cell (206) 909-0505

=(((< >=(((< >=(((`<

=(((`<

[Rick-Methot-NOAA]

totally spiffed up control.ss_new now looks like below. Note that for F_Method 2, the 3 inputs are now on separate lines:

2 # F_Method:  1=Pope midseason rate; 2=F as parameter; 3=F as hybrid; 4=fleet-specific parm/hybrid (#4 is superset of #2 and #3 and is recommended)
3 # max F (methods 2-4) or harvest fraction (method 1)
0.05 # overall start F value (all fleets; used if start phase = 1 and not reading parfile)
1 # start phase for parms (does hybrid in early phases)
0 # N detailed inputs to read
# detailed setup for F_Method=2; -Yr to fill remaining years
#Fleet Yr Seas F_value se phase
#

and for F_Method 4

4 # F_Method:  1=Pope midseason rate; 2=F as parameter; 3=F as hybrid; 4=fleet-specific parm/hybrid (#4 is superset of #2 and #3 and is recommended)
3 # max F (methods 2-4) or harvest fraction (method 1)
# read list of fleets that do F as parameter; unlisted fleets stay hybrid, bycatch fleets must be included with start_PH=1, high F fleets should switch early
# (A) fleet, (B) F_initial_value (used if start_PH=1), (C) start_PH for parms (99 to stay in hybrid)
# (A) (B) (C)  (terminate list with -9999 for fleet)
 1 0.05 5 # CM_E
 2 0.05 5 # CM_W
 3 0.1 5 # REC
 4 0.04 1 # SMP_BYC
-9999 1 1 # end of list
2 #_number of loops for hybrid tuning; 4 good; 3 faster; 2 enough if switching to parms is enabled