search

afsc-gap-products / gap_products

This repository supports code used to create tables in the GAP_PRODUCTS Oracle schema. These tables include the master production tables, tables shared with AKFIN, and tables publicly shared on FOSS.
https://afsc-gap-products.github.io/gap_products/
Creative Commons Zero v1.0 Universal
5 stars 5 forks source link

1980s GOA data in SPLIT_FRACTIONS #32

Closed JaneSullivan-NOAA closed 2 months ago

JaneSullivan-NOAA commented 4 months ago

Question

You all have removed almost all of the pre-1990 GOA bottom trawl survey data, however it still exists in the AKFIN_SPLIT_FRACTIONS table. Is this intentional?

JaneSullivan-NOAA commented 4 months ago

Also 2023 EGOA fractions are not yet on AKFIN.

JaneSullivan-NOAA commented 4 months ago

image

zoyafuso-NOAA commented 3 months ago

The creation of the split fractions table is a separate process from the other gap_products tables. The split fractions table is mirrored from a table in the GOA schema in the AFSC database, so it still has those data from the 80s that we’re recommending not to use. The version of the split fractions table the gap_products schema in the akfin database has not been refreshed since June 2023 although the version on the GOA schema in the AFSC data is current up to 2023.

So is the proposed solution here to refresh the akfin version of the split fraction table to include the 2023 data and to continue to recommend not using the 1980s data as we have been establishing for the main gap_products tables?

zoyafuso-NOAA commented 3 months ago

@Ned-Laman-NOAA ,

The split fractions table is outside the scope of the GAP_PRODUCTS tables that we transfer to AKFIN. The AKFIN version of the split fractions table has not been updated since June 2023. Do you mind sending AKFIN the current version of split fractions table that is in the GOA schema? The current version should have the 2023 estimates and it still contains those 1980s data that we recently "unstandardized" from our haul database. Maybe @Ned-Laman-NOAA can remove those 1980s data before sending over the table?

Sorry for the delay on this, Jane.

JaneSullivan-NOAA commented 3 months ago

No worries, @zoyafuso-NOAA. Thanks for the update. Will the new SPLIT_FRACTIONS not be updated with other GAP_PRODUCTS tables on AKFIN? I don't really know what you mean by "outside the scope."

Unfortunately I have a conflict with office hours again this week. Sorry to miss out.

zoyafuso-NOAA commented 2 months ago

Outside the scope behind the scenes means that the split-fraction table is calculated outside of the workflow that creates the main data product tables in the GAP_PRODUCTS schema (GAP_PRODUCTS.CPUE, GAP_PRODUCTS.BIOMASS, GAP_PRODUCTS.SIZECOMP, GAP_PRODUCTS.AGECOMP). Essentially, Ned runs a separate SQL query that creates the table within the GOA schema and then pushes that table to AKFIN. We have discussed here the possibility of not updating this table anymore and instead using the route of the afsc-gap-products/data-requests repo to obtain these estimates because 1) it is a non-standard data product and 2) we can easily replicate the table using the gapindex R package and a simple vignette script.

Zooming out, the new 2025 GOA strata are contained within the NMFS areas (610 -> 650) and so going forth, the split-fraction table will not apply to GOA 2025-on biomass estimates because strata will be separated and not split by the 140 longitude line (which conveniently is the boundary that separates 640 and 650). At that point, the split-fraction table will only apply to GOA years 1990-2023 and thus will not need to be updated from 2025-on.

Out of curiosity, what do you use the split fraction table for?

Ned-Laman-NOAA commented 2 months ago

Good morning and sorry that I'm slow to respond here; just busy...no other excuse.

I just looked at AKFIN and it looks to me like the EW Fractions for GOA are there from 1984 through 2023: Capture Is there a specific species or management group that Jane is looking for and not finding?

Since this table is now deprecated (post-2023), I'm not inclined to modify it to remove non-standard survey years (i.e., pre-1990) and users of these data need to be aware that the 1984 and and 1987 surveys were fished with different gear and aren't directly comparable to the surveys that followed because of assumed differences in catch efficiency of the different gear types.

I'm also curious what split fraction data are used for and realize I've always assumed that these objects existed to mimic the NMFS management zones which will now be installed in the 2025 and future stratification of the Gulf survey area.