search

pfmc-assessments / PacFIN.Utilities

R code to manipulate data from the PacFIN database for assessments
http://pfmc-assessments.github.io/PacFIN.Utilities
Other
7 stars 2 forks source link

Missing INPFC column #34

Closed chantelwetzel-noaa closed 3 years ago

chantelwetzel-noaa commented 3 years ago

Historically there was a INPFC_AREA column which the cleanPacFIN function would then use to determine which records that were out of area and should be removed. The new bds pull does not have this column. We should determine if there a corresponding column in the comp table that we should be pulling or if the PSMFC_AREA column that is provided could provide the same filtering level (remove Puget Sound or Canadian samples) and that the cleanPacFIN code should be revised.

mhaltuch commented 3 years ago

Hi Chantel,

Back in the day some (all?) species did not have complete data entered for INPFC_AREA in the tables that were/are used to compile catches, but the spatial records for PSMFC_AREA were complete. If we compiled catches using INPFC_AREA then some landings would be missing. I believe that I learned this in the early petrale days and just transitioned to PSMFC_AREA to avoid the problem. Before changing to INFPC_AREA we should cross check the landings with PSMFC so make sure that we are not missing anything.

Cheers, Melissa

On Thu, Nov 5, 2020 at 5:02 PM Chantel Wetzel notifications@github.com wrote:

Historically there was a INPFC_AREA column which the cleanPacFIN function would then use to determine which records that were out of area and should be removed. The new bds pull does not have this column. We should determine if there a corresponding column in the comp table that we should be pulling or if the PSMFC_AREA column that is provided could provide the same filtering level (remove Puget Sound or Canadian samples) and that the cleanPacFIN code should be revised.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/nwfsc-assess/PacFIN.Utilities/issues/34, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACZOPK54UOGS4ONMTP3SBT3SONDH3ANCNFSM4TMCA7UQ .

-- Cheers, Melissa

chantelwetzel-noaa commented 3 years ago

@mhaltuch I think you are correct about catches being missing if you used INPFC_AREA which is why I have suggested that we never give those out in terms of the PacFIN catch pull. However, this post is about the bds data where by default we are removing bds records based on their INPFC_AREA in PacFIN.Utilities. I suspect this part of the code arose based on how someone's personal script back in the day was filtering. The larger issue at hand here is us needed to move our bds pulling process from old no longer updated PacFIN tables to their new tables which is requiring us to do some testing and adjusting to ensure that we are getting the same product in the end. I agree that PSMFC_AREA may be the better field to filter by but we should check that this results in the same removals as the older approach across species.

mhaltuch commented 3 years ago

Ah, yes, thanks for the redirect to the bds. Thanks for all of your work on this.

On Fri, Nov 6, 2020 at 5:17 AM Chantel Wetzel notifications@github.com wrote:

@mhaltuch https://github.com/mhaltuch I think you are correct about catches being missing if you used INPFC_AREA which is why I have suggested that we never give those out in terms of the PacFIN catch pull. However, this post is about the bds data where by default we are removing bds records based on their INPFC_AREA in PacFIN.Utilities. I suspect this part of the code arose based on how someone's personal script back in the day was filtering. The larger issue at hand here is us needed to move our bds pulling process from old no longer updated PacFIN tables to their new tables which is requiring us to do some testing and adjusting to ensure that we are getting the same product in the end. I agree that PSMFC_AREA may be the better field to filter by but we should check that this results in the same removals as the older approach across species.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/nwfsc-assess/PacFIN.Utilities/issues/34#issuecomment-723075352, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACZOPKZU3QVWWPPIE3Y3QTLSOPZLJANCNFSM4TMCA7UQ .

-- Cheers, Melissa

kellijohnson-NOAA commented 3 years ago

@chantelwetzel-noaa can we close this or are there still coding tasks that we need to do prior to closing it?

chantelwetzel-noaa commented 3 years ago

I think we have found a comparable approach using PSMFC areas