search

pfmc-assessments / PacFIN.Utilities

R code to manipulate data from the PacFIN database for assessments
http://pfmc-assessments.github.io/PacFIN.Utilities
Other
7 stars 2 forks source link

Early years of length data being filtered out #63

Closed lee-qi closed 3 years ago

lee-qi commented 3 years ago

cleanPacFIN() resulted in early years of the sablefish fishery length comp data to be filtered out (i.e. the length column is shown as NA even though there is a value for FISH_LENGTH). I dug a tiny bit deeper into it, and it might be due to the fact that FISH_LENGTH_TYPE was U, but am unsure. Any chance this is a feature and not a bug?

It basically resulted in there being no length composition data for the sablefish fisheries from 1970-1978, when those were historically present in the assessment sensitivity run. Happy to share files as needed.

kellijohnson-NOAA commented 3 years ago

Thanks @lee-qi for reporting. I have access to the file, but can you share your call to cleanPacFIN() or are you just using the default settings for all parameters?

lee-qi commented 3 years ago

Ah good point. I used the code from the last assessment cycle, so that might have been why?

Pdata <- cleanPacFIN(Pdata = Pdat,
  keep_gears = c("HKL", "POT", "TWL"),
  keep_sample_type = c("", "C", "M"),
  keep_sample_method = "R",
  keep_length_type = c("", "A", "F"), # Double check this
  CLEAN = FALSE,
  spp = "sablefish")
kellijohnson-NOAA commented 3 years ago

I filtered the .RData object provided for this year using the code you have above, except I changed CLEAN = TRUE and I am getting samples with a length_cm > 0 in 1978. image

lee-qi commented 3 years ago

Right, thanks for checking that. Historically though that time series extended back to 1970. Before I updated the R package on my local machine, the same code had given those 1970 lengths back in March.

kellijohnson-NOAA commented 3 years ago

BINGO -- they have no expanded weight, so even if you change your call to cleanPacFIN to include keep_length_type = c("U", ...) they are still cleaned. Look at EXP_WT. These should not be included because we have no way of knowing how big the tow was to expand the sample, whereas previously I guessed a weight based on surrounding years.

lee-qi commented 3 years ago

Ah, perfect. So feature, and not a bug! Awesome, thanks for talking through this!