search

noaa-afsc / berchukseals-haulout

Repository and research compendium in support of the manuscript "Spring haul-out behavior of seals in the Bering and Chukchi seas." Maintained by Josh London (@jmlondon / josh.london@noaa.gov)
Other
5 stars 0 forks source link

underlying data inconsistencies #47

Closed jmlondon closed 1 year ago

jmlondon commented 1 year ago

I've identified some data inconsistencies and data not included in the analysis that should be included. The main reason for this seems tied to the deployments not having a specified 'end date' in our internal telemetry database.

At a minimum, the issue impacts the following spenos: PL2019_9001,PL2019_9002,PL2019_9003,PL2019_9004,PL2019_9005,PL2019_9006, EB2019_9001

Will work with @staciekoslovsky-noaa to resolve in the database and proceed with additional data integrity tests.

jmlondon commented 1 year ago

corrections to the internal NOAA/PEP database and improvements to the queries seem to have resolved this issue.

this was addressed by providing end dates for the mentioned deployments and also adjusting the SQL query to not pull data from any deployments where qa_status is 'tag actively transmitting'. previously, I had a step that relied on the between() function to filter data outside of the start/end dates. But, when end date is NA, this approach fails.

see 67ff868a5d4d0b9d426502f57f479175dd5d286e