search

HakaiInstitute / hakai-ctd-qc

Series of tests applied to the Hakai CTD profile data based on the QARTOD tests and other Hakai Specific ones.
0 stars 0 forks source link

Test PAR Shawdow #4

Closed JessyBarrette closed 3 years ago

JessyBarrette commented 3 years ago

Test for PAR shadow

@jdelbel You may be interested in the way I've used to detect PAR profiles affected by a shadow. See code below: https://github.com/HakaiInstitute/hakai-profile-qaqc/blob/32e4880d1f768113df66a5991f7c2fd256c303ee/hakai_qc/hakai_tests.py#L144-L170

Essentially:

  1. Sort each profiles by depth in reverse order
  2. Apply a cummax for each individual profiles above a PAR threshold (in this case 5). Get from the deepest to shallowest the highest value measured for each profiles
  3. If PAR value is lower than the maximum measured so far True. Flag data as SUSPECT False. Flag data as GOOD
jdelbel commented 3 years ago

Thanks Jessy. Very helpful. Some of my analysis of z1% showed even the good profiles may have issues, but will keep working on it.

On Mon, 25 Oct 2021 at 14:41, Jessy Barrette @.***> wrote:

Test for PAR shadow

@jdelbel https://github.com/jdelbel You may be interested in the way I've used to detect PAR profiles affected by a shadow. See code below:

https://github.com/HakaiInstitute/hakai-profile-qaqc/blob/32e4880d1f768113df66a5991f7c2fd256c303ee/hakai_qc/hakai_tests.py#L144-L170

Essentially:

  1. Sort each profiles by depth in reverse order
  2. Apply a cummax for each individual profiles above a PAR threshold (in this case 5). Get from the deepest to shallowest the highest value measured for each profiles
  3. If PAR value is lower than the maximum measured so far True. Flag data as SUSPECT False. Flag data as GOOD

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/HakaiInstitute/hakai-profile-qaqc/issues/4, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJPW44TXCYPBCD3RKLU5B5DUIXFIDANCNFSM5GWIU6WA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

-- Justin Del Bel Belluz, MSc. Research Technician - Bio-Optical Oceanography Hakai Institute 100 - 1002 Wharf Street Victoria, BC Canada V8W 1T4 www.hakai.org

JessyBarrette commented 3 years ago

yeah that method only detects the ones that are obviously wrong. not the ones slightly off.

Also you need to consider that the your 100% value can't in the air since a lof of the ambient light gets reflect by in the air and never make it through the air-sea interface. Using a 0m value is very likely in air PAR value since the sensor is ~1m above the pressure sensor.

On Fri, Oct 29, 2021 at 10:20 AM jdelbel @.***> wrote:

Thanks Jessy. Very helpful. Some of my analysis of z1% showed even the good profiles may have issues, but will keep working on it.

On Mon, 25 Oct 2021 at 14:41, Jessy Barrette @.***> wrote:

Test for PAR shadow

@jdelbel https://github.com/jdelbel You may be interested in the way I've used to detect PAR profiles affected by a shadow. See code below:

https://github.com/HakaiInstitute/hakai-profile-qaqc/blob/32e4880d1f768113df66a5991f7c2fd256c303ee/hakai_qc/hakai_tests.py#L144-L170

Essentially:

  1. Sort each profiles by depth in reverse order
  2. Apply a cummax for each individual profiles above a PAR threshold (in this case 5). Get from the deepest to shallowest the highest value measured for each profiles
  3. If PAR value is lower than the maximum measured so far True. Flag data as SUSPECT False. Flag data as GOOD

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/HakaiInstitute/hakai-profile-qaqc/issues/4, or unsubscribe < https://github.com/notifications/unsubscribe-auth/AJPW44TXCYPBCD3RKLU5B5DUIXFIDANCNFSM5GWIU6WA

. Triage notifications on the go with GitHub Mobile for iOS < https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675

or Android < https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub .

-- Justin Del Bel Belluz, MSc. Research Technician - Bio-Optical Oceanography Hakai Institute 100 - 1002 Wharf Street Victoria, BC Canada V8W 1T4 www.hakai.org

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/HakaiInstitute/hakai-profile-qaqc/issues/4#issuecomment-954913211, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHICYOPSLJ5HNWQYX7GZ7ULUJLJXJANCNFSM5GWIU6WA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

-- Jessy Barrette M.Sc. Marine Instrumentation Specialist Hakai Institute https://www.hakai.org/ | @.*** | (C) (250) 208-7806