search

HakaiInstitute / hakai-ctd-qc

Series of tests applied to the Hakai CTD profile data based on the QARTOD tests and other Hakai Specific ones.
0 stars 0 forks source link

Incorrect station locations within datasets #22

Open JessyBarrette opened 1 year ago

JessyBarrette commented 1 year ago

@ah-hakai

incorrect station location (Jan. 24, 2023)

Problem: Sometimes station QU39 is located (by map) at the Quadra Island Field station. This was caught by JB and AH during a review of the research ready dataset. However, it was agreed that it was inappropriate for the reviewer to change the station name or location, unless they were on that specific survey. It would be better if the annotation step required a correct station name prior to permitting the annotated file from getting into the databased.

Action (Jan. 26, 2023):

Front-end issue, Matt Foster has been notified, requested to not permit annotation to proceed unless the station name and station location have been reconciled.

JessyBarrette commented 1 year ago

@ah-hakai I ran the updated version of the station list on our development database. You can access the results through the hakai development portal

It seems a lot better with just a few problematic stations: I can the following query on our database to retrive the flag casts and stations:

select station, count(x.hakai_id) as n_profiles, string_agg(x.hakai_id,',') as hakai_id_list from ( 
select distinct station, hakai_id  FROM ctd.ctd_file_cast_data
WHERE temperature_flag LIKE '%depth_in_station_range_test%'
group by station,hakai_id) as x
group by station
order by n_profiles desc,station asc;

The results suggest that maybe the following stations are too shallow within our reference:

station n_profiles hakai_id_list
KWY02 13 018066_2014-04-03T20:40:53.500Z,018066_2014-04-22T19:50:10.333Z,018066_2014-05-09T19:50:42.500Z,018066_2014-05-14T18:26:07.166Z,018066_2014-06-01T18:42:06.333Z,018066_2014-06-06T18:57:28.000Z,018066_2014-06-22T19:21:30.000Z,018066_2014-06-27T19:18:21.834Z,018066_2014-08-03T18:24:03.500Z,080217_2014-08-11T10:50:51.667Z,080217_2014-08-25T12:21:02.667Z,080217_2014-08-29T11:27:44.000Z,080217_2014-09-19T18:04:01.833Z
NMU01 11 018032_2013-06-01T19:35:33.000Z,018032_2013-07-14T19:32:14.667Z,018032_2013-08-12T19:45:34.000Z,018032_2015-03-17T20:37:03.667Z,018032_2015-04-29T19:23:10.334Z,018066_2013-04-19T20:55:56.834Z,018066_2014-04-17T19:28:57.000Z,018066_2014-04-24T22:57:30.501Z,018066_2014-06-02T21:10:00.499Z,018066_2014-07-15T22:57:48.167Z,080217_2014-03-29T10:03:02.500Z
D27 7 065679_2018-05-10T17:10:56.333Z,065679_2018-05-23T19:51:53.667Z,065679_2018-05-29T18:04:15.500Z,065679_2018-06-12T17:02:57.000Z,065679_2018-06-19T15:48:19.833Z,065679_2018-06-26T18:25:16.500Z,065679_2018-07-04T15:52:46.667Z
D22 6 010856_2017-05-23T15:39:39.833Z,065679_2018-05-15T14:06:15.167Z,065679_2018-06-12T21:04:40.666Z,065679_2018-06-19T19:59:10.500Z,080217_2019-05-26T17:44:14.667Z,080217_2019-07-09T17:03:43.500Z
QU4 4 018032_2014-08-28T17:35:08.334Z,018066_2014-08-14T18:13:11.501Z,080217_2014-07-24T19:26:58.167Z,080217_2014-07-30T18:43:19.500Z
QU2 3 018032_2014-08-28T17:03:58.667Z,018066_2014-08-14T17:40:38.667Z,080217_2014-07-30T18:12:07.333Z
BED10 2 065679_2019-08-28T15:13:00.500Z,080217_2019-07-03T15:02:35.166Z
JS6 2 065679_2015-05-05T19:05:54.000Z,065679_2015-07-14T18:30:38.167Z
D35 1 080217_2019-05-14T18:30:40.166Z
DI18 1 01907674_2016-05-06T17:10:02Z
FZH01 1 018032_2017-02-23T20:35:14.833Z
HKP03 1 018066_2014-03-27T16:35:12.000Z
JS2 1 065679_2015-05-05T19:30:48.667Z
KC12 1 018066_2013-04-17T21:01:50.833Z
KFPS08 1 018032_2013-07-20T16:34:30.333Z
MG3 1 018066_2017-07-20T20:26:34.167Z
NLU03 1 018066_2014-04-05T18:42:45.667Z
QSD05 1 018032_2015-07-04T20:34:25.667Z
QU17 1 018066_2014-10-09T20:00:39.500Z
QU28 1 018066_2014-08-15T18:49:25.501Z
QU32 1 080217_2015-03-16T17:39:06.500Z
QU5 1 080217_2017-01-05T19:52:15.333Z
ah-hakai commented 1 year ago

Hi Jessy. I'm not exactly sure what the problem is here. Are the stations you listed flagged because they have measurements at depths that exceed the bottom depths from the new bathymetry?

On Wed, Feb 22, 2023 at 10:26 AM Jessy Barrette @.***> wrote:

@ah-hakai https://github.com/ah-hakai I ran the updated version of the station list on our development database. You can access the results through the hakai development portal https://goose.hakai.org/portal2

It seems a lot better with just a few problematic stations: I can the following query on our database to retrive the flag casts and stations:

select station, count(x.hakai_id) as n_profiles, string_agg(x.hakai_id,',') as hakai_id_list from ( select distinct station, hakai_id FROM ctd.ctd_file_cast_dataWHERE temperature_flag LIKE '%depth_in_station_range_test%'group by station,hakai_id) as xgroup by stationorder by n_profiles desc,station asc;

The results suggest that maybe the following stations are too shallow within our reference:

  • KWY02
  • NMU01
  • D27
  • D22
  • QU4
  • QU2

station n_profiles hakai_id_list KWY02 13 018066_2014-04-03T20:40:53.500Z,018066_2014-04-22T19:50:10.333Z,018066_2014-05-09T19:50:42.500Z,018066_2014-05-14T18:26:07.166Z,018066_2014-06-01T18:42:06.333Z,018066_2014-06-06T18:57:28.000Z,018066_2014-06-22T19:21:30.000Z,018066_2014-06-27T19:18:21.834Z,018066_2014-08-03T18:24:03.500Z,080217_2014-08-11T10:50:51.667Z,080217_2014-08-25T12:21:02.667Z,080217_2014-08-29T11:27:44.000Z,080217_2014-09-19T18:04:01.833Z NMU01 11 018032_2013-06-01T19:35:33.000Z,018032_2013-07-14T19:32:14.667Z,018032_2013-08-12T19:45:34.000Z,018032_2015-03-17T20:37:03.667Z,018032_2015-04-29T19:23:10.334Z,018066_2013-04-19T20:55:56.834Z,018066_2014-04-17T19:28:57.000Z,018066_2014-04-24T22:57:30.501Z,018066_2014-06-02T21:10:00.499Z,018066_2014-07-15T22:57:48.167Z,080217_2014-03-29T10:03:02.500Z D27 7 065679_2018-05-10T17:10:56.333Z,065679_2018-05-23T19:51:53.667Z,065679_2018-05-29T18:04:15.500Z,065679_2018-06-12T17:02:57.000Z,065679_2018-06-19T15:48:19.833Z,065679_2018-06-26T18:25:16.500Z,065679_2018-07-04T15:52:46.667Z D22 6 010856_2017-05-23T15:39:39.833Z,065679_2018-05-15T14:06:15.167Z,065679_2018-06-12T21:04:40.666Z,065679_2018-06-19T19:59:10.500Z,080217_2019-05-26T17:44:14.667Z,080217_2019-07-09T17:03:43.500Z QU4 4 018032_2014-08-28T17:35:08.334Z,018066_2014-08-14T18:13:11.501Z,080217_2014-07-24T19:26:58.167Z,080217_2014-07-30T18:43:19.500Z QU2 3 018032_2014-08-28T17:03:58.667Z,018066_2014-08-14T17:40:38.667Z,080217_2014-07-30T18:12:07.333Z BED10 2 065679_2019-08-28T15:13:00.500Z,080217_2019-07-03T15:02:35.166Z JS6 2 065679_2015-05-05T19:05:54.000Z,065679_2015-07-14T18:30:38.167Z D35 1 080217_2019-05-14T18:30:40.166Z DI18 1 01907674_2016-05-06T17:10:02Z FZH01 1 018032_2017-02-23T20:35:14.833Z HKP03 1 018066_2014-03-27T16:35:12.000Z JS2 1 065679_2015-05-05T19:30:48.667Z KC12 1 018066_2013-04-17T21:01:50.833Z KFPS08 1 018032_2013-07-20T16:34:30.333Z MG3 1 018066_2017-07-20T20:26:34.167Z NLU03 1 018066_2014-04-05T18:42:45.667Z QSD05 1 018032_2015-07-04T20:34:25.667Z QU17 1 018066_2014-10-09T20:00:39.500Z QU28 1 018066_2014-08-15T18:49:25.501Z QU32 1 080217_2015-03-16T17:39:06.500Z QU5 1 080217_2017-01-05T19:52:15.333Z

— Reply to this email directly, view it on GitHub https://github.com/HakaiInstitute/hakai-profile-qaqc/issues/22#issuecomment-1440569426, or unsubscribe https://github.com/notifications/unsubscribe-auth/A4LMM2PQNDZG4GRNEIXYZVDWYZK7DANCNFSM6AAAAAAU7XGYP4 . You are receiving this because you were mentioned.Message ID: @.***>

JessyBarrette commented 1 year ago

Yes that is correct, a lot of the single problematic casts are related to a bad annotation. As an example the QU5 profile listed there goes to 170m depth, which suggests it wasn't collected at QU5. Hard to say anything about all the others, but most of them come 2014-2017.

The FZH01 site goes to 294m while this specific profile is going down ~350m

ah-hakai commented 1 year ago

Ok, thanks for explaining Jessy. I believe you are allowing a buffer of +15% on the station depth before flagging, is that right?

If that is correct, then it sounds to me like the annotation is wrong, as you suggested, rather than the reference bathymetry is wrong. Flagging those entire casts as SVD is also the right decision. I think it's too late to recover those casts by the field crew.

Is there also a problem with the first list of stations - KWY02, NMU01, D27, D22 QU4, and QU2 - is the reference depth for these stations ok?

On Thu, Feb 23, 2023 at 2:01 PM Jessy Barrette @.***> wrote:

Yes that is correct, a lot of the single problematic casts are related to a bad annotation. As the an example the QU5 profile listed there goes to 170m depth, which suggests it wasn't collected at QU5. Hard to say anything about all the others, but most of them come 2014-2017.

The FZH01 site goes to 294m while this specific profile is going down ~350m

— Reply to this email directly, view it on GitHub https://github.com/HakaiInstitute/hakai-profile-qaqc/issues/22#issuecomment-1442487307, or unsubscribe https://github.com/notifications/unsubscribe-auth/A4LMM2KHGAWGK47MHWDUXEDWY7M43ANCNFSM6AAAAAAU7XGYP4 . You are receiving this because you were mentioned.Message ID: @.***>

JessyBarrette commented 1 year ago

I'm allowing 10% above to be flag as suspect and 30% for been bad . We can certainly increase those thresholds.

My first guest regarding those first stations with more flagged profiles is that we may not have the right depth.

fostermh commented 1 year ago

I'm not sure when the D27 and D22 stations were sampled but at one point I remember the JSP program going to these sites and collecting samples. They tended to collect in a general vicinity of a station rather than at a specific point, I believe. might be worth checking the lat/longs of what the tablet had to say if available.

ah-hakai commented 1 year ago

Jessy: I think your 10 % and 30 % thresholds for SVC and SVD are fine and don't need to be changed.

Matt: That's a good point. I believe Will McInnis computed a polygon average depth within a given radius (100 m?) but I don't see it anywhere in our online oceanography map metadata. If those stations aren't far off (D27, D22), are they caught within the 10% buffer and flagged SVD? I think that's ok. Otherwise, perhaps we should just manually change the station reference to the max cast depth for those sites. See below -

Ok, I see the n_profiles column now, and understand its value. Yes, I agree that if there are many casts at a station caught by this test, it's more likely an incorrect reference depth than incorrect annotation. I was thinking of your comment on QU5, which is clearly not an incorrect reference depth, since that site is about 70 m deep (it sounds like it may be QU17, which is 170 m deep and nearby, but I think it's too late to ask the field crew to remember it and we shouldn't make assumptions if possible). And it's only one cast.

What do you both think about this - for the list of stations with multiple casts caught by this test, I suggest we look at their maximum depths, and see if they are similar. If they are, we should just change the reference depth to the max depth in that grouping, and notify Will McInnis who could record it in the oceanography master map metadata, along with a new depth_src value as something like 'field_obs'. I think it's fine to manually correct the reference bathymetry when we have good evidence to do so - such as a batch of casts located close together, and all ending at a similar depth that differs from the reference data.

Alex

On Thu, Feb 23, 2023 at 7:25 PM Matthew Foster @.***> wrote:

I'm not sure when the D27 and D22 stations were sampled but at one point I remember the JSP program going to these sites and collecting samples. They tended to collect in a general vicinity of a station rather than at a specific point, I believe. might be worth checking the lat/longs of what the tablet had to say if available.

— Reply to this email directly, view it on GitHub https://github.com/HakaiInstitute/hakai-profile-qaqc/issues/22#issuecomment-1442753568, or unsubscribe https://github.com/notifications/unsubscribe-auth/A4LMM2M6VU7LM5MCHN5IM53WZAS3XANCNFSM6AAAAAAU7XGYP4 . You are receiving this because you were mentioned.Message ID: @.***>

fostermh commented 1 year ago

adjusting depth based on repeated sample results, where appropriate, seems reasonable to me.

JessyBarrette commented 1 year ago

agree I can certainly add an input in the qc to overwrite any specific stations maximum to something else than what listed in the arcgis map. I just need to those values and that we're sure the present stations above are good.

Once we have all this I can roll it all out on the whole production dataset

ah-hakai commented 1 year ago

I think the better approach is to correct the reference station bathymetry list, rather than overwrite the expected depth for these stations in the automated QC process. Otherwise, we'll end up discussing these discrepancies again when someone else uses these stations or their bottom depths from the reference list.

I would suggest that for the first four stations where there are a reasonable number of flagged casts (KWY02, NMU01, D27, D22) we first confirm that their maximum cast depths are roughly equal, and if so, use the maximum cast depth as the 'official' depth in the reference list (and update Will when we do so). Can you easily pull out the maximum depth for all of those flagged casts? A simple mean + std deviation will show us the spread.

We could do the same for the two Quadra stations (QU2, QU4), though they are also located close to home and have fewer instances. So we could also just ask the field crew to go and measure their depth on the next calm survey - they should both be within the depth range of their sounder. That would settle it. I can ask them to put that on their to-do list.

For the remainder, where there's only 1 or 2 flagged casts, I suggest we just leave them as SVC or SVD, depending on how different their maximum depth is from the reference depth. We shouldn't change official depths based on 1 or 2 discrepancies, I think that is insufficient evidence that the depth is incorrect. For BED10 and JS6, I know the bathymetry is surprisingly bumpy and steep sided, respectively, and moreover were collected in a scattering of actual positions, so we're trying to hit an inconsistent target anyways.

On Fri, Feb 24, 2023 at 10:39 AM Jessy Barrette @.***> wrote:

agree I can certainly add an input in the qc to overwrite any specific stations maximum to something else than what listed in the arcgis map. I just need to those values and that we're sure the present stations above are good.

Once we have all this I can roll it all out on the whole production dataset

— Reply to this email directly, view it on GitHub https://github.com/HakaiInstitute/hakai-profile-qaqc/issues/22#issuecomment-1444233002, or unsubscribe https://github.com/notifications/unsubscribe-auth/A4LMM2NING5VQTUVY73BVALWZD56FANCNFSM6AAAAAAU7XGYP4 . You are receiving this because you were mentioned.Message ID: @.***>

JessyBarrette commented 1 year ago

Here's a table that captures most of the stats you asked @ah-hakai 

select
    bad_casts.station as station,
    bad_casts.n_profiles as flagged_profiles_count,
    total_drops_count,
    bad_casts.avg_max_depth as flagged_avg_max_depth,
    total_avg_max_depth,
    total_median_max_depth,
    std_max_depth as flagged_std_max_depth,
    total_std_max_depth
from
    (
    select
        station,
        count(x.hakai_id) as n_profiles,
        string_agg(x.hakai_id, ',') as hakai_id_list,
        avg(max_depth) as avg_max_depth,
        stddev(max_depth) as std_max_depth
    from
        (
        select
            distinct station,
            hakai_id,
            max(depth) as max_depth
        from
            ctd.ctd_file_cast_data
        where
            temperature_flag like '%depth_in_station_range_test%'
        group by
            station,
            hakai_id ) as x
    group by
        station) as bad_casts
left join (
    select
        station,
        count(hakai_id) as total_drops_count,
        avg(max_depth) as total_avg_max_depth,
        stddev(max_depth) as total_std_max_depth,
        PERCENTILE_DISC(0.5) within group(
        order by max_depth) as total_median_max_depth
    from
        (
        select
            distinct station,
            hakai_id,
            max(depth) as max_depth
        from
            ctd.ctd_file_cast_data
        group by
            station,
            hakai_id ) as y
    group by
        station) as total_casts
    on
    bad_casts.station = total_casts.station
order by
        bad_casts.n_profiles desc,
        bad_casts.station asc;
station flagged_profiles_count total_drops_count flagged_avg_max_depth total_avg_max_depth total_median_max_depth flagged_std_max_depth total_std_max_depth
KWY02 13 76 261.4896923076923077 215.8901435342538388 214.156 7.8824098618867334 24.9043180607112190195806036656
NMU01 11 15 82.0389090909090909 78.9751333333333333 80.821 2.3657670407098605 6.4116024948823015
D27 7 28 58.1561126471573563 35.4515336405560141 31.887 11.187503761341730420836479305418 16.165524695702734495771028904877
D22 6 21 85.8326666666666667 64.6508303640987562 66.518 8.0587891563600711 18.4442851994025370072468974417
QU4 4 4 126.1815000000000000 126.1815000000000000 125.216 1.9790459822854041 1.9790459822854041
QU2 3 12 216.5920000000000000 125.4513333333333333 102.134 1.5707727397685510 57.1591015886730188
BED10 2 2 62.5360000000000000 62.5360000000000000 62.506 0.04242640687119285146 0.04242640687119285146
JS6 2 4 387.2895000000000000 310.8972500000000000 344.936 1.8024151852445096 126.143256739775
D35 1 6 75.5690000000000000 28.8585000000000000 15.436 23.8959803042269015
DI18 1 13 109.0280000000000000 66.4173076923076923 63.441 13.9422015620717483
FZH01 1 170 366.9950000000000000 227.2381626263339626 265.3 89.1950756437168430750986230004
HKP03 1 77 196.0100000000000000 112.6359087581701073 114.923 14.3327235542491194237278492735
JS2 1 84 392.2050000000000000 307.3042380952380952 312.167 43.1859003869525022
KC12 1 117 92.0490000000000000 16.7736820527998430 17.64080017758163 8.7361714325734660188338137059
KFPS08 1 49 95.7790000000000000 13.1996530612244898 11.523 12.4143975530950561
MG3 1 3 35.5520000000000000 17.5103333333333333 10.99 15.8233625482491342
NLU03 1 18 83.0550000000000000 52.4707777777777778 51.154 9.7374243457337777
QSD05 1 10 180.0960000000000000 68.8395000000000000 57.116 39.8621448891886231
QU17 1 248 211.0350000000000000 155.5346774193548387 157.576 26.8529083920177134
QU28 1 72 138.8740000000000000 81.4717083333333333 81.281 9.0511402494459694
QU32 1 68 138.8270000000000000 94.7555882352941176 97.136 16.3689904932533251
QU5 1 306 140.4190000000000000 62.3566928104575163 62.45 6.8061009374603725

I think the better approach is to correct the reference station bathymetry list Sure sounds good to me! How about adding an new column on the reference map that overwrites the Bot_depth_GIS column? Bot_depth_max_limit?

I think regarding all the upper stations ( up to D35) are within a sloppy region where depth range varies quickly over the covered area,hence why we exceeded the maximum depth expected. Adding a new tolerated depth should resolve the issue.

JessyBarrette commented 1 year ago

I have upgraded the StationLocation reference file to reflect the latest changes added to the Arcgis map.

ah-hakai commented 1 year ago

Awesome, thanks Jessy.

On Thu, Mar 2, 2023 at 7:54 AM Jessy Barrette @.***> wrote:

I have upgraded the StationLocation reference file to reflect the latest changes added to the Arcgis map.

— Reply to this email directly, view it on GitHub https://github.com/HakaiInstitute/hakai-profile-qaqc/issues/22#issuecomment-1452097781, or unsubscribe https://github.com/notifications/unsubscribe-auth/A4LMM2NNJQZEMIZMWJCNLJLW2C7DNANCNFSM6AAAAAAU7XGYP4 . You are receiving this because you were mentioned.Message ID: @.***>

JessyBarrette commented 1 year ago

Sorry @ah-hakai for the late reply on this issue. I upgraded the qc tool to integrate the latest GIS depths for each stations.

Looks like the stations BUR3 and BUR4 are still problematic most of the other stations listed below have a low count (likely bad value) or stations with high depth variability. We can certainly update the depth associated wtih those specific depths or include a manual depth value for some specific stations.

Here's the results of the same query above: station flagged_profiles_count total_drops_count flagged_avg_max_depth total_avg_max_depth total_median_max_depth flagged_std_max_depth total_std_max_depth
BUR3 17 17 317.2572033335180453 317.2572033335180453 316.062 17.3990484186216723815164218190 17.3990484186216723815164218190
KWY02 13 102 261.4913076923076923 211.0399421733118499 206.752 7.8823698465268615 23.2962705915940961150313047133
NMU01 11 15 82.0032727272727273 78.9493333333333333 80.811 2.3793074660879409 6.4054905763578736
BUR4 9 9 392.1257346978495244 392.1257346978495244 394.187293070688 10.1295706626281348996101715357 10.1295706626281348996101715357
D22 7 21 84.2044285714285714 64.8871160783844705 66.518 8.5131259800604943 18.5390871027815011934177896925
D27 7 42 58.1482555043002134 31.8615398634898295 28.646 11.190143964731808887789925270763 15.669919215857191550153775287232
QU2 4 15 215.7072500000000000 122.0578600223595293 104.433 2.1854095535314809 65.0146091010773260033820323334
QU4 4 4 126.1815000000000000 126.1815000000000000 125.215 1.9795074303135785 1.9795074303135785
BED10 2 2 62.5474130183839450 62.5474130183839450 62.53082603676789 0.0234575343602581786534247550 0.0234575343602581786534247550
D35 2 10 74.4210000000000000 35.7475000000000000 19.484 1.6122034611053284 24.3361079388449276
JS6 2 4 387.2930000000000000 310.8992500000000000 344.937 1.7932227970890845 126.144716085336
QU17 2 249 233.3780968381516650 156.4123024687841036 157.576 31.5993247874918755561405841723 26.6206436234444527457562593306
DI18 1 13 109.0280000000000000 66.4160000000000000 63.441 13.9422113860511143
FZH01 1 196 367.0010000000000000 232.9130508913471094 265.296 84.4329108115411905605520911809
HKP03 1 91 196.0020000000000000 111.4011748505268908 112.352 13.8282218889263757610905002537
JS2 1 83 392.2050000000000000 307.4243373493975904 312.178 43.4367160940044935
KC1 1 1365 67.4460000000000000 5.6538125551027804 5.843 2.47535367366406975967138494990472
KC12 1 119 92.0340000000000000 17.1530305625860564 17.54427979240153 10.276689186784880269954986224957
KC14 1 110 75.9270000000000000 32.8070260085338211 34.581 10.852433805444301911690064624067
KC15 1 132 192.1660000000000000 26.3925632128974614 26.795 16.377686791694436190476500686613
KC16 1 112 192.1660000000000000 36.0105998978928644 35.013 18.8130036268696330910971815218
KC4 1 137 422.9890400176499000 25.3053682529434141 23.374 34.493838422638377123862662259305
KFPS08 1 50 95.7640000000000000 13.1791444192573181 11.077 12.288109738272489221166540955635
MEA02 1 112 1283.0827147708965000 58.3026679723131109 49.304 116.876231780066629266660514981890
MEA04 1 122 145.4360000000000000 7.7452672890066409 6.091 12.81925533058700513981803913887433
MG3 1 3 35.5270000000000000 17.4993333333333333 10.985 15.8112325368180369
NLU03 1 18 83.0750000000000000 52.4739444444444444 51.15 9.7382998182266499
QSD05 1 10 180.0840000000000000 68.7838000000000000 56.55 39.8769164493834843
QU28 1 72 138.8730000000000000 81.4713472222222222 81.281 9.0512180756175874
QU32 1 75 138.8200000000000000 95.2236910312199129 97.136 15.9703131597208512227002690013
QU5 1 418 140.4040000000000000 62.1204904741081657 62.45 6.858353308086790619702118496313
RVRS02 1 56 269.3560000000000000 127.8969774815097805 129.835 29.2395020456261501640739594133
JessyBarrette commented 1 year ago

from @ah-hakai

Thanks Jessy. Can you please add one more column to that output table? It would be helpful to also see the nominal depth from our updated station depth list, included as a column. Then we could see how far off the average flagged depth is from the actual depth. The total mean depth doesn't provide this information, because it doesn't account for the varying buffer distance between the max depth of the CTD cast and the actual seabed.

Yes here it is

station Bot_depth Bot_depth_GIS flagged_profiles_count total_drops_count flagged_avg_max_depth total_avg_max_depth total_median_max_depth flagged_std_max_depth total_std_max_depth
0 BUR3 nan 200 17 17 317.257 317.257 316.062 17.399 17.399
1 KWY02 218 218 13 102 261.491 211.04 206.752 7.88237 23.2963
2 NMU01 80 49 11 15 82.0033 78.9493 80.811 2.37931 6.40549
3 BUR4 nan 199 9 9 392.126 392.126 394.187 10.1296 10.1296
4 D22 43 7 21 84.2044 64.8871 66.518 8.51313 18.5391
5 D27 15 7 42 58.1483 31.8615 28.646 11.1901 15.6699
6 QU2 120 100 4 15 215.707 122.058 104.433 2.18541 65.0146
7 QU4 50 4 4 126.181 126.181 125.215 1.97951 1.97951
8 BED10 0 1 2 2 62.5474 62.5474 62.5308 0.0234575 0.0234575
9 D35 nan 27 2 10 74.421 35.7475 19.484 1.6122 24.3361
10 JS6 390 339 2 4 387.293 310.899 344.937 1.79322 126.145
11 QU17 170 180 2 249 233.378 156.412 157.576 31.5993 26.6206
12 DI18 70 69 1 13 109.028 66.416 63.441 nan 13.9422
13 FZH01 290 294 1 196 367.001 232.913 265.296 nan 84.4329
14 HKP03 120 120 1 91 196.002 111.401 112.352 nan 13.8282
15 JS2 345 355 1 83 392.205 307.424 312.178 nan 43.4367
16 KC1 8 4 1 1365 67.446 5.65381 5.843 nan 2.47535
17 KC12 22 51 1 119 92.034 17.153 17.5443 nan 10.2767
18 KC14 44 33 1 110 75.927 32.807 34.581 nan 10.8524
19 KC15 28 20 1 132 192.166 26.3926 26.795 nan 16.3777
20 KC16 35 40 1 112 192.166 36.0106 35.013 nan 18.813
21 KC4 25 22 1 137 422.989 25.3054 23.374 nan 34.4938
22 KFPS08 14 13 1 50 95.764 13.1791 11.077 nan 12.2881
23 MEA02 54 49 1 112 1283.08 58.3027 49.304 nan 116.876
24 MEA04 8 52 1 122 145.436 7.74527 6.091 nan 12.8193
25 MG3 nan 0 1 3 35.527 17.4993 10.985 nan 15.8112
26 NLU03 55 53 1 18 83.075 52.4739 51.15 nan 9.7383
27 QSD05 60 60 1 10 180.084 68.7838 56.55 nan 39.8769
28 QU28 90 87 1 72 138.873 81.4713 81.281 nan 9.05122
29 QU32 108 106 1 75 138.82 95.2237 97.136 nan 15.9703
30 QU5 70 67 1 418 140.404 62.1205 62.45 nan 6.85835
31 RVRS02 nan 152 1 56 269.356 127.897 129.835 nan 29.2395
ah-hakai commented 1 year ago

Jessy, it doesn't look like the depth-in-station QC test is using the updated station bathymetry that we prepared with Will McInnes, for the threshold values. BUR3 and BUR4 are 346 m and 428 m respectively, and with those depth values, it looks like most of the flagged casts at these stations wouldn't have been flagged - since the mean depth of flagged casts is 317 m and 392 m, respectively. Can you confirm which depth list is being used for this QC test?

I've attached a copy of the updated depth file from our effort earlier this year.

Alex

On Tue, Apr 4, 2023 at 10:23 AM Jessy Barrette @.***> wrote:

from @ah-hakai https://github.com/ah-hakai

Thanks Jessy. Can you please add one more column to that output table? It would be helpful to also see the nominal depth from our updated station depth list, included as a column. Then we could see how far off the average flagged depth is from the actual depth. The total mean depth doesn't provide this information, because it doesn't account for the varying buffer distance between the max depth of the CTD cast and the actual seabed.

Yes here it is station Bot_depth Bot_depth_GIS flagged_profiles_count total_drops_count flagged_avg_max_depth total_avg_max_depth total_median_max_depth flagged_std_max_depth total_std_max_depth 0 BUR3 nan 200 17 17 317.257 317.257 316.062 17.399 17.399 1 KWY02 218 218 13 102 261.491 211.04 206.752 7.88237 23.2963 2 NMU01 80 49 11 15 82.0033 78.9493 80.811 2.37931 6.40549 3 BUR4 nan 199 9 9 392.126 392.126 394.187 10.1296 10.1296 4 D22 43 7 21 84.2044 64.8871 66.518 8.51313 18.5391 5 D27 15 7 42 58.1483 31.8615 28.646 11.1901 15.6699 6 QU2 120 100 4 15 215.707 122.058 104.433 2.18541 65.0146 7 QU4 50 4 4 126.181 126.181 125.215 1.97951 1.97951 8 BED10 0 1 2 2 62.5474 62.5474 62.5308 0.0234575 0.0234575 9 D35 nan 27 2 10 74.421 35.7475 19.484 1.6122 24.3361 10 JS6 390 339 2 4 387.293 310.899 344.937 1.79322 126.145 11 QU17 170 180 2 249 233.378 156.412 157.576 31.5993 26.6206 12 DI18 70 69 1 13 109.028 66.416 63.441 nan 13.9422 13 FZH01 290 294 1 196 367.001 232.913 265.296 nan 84.4329 14 HKP03 120 120 1 91 196.002 111.401 112.352 nan 13.8282 15 JS2 345 355 1 83 392.205 307.424 312.178 nan 43.4367 16 KC1 8 4 1 1365 67.446 5.65381 5.843 nan 2.47535 17 KC12 22 51 1 119 92.034 17.153 17.5443 nan 10.2767 18 KC14 44 33 1 110 75.927 32.807 34.581 nan 10.8524 19 KC15 28 20 1 132 192.166 26.3926 26.795 nan 16.3777 20 KC16 35 40 1 112 192.166 36.0106 35.013 nan 18.813 21 KC4 25 22 1 137 422.989 25.3054 23.374 nan 34.4938 22 KFPS08 14 13 1 50 95.764 13.1791 11.077 nan 12.2881 23 MEA02 54 49 1 112 1283.08 58.3027 49.304 nan 116.876 24 MEA04 8 52 1 122 145.436 7.74527 6.091 nan 12.8193 25 MG3 nan 0 1 3 35.527 17.4993 10.985 nan 15.8112 26 NLU03 55 53 1 18 83.075 52.4739 51.15 nan 9.7383 27 QSD05 60 60 1 10 180.084 68.7838 56.55 nan 39.8769 28 QU28 90 87 1 72 138.873 81.4713 81.281 nan 9.05122 29 QU32 108 106 1 75 138.82 95.2237 97.136 nan 15.9703 30 QU5 70 67 1 418 140.404 62.1205 62.45 nan 6.85835 31 RVRS02 nan 152 1 56 269.356 127.897 129.835 nan 29.2395

— Reply to this email directly, view it on GitHub https://github.com/HakaiInstitute/hakai-profile-qaqc/issues/22#issuecomment-1496337197, or unsubscribe https://github.com/notifications/unsubscribe-auth/A4LMM2LDJOL6PIPTUYS5URDW7RKJJANCNFSM6AAAAAAU7XGYP4 . You are receiving this because you were mentioned.Message ID: @.***>

JessyBarrette commented 1 year ago

@ah-hakai sorry but direct emails to github doesn't conserve attached files. A link to it would work.

I'm using the data available within the arcgis map https://hakai.maps.arcgis.com/apps/webappviewer/index.html?id=38e1b1da8d16466bbe5d7c7a713d2678

Looks like the BUR3 and BUR4 values in the GIS map are then still referencing the old values:

image
JessyBarrette commented 1 year ago

@ah-hakai after rerunning with the whole QC on the hakaidev database. Here's the results of the stations where depths exceeded the threshold. Looks like for most of the higher count ones the max depth reached by those stations is right the limit. We could potentially finetune the treshold a little to drop most the first listed drops which are actually good:

station Depth flagged_profiles_count total_drops_count flagged_avg_max_depth total_avg_max_depth total_median_max_depth flagged_std_max_depth total_std_max_depth
JS2 300 24 83 341.621 307.424 312.178 12.3673 43.4367
D22 20 16 21 72.648 64.8871 66.518 12.7738 18.5391
KWY02 218 13 102 261.491 211.04 206.752 7.88237 23.2963
NMU01 49 11 15 82.0033 78.9493 80.811 2.37931 6.40549
D27 15 7 42 58.1483 31.8615 28.646 11.1901 15.6699
DAWSONS 15 4 108 48.0192 21.17 18.753 2.37756 9.75339
QU2 101 4 15 215.707 122.058 104.433 2.18541 65.0146
QU4 43 4 4 126.181 126.181 125.215 1.97951 1.97951
QU45 0 4 157 35.419 17.1721 15.862 6.08867 5.17375
BED10 1 2 2 62.5474 62.5474 62.5308 0.0234575 0.0234575
D35 30 2 10 74.421 35.7475 19.484 1.6122 24.3361
KC12 10 2 119 83.9805 17.153 17.5443 11.3894 10.2767
KC16 35 2 112 130.17 36.0106 35.013 87.6759 18.813
QU17 178 2 249 233.378 156.412 157.576 31.5993 26.6206
D25 14 1 1 47.609 47.609 47.609 nan nan
DI18 69 1 13 109.028 66.416 63.441 nan 13.9422
FZH01 294 1 196 367.001 232.913 265.296 nan 84.4329
HKP03 120 1 91 196.002 111.401 112.352 nan 13.8282
KC1 3 1 1365 67.446 5.65381 5.843 nan 2.47535
KC14 32 1 110 75.927 32.807 34.581 nan 10.8524
KC15 20 1 132 192.166 26.3926 26.795 nan 16.3777
KC4 22 1 137 422.989 25.3054 23.374 nan 34.4938
KFPS08 13 1 50 95.764 13.1791 11.077 nan 12.2881
MEA02 50 1 112 1283.08 58.3027 49.304 nan 116.876
MEA04 2 1 122 145.436 7.74527 6.091 nan 12.8193
MG3 0 1 3 35.527 17.4993 10.985 nan 15.8112
NLU03 53 1 18 83.075 52.4739 51.15 nan 9.7383
QSD05 60 1 10 180.084 68.7838 56.55 nan 39.8769
QU28 87 1 72 138.873 81.4713 81.281 nan 9.05122
QU32 106 1 75 138.82 95.2237 97.136 nan 15.9703
QU5 59 1 418 140.404 62.1205 62.45 nan 6.85835
RVRS02 153 1 56 269.356 127.897 129.835 nan 29.2395
ah-hakai commented 1 year ago

Hmmm. Thanks Jessy, there's a number of things to think about here. What are your thresholds - for exceeding station depth, and also for station position? Are CTD casts also screened at the automated QC level for how close they are to the station coordinates that Will used to extract those depths? I wonder how this list would look if there was a tight circle around the coordinates that the CTD casts were filtered by.

On Wed, Apr 12, 2023 at 2:31 PM Jessy Barrette @.***> wrote:

@ah-hakai https://github.com/ah-hakai after rerunning with the whole QC on the hakaidev database. Here's the results of the stations where depths exceeded the threshold. Looks like for most of the higher count ones the max depth reached by those stations is right the limit. We could potentially finetune the treshold a little to drop most the first listed drops which are actually good: station Depth flagged_profiles_count total_drops_count flagged_avg_max_depth total_avg_max_depth total_median_max_depth flagged_std_max_depth total_std_max_depth JS2 300 24 83 341.621 307.424 312.178 12.3673 43.4367 D22 20 16 21 72.648 64.8871 66.518 12.7738 18.5391 KWY02 218 13 102 261.491 211.04 206.752 7.88237 23.2963 NMU01 49 11 15 82.0033 78.9493 80.811 2.37931 6.40549 D27 15 7 42 58.1483 31.8615 28.646 11.1901 15.6699 DAWSONS 15 4 108 48.0192 21.17 18.753 2.37756 9.75339 QU2 101 4 15 215.707 122.058 104.433 2.18541 65.0146 QU4 43 4 4 126.181 126.181 125.215 1.97951 1.97951 QU45 0 4 157 35.419 17.1721 15.862 6.08867 5.17375 BED10 1 2 2 62.5474 62.5474 62.5308 0.0234575 0.0234575 D35 30 2 10 74.421 35.7475 19.484 1.6122 24.3361 KC12 10 2 119 83.9805 17.153 17.5443 11.3894 10.2767 KC16 35 2 112 130.17 36.0106 35.013 87.6759 18.813 QU17 178 2 249 233.378 156.412 157.576 31.5993 26.6206 D25 14 1 1 47.609 47.609 47.609 nan nan DI18 69 1 13 109.028 66.416 63.441 nan 13.9422 FZH01 294 1 196 367.001 232.913 265.296 nan 84.4329 HKP03 120 1 91 196.002 111.401 112.352 nan 13.8282 KC1 3 1 1365 67.446 5.65381 5.843 nan 2.47535 KC14 32 1 110 75.927 32.807 34.581 nan 10.8524 KC15 20 1 132 192.166 26.3926 26.795 nan 16.3777 KC4 22 1 137 422.989 25.3054 23.374 nan 34.4938 KFPS08 13 1 50 95.764 13.1791 11.077 nan 12.2881 MEA02 50 1 112 1283.08 58.3027 49.304 nan 116.876 MEA04 2 1 122 145.436 7.74527 6.091 nan 12.8193 MG3 0 1 3 35.527 17.4993 10.985 nan 15.8112 NLU03 53 1 18 83.075 52.4739 51.15 nan 9.7383 QSD05 60 1 10 180.084 68.7838 56.55 nan 39.8769 QU28 87 1 72 138.873 81.4713 81.281 nan 9.05122 QU32 106 1 75 138.82 95.2237 97.136 nan 15.9703 QU5 59 1 418 140.404 62.1205 62.45 nan 6.85835 RVRS02 153 1 56 269.356 127.897 129.835 nan 29.2395

— Reply to this email directly, view it on GitHub https://github.com/HakaiInstitute/hakai-profile-qaqc/issues/22#issuecomment-1505976682, or unsubscribe https://github.com/notifications/unsubscribe-auth/A4LMM2PJCGPQRWTV5N7MORTXA4NMDANCNFSM6AAAAAAU7XGYP4 . You are receiving this because you were mentioned.Message ID: @.***>

JessyBarrette commented 1 year ago

Sorry for the late reply. We use a radius of 3km i believe from the target station which is based on the map.

The threshold are 10 suspect and 25 % fail for the reference depth vs maximum reached.

Let's have a look next week, Jessy

Le mer. 12 avr. 2023 5:48 p.m., ah-hakai @.***> a écrit :

Hmmm. Thanks Jessy, there's a number of things to think about here. What are your thresholds - for exceeding station depth, and also for station position? Are CTD casts also screened at the automated QC level for how close they are to the station coordinates that Will used to extract those depths? I wonder how this list would look if there was a tight circle around the coordinates that the CTD casts were filtered by.

On Wed, Apr 12, 2023 at 2:31 PM Jessy Barrette @.***> wrote:

@ah-hakai https://github.com/ah-hakai after rerunning with the whole QC on the hakaidev database. Here's the results of the stations where depths exceeded the threshold. Looks like for most of the higher count ones the max depth reached by those stations is right the limit. We could potentially finetune the treshold a little to drop most the first listed drops which are actually good: station Depth flagged_profiles_count total_drops_count flagged_avg_max_depth total_avg_max_depth total_median_max_depth flagged_std_max_depth total_std_max_depth JS2 300 24 83 341.621 307.424 312.178 12.3673 43.4367 D22 20 16 21 72.648 64.8871 66.518 12.7738 18.5391 KWY02 218 13 102 261.491 211.04 206.752 7.88237 23.2963 NMU01 49 11 15 82.0033 78.9493 80.811 2.37931 6.40549 D27 15 7 42 58.1483 31.8615 28.646 11.1901 15.6699 DAWSONS 15 4 108 48.0192 21.17 18.753 2.37756 9.75339 QU2 101 4 15 215.707 122.058 104.433 2.18541 65.0146 QU4 43 4 4 126.181 126.181 125.215 1.97951 1.97951 QU45 0 4 157 35.419 17.1721 15.862 6.08867 5.17375 BED10 1 2 2 62.5474 62.5474 62.5308 0.0234575 0.0234575 D35 30 2 10 74.421 35.7475 19.484 1.6122 24.3361 KC12 10 2 119 83.9805 17.153 17.5443 11.3894 10.2767 KC16 35 2 112 130.17 36.0106 35.013 87.6759 18.813 QU17 178 2 249 233.378 156.412 157.576 31.5993 26.6206 D25 14 1 1 47.609 47.609 47.609 nan nan DI18 69 1 13 109.028 66.416 63.441 nan 13.9422 FZH01 294 1 196 367.001 232.913 265.296 nan 84.4329 HKP03 120 1 91 196.002 111.401 112.352 nan 13.8282 KC1 3 1 1365 67.446 5.65381 5.843 nan 2.47535 KC14 32 1 110 75.927 32.807 34.581 nan 10.8524 KC15 20 1 132 192.166 26.3926 26.795 nan 16.3777 KC4 22 1 137 422.989 25.3054 23.374 nan 34.4938 KFPS08 13 1 50 95.764 13.1791 11.077 nan 12.2881 MEA02 50 1 112 1283.08 58.3027 49.304 nan 116.876 MEA04 2 1 122 145.436 7.74527 6.091 nan 12.8193 MG3 0 1 3 35.527 17.4993 10.985 nan 15.8112 NLU03 53 1 18 83.075 52.4739 51.15 nan 9.7383 QSD05 60 1 10 180.084 68.7838 56.55 nan 39.8769 QU28 87 1 72 138.873 81.4713 81.281 nan 9.05122 QU32 106 1 75 138.82 95.2237 97.136 nan 15.9703 QU5 59 1 418 140.404 62.1205 62.45 nan 6.85835 RVRS02 153 1 56 269.356 127.897 129.835 nan 29.2395

— Reply to this email directly, view it on GitHub < https://github.com/HakaiInstitute/hakai-profile-qaqc/issues/22#issuecomment-1505976682 , or unsubscribe < https://github.com/notifications/unsubscribe-auth/A4LMM2PJCGPQRWTV5N7MORTXA4NMDANCNFSM6AAAAAAU7XGYP4

. You are receiving this because you were mentioned.Message ID: @.***>

— Reply to this email directly, view it on GitHub https://github.com/HakaiInstitute/hakai-profile-qaqc/issues/22#issuecomment-1506000359, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHICYOOJWN2Z6BKFNGCNAZLXA4PLVANCNFSM6AAAAAAU7XGYP4 . You are receiving this because you authored the thread.Message ID: @.***>

ah-hakai commented 1 year ago

Ok thanks Jessy. 3 km seems really large...do you know the rationale behind such a large allowable distance from actual station coordinates?

Could we reduce that by quite a bit and then see what shows up? I'd bet quite a few of those flagged casts would disappear if we used a smaller allowable distance error, such as 250 m (i.e., a half km diameter circle). Perhaps the missing stations would then have a distance-flag error, but that would be ok because it would be correct - their depth error now would then be a validation of why they count as a different station.

On Fri, Apr 14, 2023 at 2:22 PM Jessy Barrette @.***> wrote:

Sorry for the late reply. We use a radius of 3km i believe from the target station which is based on the map.

The threshold are 10 suspect and 25 % fail for the reference depth vs maximum reached.

Let's have a look next week, Jessy

Le mer. 12 avr. 2023 5:48 p.m., ah-hakai @.***> a écrit :

Hmmm. Thanks Jessy, there's a number of things to think about here. What are your thresholds - for exceeding station depth, and also for station position? Are CTD casts also screened at the automated QC level for how close they are to the station coordinates that Will used to extract those depths? I wonder how this list would look if there was a tight circle around the coordinates that the CTD casts were filtered by.

On Wed, Apr 12, 2023 at 2:31 PM Jessy Barrette @.***> wrote:

@ah-hakai https://github.com/ah-hakai after rerunning with the whole QC on the hakaidev database. Here's the results of the stations where depths exceeded the threshold. Looks like for most of the higher count ones the max depth reached by those stations is right the limit. We could potentially finetune the treshold a little to drop most the first listed drops which are actually good: station Depth flagged_profiles_count total_drops_count flagged_avg_max_depth total_avg_max_depth total_median_max_depth flagged_std_max_depth total_std_max_depth JS2 300 24 83 341.621 307.424 312.178 12.3673 43.4367 D22 20 16 21 72.648 64.8871 66.518 12.7738 18.5391 KWY02 218 13 102 261.491 211.04 206.752 7.88237 23.2963 NMU01 49 11 15 82.0033 78.9493 80.811 2.37931 6.40549 D27 15 7 42 58.1483 31.8615 28.646 11.1901 15.6699 DAWSONS 15 4 108 48.0192 21.17 18.753 2.37756 9.75339 QU2 101 4 15 215.707 122.058 104.433 2.18541 65.0146 QU4 43 4 4 126.181 126.181 125.215 1.97951 1.97951 QU45 0 4 157 35.419 17.1721 15.862 6.08867 5.17375 BED10 1 2 2 62.5474 62.5474 62.5308 0.0234575 0.0234575 D35 30 2 10 74.421 35.7475 19.484 1.6122 24.3361 KC12 10 2 119 83.9805 17.153 17.5443 11.3894 10.2767 KC16 35 2 112 130.17 36.0106 35.013 87.6759 18.813 QU17 178 2 249 233.378 156.412 157.576 31.5993 26.6206 D25 14 1 1 47.609 47.609 47.609 nan nan DI18 69 1 13 109.028 66.416 63.441 nan 13.9422 FZH01 294 1 196 367.001 232.913 265.296 nan 84.4329 HKP03 120 1 91 196.002 111.401 112.352 nan 13.8282 KC1 3 1 1365 67.446 5.65381 5.843 nan 2.47535 KC14 32 1 110 75.927 32.807 34.581 nan 10.8524 KC15 20 1 132 192.166 26.3926 26.795 nan 16.3777 KC4 22 1 137 422.989 25.3054 23.374 nan 34.4938 KFPS08 13 1 50 95.764 13.1791 11.077 nan 12.2881 MEA02 50 1 112 1283.08 58.3027 49.304 nan 116.876 MEA04 2 1 122 145.436 7.74527 6.091 nan 12.8193 MG3 0 1 3 35.527 17.4993 10.985 nan 15.8112 NLU03 53 1 18 83.075 52.4739 51.15 nan 9.7383 QSD05 60 1 10 180.084 68.7838 56.55 nan 39.8769 QU28 87 1 72 138.873 81.4713 81.281 nan 9.05122 QU32 106 1 75 138.82 95.2237 97.136 nan 15.9703 QU5 59 1 418 140.404 62.1205 62.45 nan 6.85835 RVRS02 153 1 56 269.356 127.897 129.835 nan 29.2395

— Reply to this email directly, view it on GitHub <

https://github.com/HakaiInstitute/hakai-profile-qaqc/issues/22#issuecomment-1505976682

, or unsubscribe <

https://github.com/notifications/unsubscribe-auth/A4LMM2PJCGPQRWTV5N7MORTXA4NMDANCNFSM6AAAAAAU7XGYP4

. You are receiving this because you were mentioned.Message ID: @.***>

— Reply to this email directly, view it on GitHub < https://github.com/HakaiInstitute/hakai-profile-qaqc/issues/22#issuecomment-1506000359 , or unsubscribe < https://github.com/notifications/unsubscribe-auth/AHICYOOJWN2Z6BKFNGCNAZLXA4PLVANCNFSM6AAAAAAU7XGYP4

. You are receiving this because you authored the thread.Message ID: @.***>

— Reply to this email directly, view it on GitHub https://github.com/HakaiInstitute/hakai-profile-qaqc/issues/22#issuecomment-1509289323, or unsubscribe https://github.com/notifications/unsubscribe-auth/A4LMM2OMZ5YT5LN2I2TMYPLXBG5YBANCNFSM6AAAAAAU7XGYP4 . You are receiving this because you were mentioned.Message ID: @.***>

JessyBarrette commented 1 year ago

We certainly could reduce that threshold value. This is where it is done within the hakai-database https://github.com/HakaiInstitute/hakai-database/blob/main/SQL/ctd/ctd-views.sql

Limits are actually:

I would suggest creating a new issue to talk about this since it is a separate issue from the reference station depths