explore better threshold values

[rosepearson]

Implement the attached thresholds

Kelp_Algorithm_LT.txt

Note banding might be different depending on the STAC collection at play

[rosepearson]

info__ocean_less_than_10percent_defective_cirrus_high_probability_cloud.csv info_ocean_less_than_10percent_defective_high_probability_cloud.csv

Attached CSV from the last two runs. These can be used to target those particularly high area days and understand what is causing this unusual thresholding.

Screenshot using cloud percentage threshold in image selection and masking out pixels with 'high cloud probability', 'defective' classifications.

Screenshot using cloud percentage threshold in image selection, then removing selected images if more than that same percentage over the ocean is 'high cloud probability', 'high cirrus', and 'no data' over the ocean, and masking out pixels with 'high cloud probability', 'defective' classifications.

Screenshot using cloud percentage threshold in image selection, then removing selected images if more than 10% over the ocean is 'high cirrus', 'defective', and 'no data' over the ocean, and masking out pixels with 'high cloud probability', 'high cirrus', 'defective' classifications.

Note there are more in this last one as the low thresholds for some of the later years were increased.

[rosepearson]

Adding leighs thresholding changes in place, with the same filtering to remove passes with more than 10% high probability cloud, defective, no data and thin cirrus over the ocean. Attached is the csv with area by date to target problem dates.

info_ocean_less_than_10percent_filtered_out_high_prob_cloud_cirrus_defective_leigh_thresholds.csv

Adding leighs thresholding changes in place, with the extended filtering to remove passes with more than 10% high probability and medium probability cloud, defective, no data and thin cirrus over the ocean. Attached is the csv with area by date to target problem dates.

[rosepearson]

Some imagery illustrating the differences achieved through Leighs thresholds: Old thresholds -

New thresholds but accidentally still using max_ndvi and min_ndvri -

New thresholds -

[rosepearson]

Have applied the above to all available times through he end of 2023 for the 50km^2 tile 1607 including Waikouaiti. This includes corrections:

• To exactly implement Leigh's thresholds
• To correctly mask to remove all passes with more than 10% of the ocean covered in cloud
• To save out the kelp detection and rgb This results in much better results (main issue is detecting estuaries' in some images) and a total of 4.8G of memory. <100MB of this is from the kelp detection.

Dates working well but with false estuary detections are: 2016-04-09, 2017-02-26 Dates still with a little bit of glint/speckle false detection and estuaries: 2016-06-18, 2016-07-28 Dates with no kelp detected but still estuary and some glint: 2017-01-24

Example of working well but with Estuaries - looks like reintroducing a min NDVI could be all that is required.

If you want to run the larger area you just need to run the notebook: https://github.com/niwa/kelp-detection/blob/4-update-streamlit/notebooks/split_geometries.ipynb to create the tile file then update the line geometry_df = geopandas.read_file(data_path / "vectors" / f"{name}.gpkg") with the lines

geometry_df = geopandas.read_file(data_path / "vectors" / f"tiles.gpkg")
geometry_df = geometry_df[geometry_df["name"]=="1607"]]