Skipped L8 images?

[NO645]

I'm using CoastSat to collect ~1 year of images from L8 and S2. About 20 images are downloaded from L8 and about 40 from S2.

After mapping the reference shoreline the code outputs: Mapping shorelines: L8: 100%

and then moves onto the selection of the S2 images. I'm never given an option to select from the L8 imagery. Both intuition from observing the S2 images and manual inspection of the downloaded files suggests that the problem is not that they are ALL cloudy. I'm not sure what else would cause all of the the L8 imagery to be rejected wholesale. I'm interested in using CoastSat.slope so I'd like to utilize the L8 images.

Thank you for your help!

[kvos]

for every issue, you need to provide enough information so that someone can reproduce your problem (your script with inputs,...etc), otherwise there is no way I, or anyone else, can help.

[NO645]

Let me know if you require the imports as well. This is an attempt to distill only the coastsat part of the code, since I have other work mixed in as well.

Thank you.

sitename= 'DUCK2'
epsg=32618
polygon = [[[-75.750933,  36.191795],
            [-75.758622,  36.191097],
            [-75.747470,  36.164576],
            [-75.738439,  36.167756],
            [-75.750933, 36.191795]]]
long = -75.575538
lat = 36.196656
dates = ['2016-12-01', '2018-01-01']
sat_list = ['L5', 'L8', 'S2']

filepath_data = os.path.join(os.getcwd(), 'data')
inputs = {
    'polygon': polygon,
    'dates': dates,
    'sat_list': sat_list,
    'sitename': sitename,
    'filepath': filepath_data
        }
settings = { 
    # general parameters:
    'cloud_thresh': 0.5,        # threshold on maximum cloud cover
    'output_epsg': epsg,       # epsg code of spatial reference system desired for the output   
    # quality control:
    'check_detection': True,    # if True, shows each shoreline detection to the user for validation
    'save_figure': True,        # if True, saves a figure showing the mapped shoreline for each image
    # add the inputs defined previously
    'inputs': inputs,
    # [ONLY FOR ADVANCED USERS] shoreline detection parameters:
    'min_beach_area': 4500,     # minimum area (in metres^2) for an object to be labelled as a beach
    'buffer_size': 150,         # radius (in metres) of the buffer around sandy pixels considered in the shoreline detection
    'min_length_sl': 200,       # minimum length (in metres) of shoreline perimeter to be valid
    'cloud_mask_issue': False,  # switch this parameter to True if sand pixels are masked (in black) on many images  
    'sand_color': 'default',    # 'default', 'dark' (for grey/black sand beaches) or 'bright' (for white sand beaches)
}
settings['max_dist_ref'] = 100  
settings['along_dist'] = 25
#download
SDS_download.check_images_available(inputs);

#metadata
metadata = SDS_download.retrieve_images(inputs)

#preprocess
SDS_preprocess.save_jpg(metadata, settings)

# get reference shore from user
settings['reference_shoreline'] = SDS_preprocess.get_reference_sl(metadata, settings)

# get shoreline shapes
output = SDS_shoreline.extract_shorelines(metadata, settings)

[kvos]

it looks like it's an issue with the new version of the earthengine api and the way they crop the images. What is happening is that those L8 images are being cropped around your polygon and the rest is being considered as cloud (it should not consider it as cloud). I'll try to fix this.

In the meanwhile you can either set the cloud threshold to 0.99 and then you will get the images:

or you can downgrade to a previous version of the earthengine api (<= 0.1.201) so that the cropping is better: I did look at Duck sometime ago and with the old version of the api it was fine https://twitter.com/_kvos/status/1179265715649548288?s=20

[kvos]

I just pushed on the development branch a quick and dirty fix so it should work fine now, let me know https://github.com/kvos/CoastSat/tree/development

[NO645]

I seem to be getting this error consistently during the image selection process. I have not changed the code from the above.

Thank you.

>>>output = SDS_shoreline.extract_shorelines(metadata, settings)

Mapping shorelines:

L8:   90%Traceback (most recent call last):

  File "<ipython-input-63-858795e85b5a>", line 1, in <module>
    output = SDS_shoreline.extract_shorelines(metadata, settings)

  File "C:\Users\RDCHLNRO\AppData\Local\Continuum\anaconda3\envs\coastsat_08_28_2020\lib\site-packages\coastsat\SDS_shoreline.py", line 779, in extract_shorelines
    min_beach_area_pixels, clf)

  File "C:\Users\RDCHLNRO\AppData\Local\Continuum\anaconda3\envs\coastsat_08_28_2020\lib\site-packages\coastsat\SDS_shoreline.py", line 148, in classify_image_NN
    labels = clf.predict(vec_features)

  File "C:\Users\RDCHLNRO\AppData\Local\Continuum\anaconda3\envs\coastsat_08_28_2020\lib\site-packages\sklearn\neural_network\multilayer_perceptron.py", line 953, in predict
    y_pred = self._predict(X)

  File "C:\Users\RDCHLNRO\AppData\Local\Continuum\anaconda3\envs\coastsat_08_28_2020\lib\site-packages\sklearn\neural_network\multilayer_perceptron.py", line 658, in _predict
    X = check_array(X, accept_sparse=['csr', 'csc', 'coo'])

  File "C:\Users\RDCHLNRO\AppData\Local\Continuum\anaconda3\envs\coastsat_08_28_2020\lib\site-packages\sklearn\utils\validation.py", line 582, in check_array
    context))

ValueError: Found array with 0 sample(s) (shape=(0, 20)) while a minimum of 1 is required.

[kvos]

thanks for reporting that, I fixed it now. Download (or pull updates if you cloned the repo) the new version and it should be all good. What was happening is that something was funny with the cloud cover and so it was trying to classify an image that didn't have any valid pixel.

[kvos]

did the update work?

[NO645]

I haven't had a chance to check yet. I will probably get back to this in the next four days. I'll let you know what happens as soon as I do.

[NO645]

It seems to be working great. Thank you very much for all your work.