Problem on SDS_shoreline and SDS_preprocess

[jimpaulobautista]

Hello! Excellent work in automating shoreline extraction.

I was able to successfully retrieve satellite images using the workflow you established. However, upon applying the preprocessing parameters and attempting to execute outputs = SDS_shoreline.extract_shorelines(metadata, settings), the screen keeps on showing: Can you please help me figure out what's wrong?

Here's the code I used:

%load_ext autoreload
%autoreload 2
import os
import numpy as np
import pickle
import warnings 
warnings.filterwarnings("ignore")
import matplotlib
matplotlib.use('Qt5Agg')
from matplotlib import gridspec
import pandas as pd
from datetime import datetime
from coastsat import SDS_download, SDS_preprocess, SDS_shoreline, SDS_tools, SDS_transects

polygon = [[[121.3530, 15.3094], [121.3668, 15.2077], [121.4533, 15.2209], [121.4293, 15.3138]]]
#polygon = SDS_tools.smallest_rectangle(polygon)
dates = ['2019-02-01', '2019-02-28']
sat_list = ['L8']
collection = 'C02'
sitename = 'AURORA_February'
filepath = os.path.join(os.getcwd(), 'data')
inputs = {'polygon' : polygon, 'dates' : dates, 'sat_list' : sat_list, 'sitename' : sitename, 'filepath' : filepath, 'landsat_collection' : collection}
SDS_download.check_images_available(inputs);
metadata = SDS_download.get_metadata(inputs);
SDS_download.retrieve_images(inputs);

metadata = SDS_download.get_metadata(inputs) 

settings = {
    # general parameters
    'cloud_tresh' : 0.8,
    'dist_clouds' : 50,
    'output_epsg' : 3857,
    # quality control
    'check_detection' : True,
    'adjust_detection' : False,
    'save_figure' : True
    # shoreline detection parameters for ADVANCED USERS
    #'min_beach_area' : 1000,
    #'min_length_sl' : 500,
    #'cloud_mask_issue' : False,
    #'sand_color' : 'default',
    #'pan_off' : False,

    #'inputs': inputs,

}

%matplotlib qt
outputs = SDS_shoreline.extract_shorelines(metadata, settings)

Also, it seems that the advanced shoreline detection parameters are not working and jupyter keeps on telling me "Syntax Error". Moreover, I tried executing the example on Narrabeen beach using solely the code you developed but I am stuck with this problem:

Thank you so much!

[kvos]

do not comment out the parameters in the dictionary, just leave them as by default. Then it will work.

[jimpaulobautista]

Hi Kilian @kvos ! Thank you for your response. I have tried your suggestion but I keep on getting this error:

Thanks.

[kvos]

it's missing a comma in the 'save_figure' line

[jimpaulobautista]

Thanks for the swift response Kilian @kvos . I think I am having the same issue with @lexterspiritus97 based on the recently opened thread. I followed your advice there and it worked! I have moved forward in reference shoreline digitization, but similar to his/her/their problem, I also got this error: AttributeError: 'GeoDataFrame' object has no attribute 'append'. Can't find any solution yet but I hope we can figure this out.

`--------------------------------------------------------------------------- AttributeError Traceback (most recent call last) Cell In[10], line 2 1 get_ipython().run_line_magic('matplotlib', 'qt') ----> 2 settings['reference_shoreline'] = SDS_preprocess.get_reference_sl(metadata, settings) 3 settings['max_dist_ref'] = 100 # max distance (in meters) allowed from the reference shoreline

File ~\Downloads\CoastSat-master\coastsat\SDS_preprocess.py:848, in get_reference_sl(metadata, settings) 846 gdf_all = gdf 847 else: --> 848 gdf_all = gdf_all.append(gdf) 849 gdf_all.crs = CRS(image_epsg) 850 # convert from image_epsg to user-defined coordinate system

File ~\anaconda3\envs\coastsat\lib\site-packages\pandas\core\generic.py:5989, in NDFrame.getattr(self, name) 5982 if ( 5983 name not in self._internal_names_set 5984 and name not in self._metadata 5985 and name not in self._accessors 5986 and self._info_axis._can_hold_identifiers_and_holds_name(name) 5987 ): 5988 return self[name] -> 5989 return object.getattribute(self, name)

AttributeError: 'GeoDataFrame' object has no attribute 'append'`

[lexterspiritus97]

Hi Jim Paulo Bautista, I see that you got the same AttributeError that I got. I'll try to find out what happens with this, maybe like Kilian says the .append is deprecated for geodataframe. Let's hope to find the answer soon on the web. I'm using the coastsat 2.2 version, what version are you using? Maybe an older version works without these errors. Well, thankyou Jim, by your comment. keep in touch, greetings, Lexter.

El vie, 21 abr 2023 a las 9:29, Jim Paulo Bautista (< @.***>) escribió:

Thanks for the swift response Kilian @kvos https://github.com/kvos . I think I am having the same issue with @lexterspiritus97 https://github.com/lexterspiritus97 based on the recently opened thread. I followed your advice there and it worked! I have moved forward in reference shoreline digitization, but similar to his/her/their problem, I also got this error: AttributeError: 'GeoDataFrame' object has no attribute 'append'. Can't find any solution yet but I hope we can figure this out.

`--------------------------------------------------------------------------- AttributeError Traceback (most recent call last) Cell In[10], line 2 1 get_ipython().run_line_magic('matplotlib', 'qt') ----> 2 settings['reference_shoreline'] = SDS_preprocess.get_reference_sl(metadata, settings) 3 settings['max_dist_ref'] = 100 # max distance (in meters) allowed from the reference shoreline

File ~\Downloads\CoastSat-master\coastsat\SDS_preprocess.py:848, in get_reference_sl(metadata, settings) 846 gdf_all = gdf 847 else: --> 848 gdf_all = gdf_all.append(gdf) 849 gdf_all.crs = CRS(image_epsg) 850 # convert from image_epsg to user-defined coordinate system

File ~\anaconda3\envs\coastsat\lib\site-packages\pandas\core\generic.py:5989, in NDFrame.getattr(self, name) 5982 if ( 5983 name not in self._internal_names_set 5984 and name not in self._metadata 5985 and name not in self._accessors 5986 and self._info_axis._can_hold_identifiers_and_holds_name(name) 5987 ): 5988 return self[name] -> 5989 return object.getattribute(self, name)

AttributeError: 'GeoDataFrame' object has no attribute 'append'`

— Reply to this email directly, view it on GitHub https://github.com/kvos/CoastSat/issues/396#issuecomment-1517838386, or unsubscribe https://github.com/notifications/unsubscribe-auth/A7JEGEJIIPMMDP5G7MOOLJ3XCKDVNANCNFSM6AAAAAAXDXQUSE . You are receiving this because you were mentioned.Message ID: @.***>

[kvos]

ok seems to be a recurring error, I'll troubleshoot next week. But you can still continue with the notebook, don't worry if you don't have then .geojson

[jimpaulobautista]

Hello! Based on my web search the frame.append (gdf.append) has already been deprecated since panda v1.4.0.

(The frame.append method is depreciated and will be removed from pandas in a future version. Use pandas.concat instead.)

Some users made mention about using concat but I haven't figured it out as of this moment.

[lexterspiritus97]

Thank You Jim!!! Good to know this. Now I'm aware of this. I searched in the web now, and is like you said, it is being deprecated. Someone posted a new issue (29) in the CoastSat GitHub, with the same situation: AttributeError: 'GeoDataFrame' object has no attribute 'append. So, maybe Kilian is now aware of this too, and can find a solution for this part of the code. Greetings, Lexter.

El lun, 24 abr 2023 a las 4:29, Jim Paulo Bautista (< @.***>) escribió:

Hello! Based on my web search https://github.com/geopandas/geopandas/issues/2606 the frame.append (gdf.append) has already been deprecated since panda v1.4.0.

(The frame.append method is depreciated and will be removed from pandas in a future version. Use pandas.concat instead.)

Some users made mention about using concat but I haven't figured it out as of this moment.

— Reply to this email directly, view it on GitHub https://github.com/kvos/CoastSat/issues/396#issuecomment-1519620055, or unsubscribe https://github.com/notifications/unsubscribe-auth/A7JEGEN6ZH2ENLEWQZCUMOTXCY2VHANCNFSM6AAAAAAXDXQUSE . You are receiving this because you were mentioned.Message ID: @.***>

[jimpaulobautista]

Update: @kvos @lexterspiritus97

I followed the changes made in the thread I mentioned last time. I traced the error to line 714 of SDS_tools.py gdf_all = gdf_all.append(gdf) and replaced it with the code

At the start: import pandas as pd

At line 714, replace the code with: res = pd.concat([gdf_all, gdf]) type(res)

I was able to extract the GEOJSON shoreline. However, upon loading in QGIS, it seems that there's a minor error with the extracted shoreline (see photo below). I don't know if this is caused by the changes I made on the code, or it is caused by the "broken shoreline" where the river drains towards the sea.

May I know this solution works on your end? Or do you know how to fix this using the adjustments on the code? Eitherway, I think the error on the GEOJSON file can be fixed manually using the tools in QGIS. Thanks.

[kvos]

hi @jimpaulobautista , thanks for the solution, this works! can you pls make a pull request with that change? directly to the master branch. The issue with the way the linestrings are connected is a separate one and existed before as well. To overcome it, you can choose points instead of lines. Thanks for the nice contribution! This solution also solves issues #398 #397

[kvos]

this has been fixed by @jimpaulobautista ! #400 thanks