JPPereira93 commented 1 year ago

Hello!

I found this toolkit on linkedin and I find it super useful,

I'm trying to follow the instructions to do all the installation steps however I'm a windows user. As such, I'm using WSL to bypass that issue. However, when installing snap.sh, the installation never stops as shown in the following image, do you have any idea of how long it takes to install it? it is been like 2 hours cycling through processes

If i have snap on windows do I have to install it again through Linux?

Thank you!

EDIT1: I'm trying to do all the steps and when I reach here I can't download the precipitation data for my AOI because it says command is not found without the python3, and module not found with python3. Any help?

JPPereira93 commented 1 year ago

I've managed to install it through ubuntu and I have ran the whole notebook. However some things gave me errors like:

Plot a time figure that Sentinel-1 acquisitions (post-flood and pre-flood are visualized)

Any help? Thanks!

kleok commented 1 year ago

Hello @JPPereira93 Thanks a lot for using FloodPy :smile: Regarding the problem with installation on windows machine, I created a new issue (#38) in order to work separately on this. Regarding the jupyter notebook problems, I pushed some changes where the AttributeError: global thresh and NameError: test are resolved. However, the plotting error seems to be related to the downloading of ERA5 data. app.run_download_Precipitation_data('Download_Precipitation_data') Can you verify that the downloading of ERA5 data was completed successfully? Looking forward for your reply, Kleanthis

JPPereira93 commented 1 year ago

Thank you so much for the windows implementation! However I am now running it on linux because that machine also has more computing resources!

I think the era5 precipitation downloaded fine!

kleok commented 1 year ago

It seems that NameError: ticker is not defined is related to a missing dependency. I recommend adding import matplotlib.ticker as ticker in the current cell.

Regarding the issue with the ValueError: plotting, it is strange because it seems that the precipitation_df dataframe column Daily Total Precipitation (mm) contains both negative and positive precipitation values.

If it is only a visualization issue you can try to replace area with line in the following command?

ax = precipitation_df.plot(kind='area', title='Precipitation and S1 products for {} case study'.format(Projectname))

However, In order to reproduce your error and understand what exactly is happening, I suggest to share with me the following configuration parameters of your study:

# B1. The datetime of flood event (Format is YYYYMMDDTHHMMSS)
Flood_datetime = ?

# B2. Days before flood event for baseline (pre-flood) stack construction
before_flood_days = ?

# B3. Days after flood event
after_flood_days = ?

# C1. AOI VECTOR FILE (if given AOI BBOX parameters can be ommited)
AOI_File = ?

# C2. AOI BBOX (WGS84)
LONMIN=?
LATMIN=?
LONMAX=?
LATMAX=?

# D1. number of consequent days that precipitation will be accumulated.
#       before each Sentinel-1 acquisition datetime
days_back = ?

# D2. The threshold of acculated precipitation [mm]
accumulated_precipitation_threshold = ?

#E1. The number of Sentinel-1 relative orbit. The default 
#       value is Auto. Auto means that the relative orbit that has
#       the Sentinel-1 image closer to the Flood_datetime is selected. 
#       S1_type can be GRD or SLC.
S1_type = ?
relOrbit = ?

#E3. The minimum mapping unit area in square meters
minimum_mapping_unit_area_m2 = ?

JPPereira93 commented 1 year ago

For sure!

I mainly used your default parameters just to make a test:


#    CONFIGURATION FILE OF FLOODPY

#   A. Project Definition  

#A1. The name of your project withough special characters.
Projectname = a7

#A2. The location that everything is going to be saved. Make sure 
#    you have enough free space disk on the specific location.
projectfolder = /home/ana/FLOODPY/a7

#A3. The location of floodpy code 
src_dir = /home/ana/FLOODPY/floodpy

#A4. SNAP ORBIT DIRECTORY
snap_dir = /home/ana/.snap/auxdata/Orbits/Sentinel-1

#A5. SNAP GPT full path
GPTBIN_PATH = /home/ana/snap/bin/gpt

#   B. Flood event temporal information  

# Your have to provide the datetime of your flood event. Make sure that
# a flood event took place at your provided datetime. 
# Based on your knowledge you can change [before_flood_days] in order
# to create a biggest 
# Sentinel-1 image that is going to be used to extract flood information
# will be between Flood_datetime and Flood_datetime+after_flood_days
# the closest Sentinel-1 to the Flood_datetime is picked
#-------------------------------------------------------------
# B1. The datetime of flood event (Format is YYYYMMDDTHHMMSS)
Flood_datetime = 20200921T030000

# B2. Days before flood event for baseline stack construction
before_flood_days = 20

# B3. Days after flood event
after_flood_days = 3

#  C. Flood event spatial information 
#-------------------------------------------------------------
# You can provide AOI VECTOR FILE or AOI BBOX. 
# Please ensure that your AOI BBOX has dimensions smaller than 100km x 100km
# If you provide AOI VECTOR, AOI BBOX parameters will be ommited
#-In case you provide AOI BBOX coordinates, set  AOI_File = None
#--------------------------------------------------------

# C1. AOI VECTOR FILE (if given AOI BBOX parameters can be ommited)
AOI_File = None

# C2. AOI BBOX (WGS84)
LONMIN=-8.62332
LATMIN=41.383467
LONMAX=-8.578227
LATMAX=41.399007
#  D. Precipitation information   

#  Based on your knowledge, provide information related to the 
# accumulated precipitation that is required in order a flooding to occur. 
# These particular values will be used to classify Sentinel-1 images
#  which images correspond to flood and non-flood conditions.

# D1. number of consequent days that precipitation will be accumulated.
#       before each Sentinel-1 acquisition datetime
days_back = 12

# D2. The threshold of acculated precipitation [mm]
accumulated_precipitation_threshold = 120

#     E.  Data access and processing   

#E1. The number of Sentinel-1 relative orbit. The default 
#       value is Auto. Auto means that the relative orbit that has
#       the Sentinel-1 image closer to the Flood_datetime is selected. 
#       S1_type can be GRD or SLC.
S1_type = GRD
relOrbit = Auto

#E3. The minimum mapping unit area in square meters
minimum_mapping_unit_area_m2=4000

kleok commented 1 year ago

Hello @JPPereira93, I think the issue regarding the ValueError: plotting, due to negative values in precipitation dataframe was resolved.

I also uploaded a notebook regarding your parameters (I slightly changed LON and LAT values). You can have a look, and try to run it by yourself :smile:
As you can see on the plots for Sentinel-1 images for [2020-09-01, 2020-09-07, 2020-09-19] you cannot visually identify a significant drop on the Sentinel-1 backscatter :slightly_frowning_face: in comparison with this notebook where we can clearly identify the flood water. According to my perspective, I suggest to drop this flood event because no safe conclusions can be drawn.

Let me know if I can do something else to help you :smile:

JPPereira93 commented 1 year ago

Thank you for the given help! I appreciate it very much

Yes I chose the same date as you did just to make a test running all the notebook from start to finish, I will choose an AOI and date where I know a flood event occured and I'll show it!

Best regards,

João Pereira

kleok / FLOODPY

Issue Installing snap.sh #36

Plot a time figure that Sentinel-1 acquisitions (post-flood and pre-flood are visualized)