search

ASFOpenSARlab / opensarlab_MintPy_Recipe_Book

A Jupyter-Book containing data recipes for creating HyP3 Sentinel-1 IINSAR_GAMMA and INSAR_ISCE_BURST Small Baseline Subset (SBAS) Time Series Analyses with MintPy
BSD 3-Clause "New" or "Revised" License
23 stars 9 forks source link

Connect component file for missing pair Erorr and .kml export issue #7

Closed suatcoskun closed 5 months ago

suatcoskun commented 5 months ago

Today I tried to run MintPy with sentinel-1 Burst datasets for Teton Landslides. I prepared a seasonal stack and I added some custom pairs to bridge the seasons. But when I started Access_HyP3_data.ipyn and subset stacks, I took .geotiff and removed the warning message.

1

Then I continue the process stacks. But when I tried to run Load_Hyp3_data.ipny step2 (load the data with MintPy) I took the connect component file for missing pair warning and it shut down not continuously.

Ekran görüntüsü 2024-06-11 152334

Ekran görüntüsü 2024-06-11 152354

Then I turned back Access_HyP3 step and deleted the removed .geotiff folder manually step2 (load the data with MintPy) worked well. So I think when Access_HyP3_data.ipyn subsetting stack was not deleted removed geotiff folders and this caused some crash in step2.

The second interesting thing is about my custom pairs. Because Access_HyP3_data.ipyn (remove any subset scenes containing no data) deleted exactly my custom pairs.

prep_network (this is demostiration)

network_result

I think these will be bugs.

The last observation about MintPy; .kml export option doesn't work correctly. It doesn't open on Google Earth. I think it will be about different coordinate systems.

Best…

Suat

Alex-Lewandowski commented 5 months ago

Hi Suat, It is interesting that after subsetting, only the connected components of your custom pairs were deleted as containing no data. If you had subset the data to an area containing no data, I would expect that all the geotiffs for that product would be empty as well, and that they would also have been deleted.

In your screenshots, it appears that you are using an old version of the Jupyter Book. We just released an update yesterday. Specifically, I noticed that the cells immediately prior to checking for rasters containing no data after subsetting do not contain the new code, which allows you to convert the data from UTM to WGS84:

image

If you are working locally, I recommend pulling in the recent update.

If you are working in OpenSARLab I recommend stopping and restarting your server, which will sync the repo, pulling in the updates.

The updates include changes to the environment, so you should rerun the 1_Software_Environment.ipynb notebook. One of the environment updates installs MintPy from a recent commit, which resolves the KMZ issue that you mentioned: https://github.com/insarlab/MintPy/pull/1210

If you still run into trouble after pulling in the update, recreating the environment, and starting the process over by accessing a fresh copy of the data, please let me know and send the following in a private message:

You can email that to me at aflewandowski@alaska.edu

With that information, I can access the exact dataset you are working with and help debug the issue.

Best, Alex

Alex-Lewandowski commented 5 months ago

Hi @suatcoskun,

Thank you for sending the details of your HyP3 project.

I downloaded all 92 interferograms and subsetted to pretty much the entire common area covered by the stack, meaning every subset raster had coverage of the selected AOI: image

No subsets were deleted as empty: image

I was able to load the time series without removing any interferograms: image

I then tried subsetting to an area larger than the rasters, just to see if I could break things, but once again, no rasters were found to be empty and deleted.

If you continue to have trouble, please email a copy of the data access notebook with all output saved in place so I can attempt to reproduce the problem. I suspect that if there is an issue, it has to do with subsetting.

Best, Alex

suatcoskun commented 5 months ago

Hi @Alex-Lewandowski

I continue to have trouble but this time I have an idea what is causing this issue. If you are subsetting stacks like you with a bounding box I have no error message.

box

no_removed

But if I use a smaller area and .shp file subseting some .tiff files were removed.  

shp_file

result_shp

(I sent my notebook access permission to your mail address.)

Best Suat

Alex-Lewandowski commented 5 months ago

Thanks for the update, @suatcoskun!

This is very helpful. I'll look into the issue when using a shapefile for subsetting.

Alex-Lewandowski commented 5 months ago

Hi @suatcoskun,

Thank you for sending the shapefile you were using. That clarified the source of the problem nicely!

The connected components "delineate regions unwrapped as contiguous units by the SNAPHU unwrapping algorithm.".

No data values in the connected components files are perfectly valid, but we do not account for that in the notebook and end up deleting connected components files filled with nans.

The connected components for your custom interferograms contain only nans in your AOI, and this why they were being deleted. You can see that there is no data in your AOI in the screenshot of one of the connected components files below:

image

Thank you so much for bringing this to our attention and working with us to identify the source of the problem. I will update the notebook so it does not delete connected components files containing no data if they are accompanied by unwrapped phase rasters that do contain data, as they are valid.

I will leave this issue open until we release the fix.

Alex-Lewandowski commented 5 months ago

Closing as resolved by https://github.com/ASFOpenSARlab/opensarlab_MintPy_Recipe_Book/pull/8