`tropo_icams_sar.read_par_orb`: modify to read isce2 `IW1.xml`

[yuankailiu]

Hello,

I wonder if you have updated the codes recently to implement the kringing method as the default interpolation method in tropo_icams.py and/or tropo_icams_sar.py?

I am trying to run ICAMS given MIntPy timeseries.h5 and geometryGeo.h5 as input. But it looks like several places in both scripts need to be updated to replace the deprecated sklm method with kriging.

In the README.md, the suggested MintPy user command lines:

 tropo_icams_sar.py  or  tropo_icams.py

 e.g. :
    tropo_icams_sar.py geometryRadar.h5 --date 20180101 
    tropo_icams.py timeseries.h5 geometryRadar.h5 --project zenith --method sklm
    tropo_icams.py timeseries.h5 geometryRadar.h5 --sar-par 20181025.slc.par --project los --method sklm    (if you are a GAMMA user) 

[yuankailiu]

Not a big issue, I think sklm keywords can be easily replaced by kriging.

I see that your example above supports using --sar-par 20181025.slc.par for GAMMA to compute the LOS delay. If I want to use MintPy geometryGeo.h5 to compute the LOS integrated delay, can this be done?

I suppose that SLC_par file is only in GAMMA format (https://github.com/ymcmrs/ICAMS/blob/main/icams/tropo_icams_sar.py#L306-L324). Do you know how can I use similar fields in ISCE2 --> MintPy products?

Thanks!

[ymcmrs]

Hello,

Yes, you can use ‘–project zenith’ (default) for without GAMMA-par file, as no need to read orbital datas from the par file. If you have incidenceAngle in geometryRadar, yes, it’s perfect, if no, you can add one use –incAngle

There are several modifications in PyKrige, please isntall this version: $CONDA_PREFIX/bin/pip install git+https://github.com/ymcmrs/PyKrige.git

One example of tropo_icams_sar.py: tropo_icams_sar.py geometryRadar.h5 –date 20180101 –ref-file velocity.h5 Here use ref-file is to provide imaging timing, earth radium, etc, as which are not always included in geoemtryRadar, and you also don’t provide slc-par here.

One example of tropo_icams.py: tropo_icams.py timeseries.h5 geometryRadar.h5 –ref-file velocity.h5 Here is a bit stupid to call -ref-file again, as tropo_icams_sar.py is called, and not as sub-function, you can modify this easily.

Using tropo_icams_sar.py as an independent script (not a sub-function in tropo_icams.py), is to make it more flexible, particularly for High-performance-computer users.

Hope this helps Cheers Yunmeng From: Yuan-Kai Liu @.> Reply to: ymcmrs/ICAMS @.> Date: Wednesday, 19 April 2023 at 11:29 AM To: ymcmrs/ICAMS @.> Cc: Subscribed @.> Subject: Re: [ymcmrs/ICAMS] updated code for kriging (Issue #3)

Not exactly a big issue, I think sklm keyword can be easily replaced by kriging. Also, I can try edit the code to read the incidenceAngle from the geometry file.

But if you already have an updated version code, that would be great though!

— Reply to this email directly, view it on GitHubhttps://github.com/ymcmrs/ICAMS/issues/3#issuecomment-1513909169, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AFQZYFHVB4ASBRULP43HRX3XB4PXLANCNFSM6AAAAAAXDHNRQM. You are receiving this because you are subscribed to this thread.Message ID: @.***>

[yuankailiu]

Hi @ymcmrs,

Thanks for your reply. I do have Krigie installed via $CONDA_PREFIX/bin/pip install git+https://github.com/ymcmrs/PyKrige.git Thanks for updating that --ref-file in tropo_icams_sar.py so quickly.

But if I want to do the LOS integral delay (instead of projecting zenith to LOS), I will need the GAMMA-par file, correct?

[ymcmrs]

Yes, if you want to do LOS integral, you need to provide slc-par file or any other file can provide orbital data, and currently only support GAMMA par. But it’s easy to read other software-based par files, e.g., check read_par_orb function in tropo_icams_sar.py

From: Yuan-Kai Liu @.> Reply to: ymcmrs/ICAMS @.> Date: Wednesday, 19 April 2023 at 12:03 PM To: ymcmrs/ICAMS @.> Cc: Yunmeng Cao @.>, Mention @.***> Subject: Re: [ymcmrs/ICAMS] updated code for kriging (Issue #3)

Hi @ymcmrshttps://github.com/ymcmrs,

Thanks for your reply. I do have Krigie installed via $CONDA_PREFIX/bin/pip install git+https://github.com/ymcmrs/PyKrige.git Thanks for updating that --ref-file in tropo_icams_sar.py so quickly.

But if I want to do the LOS integral delay (instead of projecting zenith to LOS), I will need the GAMMA-par file, correct?

— Reply to this email directly, view it on GitHubhttps://github.com/ymcmrs/ICAMS/issues/3#issuecomment-1513927888, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AFQZYFCGIS3DJBW5CGBEZSDXB4TUHANCNFSM6AAAAAAXDHNRQM. You are receiving this because you were mentioned.Message ID: @.***>

[yuankailiu]

I see. Could you provide me with one of such SAR.slc.par file for me? I can try and see what is the required format and in/outputs. That would be helpful. Thanks.

It looks like I only need one (e.g., SAR.slc.par) where it has the orbit info of all acquisitions.

[ymcmrs]

Basically it gets four columns of orbit states from par file: Time + Position (X Y Z). Input format check the above mentioned function.

Get Outlook for iOShttps://aka.ms/o0ukef

---

From: Yuan-Kai Liu @.> Sent: Wednesday, April 19, 2023 6:37:22 PM To: ymcmrs/ICAMS @.> Cc: Yunmeng Cao @.>; Mention @.> Subject: Re: [ymcmrs/ICAMS] updated code for kriging (Issue #3)

I see. Could you provide me with one of such SAR.slc.par file for me? I can try and see what is the required format and in/outputs. That would be helpful. Thanks.

It looks like I only need one (e.g., SAR.slc.par) where it has the orbit info of all acquisitions.

— Reply to this email directly, view it on GitHubhttps://github.com/ymcmrs/ICAMS/issues/3#issuecomment-1514204098, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AFQZYFG3ZKXGRY2K7X5AHV3XB6B2FANCNFSM6AAAAAAXDHNRQM. You are receiving this because you were mentioned.Message ID: @.***>

[yuankailiu]

In GAMMA, each interferogram has its own YYYYMMDD.slc.par listing the orbit states.

When I do time series correction, tropo_icams.py timeseries.h5 geometryRadar.h5 --sar-par SAR.slc.par, this single SAR.slc.par should contain all the orbit states from all the dates?

[ymcmrs]

almost the same. given ~100m position shift to ~800km

Get Outlook for iOShttps://aka.ms/o0ukef

---

From: Yuan-Kai Liu @.> Sent: Saturday, April 22, 2023 4:23:15 PM To: ymcmrs/ICAMS @.> Cc: Yunmeng Cao @.>; Mention @.> Subject: Re: [ymcmrs/ICAMS] updated code for kriging (Issue #3)

In GAMMA, each interferogram has its own YYYYMMDD.slc.par listing the orbit states.

When I do time series correction, tropo_icams.py timeseries.h5 geometryRadar.h5 --sar-par SAR.slc.par, this single SAR.slc.par should contain all the orbit states from all the dates?

— Reply to this email directly, view it on GitHubhttps://github.com/ymcmrs/ICAMS/issues/3#issuecomment-1518504189, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AFQZYFCUY3UBU54N6DR4JLLXCNMLHANCNFSM6AAAAAAXDHNRQM. You are receiving this because you were mentioned.Message ID: @.***>

[yuankailiu]

ah, okay, thanks!

[yuankailiu]

Reading orbits and timings from ISCE2 IW*.xml works well for me. Thanks

Just a follow-up question about computation. I am working on a longer track spanning ~6 latitude degrees across 7 SAR tracks. The get_LOS_parameters (Start to calc LOS atmospheric parameters ...) step seems to be the most time-consuming part. I assume this is not parallelized (OMP_NUM_THREADS) now?

I tried setting the OMP=4 (on a server with 32 fully idle cores), but still not efficient. By any chance, have you run into an issue that runs super slow? E.g., only calculates 20 dates of LOS parameters with 12 hours of computation.

[ymcmrs]

Well done! Good to know xml file works well for you.

Regarding to get_LOS_parameteres, because tropo_icams.py calls ‘tropo_icams_sar.py’, so for each SAR image, the ‘LOS locations’ would be calculated repeatedly, which does take some time and not necessary since all of the images share the same ‘LOS location’.

So if you are trying to process time-series of datasets, one option to improve the efficiency (if you feel too slow) is to calculate ‘LOS location’ at the beginning, and then use this for all of the images. In this case, you need to re-define tropo_icams_sar as one sub-funciton of tropo_icams.py. From: Yuan-Kai Liu @.> Reply to: ymcmrs/ICAMS @.> Date: Tuesday, 25 April 2023 at 6:07 AM To: ymcmrs/ICAMS @.> Cc: Yunmeng Cao @.>, Mention @.***> Subject: Re: [ymcmrs/ICAMS] tropo_icams_sar.read_par_orb: modify to read isce2 IW1.xml (Issue #3)

Reading orbits and timings from ISCE2 IW*.xml works well for me. Thanks

Just a follow-up question about computation. I am working on a longer track spanning ~6 latitude degrees across 7 SAR tracks. The get_LOS_parametershttps://github.com/ymcmrs/ICAMS/blob/main/icams/tropo_icams_sar.py#L439-L440 (Start to calc LOS atmospheric parameters ...) step seems to be the most time-consuming part. I assume this is not parallelized now (OMP_NUM_THREADS) now?

Have you run into an issue that runs super slow? E.g., only calculates 20 dates of LOS parameters with 12 hours of computation.

— Reply to this email directly, view it on GitHubhttps://github.com/ymcmrs/ICAMS/issues/3#issuecomment-1520609991, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AFQZYFFQITM3JXQ7UZ5CX2DXC26MHANCNFSM6AAAAAAXDHNRQM. You are receiving this because you were mentioned.Message ID: @.***>

[yuankailiu]

That seems to be the right way for me. Thanks.

[yuankailiu]

it is true that tropo_icams.py calls tropo_icams_sar.py repeatedly for the time series.

After looking at the tropo_icams_sar.py, it does check the LOS location file (lat_intp_binary) and does not repeatedly compute the locations.

The main sluggishness occurs on the major computation inut.get_LOS_parameters where I think it uses pykrige to upsample/weight the ERA5 grids to get LosP, LosT, LosV at each level. So it depends on the size of the model grids (my InSAR spatial coverage).

I guess I can, in turn, use Python multithreading to parallelize the dates in this loop in tropo_icams.py to call several tropo_icams_sar.py at the same time.

[yuankailiu]

Oh, my! I know what my stupid issue is. I have all the ERA5 data (from all kinds of different regions globally) stored in the same folder.

So the script tries to get the whole lat lon to cover all these regions (across 20 deg lat and lon). That is why the model grid gets super big!

So I should just let each track have its own ERA5 folder, not mixing all .grb.

Now this is working fine. thanks.