search

ymcmrs / ICAMS

Python based toolbox for InSAR troposphere Correction using global Atmospheric Models that considers the spatial Stochastic properties of the troposphere (ICAMS).
32 stars 9 forks source link

Error getting interpolation in ERA files #5

Open erik-17 opened 1 year ago

erik-17 commented 1 year ago

Hi, I am getting this message after running an instance of tropo_icams_sar.py 

  File "/home/erikr/miniconda3/envs/mintpy/lib/python3.11/site-packages/scipy/interpolate/_rgi.py", line 27, in _check_points
    raise ValueError(
ValueError: The points in dimension 1 must be strictly ascending or descending

Apparently the issue points to this line:

fP = RGI((latv[::-1], lonv), Presi0[::-1,:],method='linear',bounds_error=False,fill_value=None)

EDIT: I feel there is an issue with areas that have the prime meridian in it

yunjunz commented 1 year ago

mintpy users reported the same issue before while using pyaps (https://github.com/insarlab/MintPy/issues/447): it was because the study area was across the 0 longitudes. The error is fixed on the pyaps side by setting ERA5 longitude range from [0, 360] to [-180, 180] (https://github.com/insarlab/PyAPS/commit/804b884016eb860862e684e1c187afd41964c31b).

ymcmrs commented 1 year ago

Can you please point out this is from which line of which function? The longitude should be all adjusted to [-180, 180] in all functions, but yes, the edge problem is still there, e.g., from 180 to -180.

If you can point out which function, you could try to add the following lines on the top lines: if np.mean(lonlist) > 150: lon_intp[lonlist<0] = lon_intp[lonlist<0] + 360

From: Erik Rivas @.> Date: Saturday, 17 June 2023 at 1:24 AM To: ymcmrs/ICAMS @.> Cc: Subscribed @.***> Subject: [ymcmrs/ICAMS] Error getting interpolation in ERA files (Issue #5)

Hi, I am getting this message after running an instance of tropo_icams_sar.py

File "/home/erikr/miniconda3/envs/mintpy/lib/python3.11/site-packages/scipy/interpolate/_rgi.py", line 27, in _check_points

raise ValueError(

ValueError: The points in dimension 1 must be strictly ascending or descending

Apparently the issue points to this line:

fP = RGI((latv[::-1], lonv), Presi0[::-1,:],method='linear',bounds_error=False,fill_value=None)

— Reply to this email directly, view it on GitHubhttps://github.com/ymcmrs/ICAMS/issues/5, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AFQZYFAHCKT6LGSGXR73IOTXLRNCBANCNFSM6AAAAAAZJIKF5E. You are receiving this because you are subscribed to this thread.Message ID: @.***>

erik-17 commented 1 year ago

Hi Sorry I wasnt so especific... here it is..

(base) erikr@mypc:~/RAIDER_test_files$ tropo_icams_sar.py ml_data.h5 --sar-par par_files/20191123.rslc.par -o out_icams --project los
/home/erikr/ICAMS/icams/tropo_icams_sar.py:22: DeprecationWarning: Please use `pearsonr` from the `scipy.stats` namespace, the `scipy.stats.stats` namespace is deprecated.
  from scipy.stats.stats import pearsonr
Average geoid height: 45.74986177448949
Start to calc LOS locations ...
Start to calc LOS atmospheric parameters ...
Traceback (most recent call last):---------------------------------------------] 2.0%    complete
  File "/home/erikr/ICAMS/icams/tropo_icams_sar.py", line 507, in <module>
    main(sys.argv[:])
  File "/home/erikr/ICAMS/icams/tropo_icams_sar.py", line 442, in main
    LosP,LosT,LosV = ut.get_LOS_parameters(latlist,lonlist,Presi,Tempi,Vpri,lat_intp, lon_intp,inps.method,inps.sklm_points_numb)
  File "/home/erikr/ICAMS/icams/_utils.py", line 1192, in get_LOS_parameters
    fP = RGI((latv[::-1], lonv), Presi0[::-1,:],method='linear',bounds_error=False,fill_value=None)
  File "/home/erikr/miniconda3/lib/python3.9/site-packages/scipy/interpolate/_interpolate.py", line 2485, in __init__
    raise ValueError("The points in dimension %d must be strictly "
ValueError: The points in dimension 1 must be strictly ascending

Following the error it made me change on line 279 of _utils.py (get_ecmwf) from 

mask = ((lats > minlat) & (lats < maxlat)) & ((lons > minlon) & (lons < maxlon))

to 

mask = ((lats > minlat) & (lats < maxlat)) & ((lons >= minlon) & (lons < maxlon))

since it was excluding values in the longitude. But the problem persisted in next steps due to inconsistencies of the dimensions of intermediate products.

my SLC Corners are: 'LAT_REF1', 'LAT_REF2', 'LAT_REF3', 'LAT_REF4' = 50.38287809, 50.67749471, 50.89605841, 51.19144711 'LON_REF1', 'LON_REF2', 'LON_REF3', 'LON_REF4' = -1.86679666, 0.6585912, -2.03178269, 0.52211881

ymcmrs commented 1 year ago

Just made a quick local test, using an example that the longitude across zero:

tropo_icams_date.py 20201024 --region " -0.4/0.5/31/33 " --imaging-time 11:24 --resolution 200.0

This works well for me, with DEM downloaded automatically. So the problems seems due to the provided geometry file, in your case it should be ml_data.h5, which should include radar-coordinates based lookup tables. Please make a double check if there is any problems with the rdc-lookup table. From: Erik Rivas @.> Date: Monday, 19 June 2023 at 8:47 PM To: ymcmrs/ICAMS @.> Cc: Yunmeng Cao @.>, Comment @.> Subject: Re: [ymcmrs/ICAMS] Error getting interpolation in ERA files (Issue #5)

Hi Sorry I wasnt so especific... here it is..

(base) @.***:~/RAIDER_test_files$ tropo_icams_sar.py ml_data.h5 --sar-par par_files/20191123.rslc.par -o out_icams --project los

/home/erikr/ICAMS/icams/tropo_icams_sar.py:22: DeprecationWarning: Please use pearsonr from the scipy.stats namespace, the scipy.stats.stats namespace is deprecated.

from scipy.stats.stats import pearsonr

Average geoid height: 45.74986177448949

Start to calc LOS locations ...

Start to calc LOS atmospheric parameters ...

Traceback (most recent call last):---------------------------------------------] 2.0% complete

File "/home/erikr/ICAMS/icams/tropo_icams_sar.py", line 507, in 

main(sys.argv[:])

File "/home/erikr/ICAMS/icams/tropo_icams_sar.py", line 442, in main

LosP,LosT,LosV = ut.get_LOS_parameters(latlist,lonlist,Presi,Tempi,Vpri,lat_intp, lon_intp,inps.method,inps.sklm_points_numb)

File "/home/erikr/ICAMS/icams/_utils.py", line 1192, in get_LOS_parameters

fP = RGI((latv[::-1], lonv), Presi0[::-1,:],method='linear',bounds_error=False,fill_value=None)

File "/home/erikr/miniconda3/lib/python3.9/site-packages/scipy/interpolate/_interpolate.py", line 2485, in init

raise ValueError("The points in dimension %d must be strictly "

ValueError: The points in dimension 1 must be strictly ascending

Following the error it made me change on line 279 of _utils.py (get_ecmwf) from

mask = ((lats > minlat) & (lats < maxlat)) & ((lons > minlon) & (lons < maxlon))

to

mask = ((lats > minlat) & (lats < maxlat)) & ((lons >= minlon) & (lons < maxlon))

since it was excluding values in the longitude. But the problem persisted in next steps due to inconsistencies of the dimensions of intermediate products.

my SLC Corners are: 'LAT_REF1', 'LAT_REF2', 'LAT_REF3', 'LAT_REF4' = 50.38287809, 50.67749471, 50.89605841, 51.19144711 'LON_REF1', 'LON_REF2', 'LON_REF3', 'LON_REF4' = -1.86679666, 0.6585912, -2.03178269, 0.52211881

— Reply to this email directly, view it on GitHubhttps://github.com/ymcmrs/ICAMS/issues/5#issuecomment-1596768918, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AFQZYFAMD7R2HKARYDOOMTTXMAGZ7ANCNFSM6AAAAAAZJIKF5E. You are receiving this because you commented.Message ID: @.***>