search

umr-lops / grdwindinversion

python library to compute wind speed from GRD SAR images
https://cerweb.ifremer.fr/datarmor/doc_sphinx/grdwindinversion/index.html
MIT License
6 stars 1 forks source link

Error processing RCM files #61

Open Skealz opened 1 week ago

Skealz commented 1 week ago 
grdwindinversion 0.2.3.post10
xsarsea 1.0
xsar 2024.6.24

All RCM files are on error when processing with these version (latest)

Error :

Tue Jun 25 13:31:40 GMT 2024
@r2i2n23:/home1/datahome/sarwing%  SAR_L1-to-L2_wind_processor --input_file /home/datawork-cersat-public/provider/asc-csa/satellite/l1/rcm/rcm1/SCLND/GRD/2023/273/RCM1_OK2767220_PK2769320_1_SCLND_20230930_214014_VV_VH_GRD --outputdir /home/datawork-cersat-public/cache/public/ftp/project/sarwing/processings/2024v6v24_1v0_0v2v3vpost10_1v0v8_1v0v6_1v0v6_1v1v11_1v0v2_1v0v1_1v2v0/all_plugins_cmod5n_ecmwf/rcm1/L1/SCLND_GRD/2023/273/RCM1_OK2767220_PK2769320_1_SCLND_20230930_214014_VV_VH_GRD --config_file /home/datawork-cersat-public/cache/project/sarwing/configs/grdwind_cmod5n_ecmwf.yaml

====================================

25/06/2024 13:31:48 INFO main.py(38) input file: /home/datawork-cersat-public/provider/asc-csa/satellite/l1/rcm/rcm1/SCLND/GRD/2023/273/RCM1_OK2767220_PK2769320_1_SCLND_20230930_214014_VV_VH_GRD
25/06/2024 13:31:49 INFO schemas.py(1266) Resource 'XMLSchema.xsd' is already loaded
/home/datawork-cersat-public/cache/project/sarwing/commits/2024v6v24_1v0_0v2v3vpost10_1v0v8_1v0v6_1v0v6_1v1v11_1v0v2_1v0v1_1v2v0/grdwindinversion/lib/python3.10/site-packages/safe_rcm/product/reader.py:59: UserWarning: Converting non-nanosecond precision datetime values to nanosecond precision. This behavior can eventually be relaxed in xarray, as it is an artifact from pandas which is now beginning to support non-nanosecond precision values. This warning is caused by passing non-nanosecond np.datetime64 or np.timedelta64 values to the DataArray or Variable constructor; it can be silenced by converting the values to nanosecond precision ahead of time.
  {"timeStamp": ds["timeStamp"].astype("datetime64")}
/home/datawork-cersat-public/cache/project/sarwing/commits/2024v6v24_1v0_0v2v3vpost10_1v0v8_1v0v6_1v0v6_1v1v11_1v0v2_1v0v1_1v2v0/grdwindinversion/lib/python3.10/site-packages/safe_rcm/product/reader.py:68: UserWarning: Converting non-nanosecond precision datetime values to nanosecond precision. This behavior can eventually be relaxed in xarray, as it is an artifact from pandas which is now beginning to support non-nanosecond precision values. This warning is caused by passing non-nanosecond np.datetime64 or np.timedelta64 values to the DataArray or Variable constructor; it can be silenced by converting the values to nanosecond precision ahead of time.
  {"timeStamp": ds["timeStamp"].astype("datetime64")}
25/06/2024 13:31:53 ERROR io.py(236) FilePath is currently a read-only interface.
25/06/2024 13:31:53 ERROR io.py(236) FilePath is currently a read-only interface.
25/06/2024 13:31:53 INFO io.py(236) Object not found in virtual filesystem: filename=b'9cf56b5c-b642-42ef-b94d-efb101e8920a/9cf56b5c-b642-42ef-b94d-efb101e8920a.aux'
25/06/2024 13:31:53 INFO io.py(236) Object not found in virtual filesystem: filename=b'9cf56b5c-b642-42ef-b94d-efb101e8920a/9cf56b5c-b642-42ef-b94d-efb101e8920a.AUX'
25/06/2024 13:31:53 INFO io.py(236) Object not found in virtual filesystem: filename=b'9cf56b5c-b642-42ef-b94d-efb101e8920a/9cf56b5c-b642-42ef-b94d-efb101e8920a.aux'
25/06/2024 13:31:53 INFO io.py(236) Object not found in virtual filesystem: filename=b'9cf56b5c-b642-42ef-b94d-efb101e8920a/9cf56b5c-b642-42ef-b94d-efb101e8920a.AUX'
25/06/2024 13:31:53 ERROR io.py(236) FilePath is currently a read-only interface.
25/06/2024 13:31:53 ERROR io.py(236) FilePath is currently a read-only interface.
25/06/2024 13:31:53 INFO io.py(236) Object not found in virtual filesystem: filename=b'3f8eec4a-74b1-492d-aba5-21d553f154d2/3f8eec4a-74b1-492d-aba5-21d553f154d2.aux'
25/06/2024 13:31:53 INFO io.py(236) Object not found in virtual filesystem: filename=b'3f8eec4a-74b1-492d-aba5-21d553f154d2/3f8eec4a-74b1-492d-aba5-21d553f154d2.AUX'
25/06/2024 13:31:53 INFO io.py(236) Object not found in virtual filesystem: filename=b'3f8eec4a-74b1-492d-aba5-21d553f154d2/3f8eec4a-74b1-492d-aba5-21d553f154d2.aux'
25/06/2024 13:31:53 INFO io.py(236) Object not found in virtual filesystem: filename=b'3f8eec4a-74b1-492d-aba5-21d553f154d2/3f8eec4a-74b1-492d-aba5-21d553f154d2.AUX'
25/06/2024 13:31:58 INFO inversion.py(600) recalibration is False : Kersten formula is not applied
25/06/2024 13:31:58 INFO utils.py(479) BlockingActorProxy: Transparent proxy for from_dict
/home/datawork-cersat-public/cache/project/sarwing/commits/2024v6v24_1v0_0v2v3vpost10_1v0v8_1v0v6_1v0v6_1v1v11_1v0v2_1v0v1_1v2v0/grdwindinversion/lib/python3.10/site-packages/safe_rcm/product/reader.py:59: UserWarning: Converting non-nanosecond precision datetime values to nanosecond precision. This behavior can eventually be relaxed in xarray, as it is an artifact from pandas which is now beginning to support non-nanosecond precision values. This warning is caused by passing non-nanosecond np.datetime64 or np.timedelta64 values to the DataArray or Variable constructor; it can be silenced by converting the values to nanosecond precision ahead of time.
  {"timeStamp": ds["timeStamp"].astype("datetime64")}
/home/datawork-cersat-public/cache/project/sarwing/commits/2024v6v24_1v0_0v2v3vpost10_1v0v8_1v0v6_1v0v6_1v1v11_1v0v2_1v0v1_1v2v0/grdwindinversion/lib/python3.10/site-packages/safe_rcm/product/reader.py:68: UserWarning: Converting non-nanosecond precision datetime values to nanosecond precision. This behavior can eventually be relaxed in xarray, as it is an artifact from pandas which is now beginning to support non-nanosecond precision values. This warning is caused by passing non-nanosecond np.datetime64 or np.timedelta64 values to the DataArray or Variable constructor; it can be silenced by converting the values to nanosecond precision ahead of time.
  {"timeStamp": ds["timeStamp"].astype("datetime64")}
25/06/2024 13:32:02 ERROR io.py(236) FilePath is currently a read-only interface.
25/06/2024 13:32:02 ERROR io.py(236) FilePath is currently a read-only interface.
25/06/2024 13:32:02 INFO io.py(236) Object not found in virtual filesystem: filename=b'3a3794b8-4d75-44f1-af1c-7136b8f2e083/3a3794b8-4d75-44f1-af1c-7136b8f2e083.aux'
25/06/2024 13:32:02 INFO io.py(236) Object not found in virtual filesystem: filename=b'3a3794b8-4d75-44f1-af1c-7136b8f2e083/3a3794b8-4d75-44f1-af1c-7136b8f2e083.AUX'
25/06/2024 13:32:02 INFO io.py(236) Object not found in virtual filesystem: filename=b'3a3794b8-4d75-44f1-af1c-7136b8f2e083/3a3794b8-4d75-44f1-af1c-7136b8f2e083.aux'
25/06/2024 13:32:02 INFO io.py(236) Object not found in virtual filesystem: filename=b'3a3794b8-4d75-44f1-af1c-7136b8f2e083/3a3794b8-4d75-44f1-af1c-7136b8f2e083.AUX'
25/06/2024 13:32:02 ERROR io.py(236) FilePath is currently a read-only interface.
25/06/2024 13:32:02 ERROR io.py(236) FilePath is currently a read-only interface.
25/06/2024 13:32:02 INFO io.py(236) Object not found in virtual filesystem: filename=b'523c29f9-6aad-42b1-ac8f-e1a71536baad/523c29f9-6aad-42b1-ac8f-e1a71536baad.aux'
25/06/2024 13:32:02 INFO io.py(236) Object not found in virtual filesystem: filename=b'523c29f9-6aad-42b1-ac8f-e1a71536baad/523c29f9-6aad-42b1-ac8f-e1a71536baad.AUX'
25/06/2024 13:32:02 INFO io.py(236) Object not found in virtual filesystem: filename=b'523c29f9-6aad-42b1-ac8f-e1a71536baad/523c29f9-6aad-42b1-ac8f-e1a71536baad.aux'

5/06/2024 13:32:09 WARNING base_dataset.py(867) Raster variable are experimental
/home/datawork-cersat-public/cache/project/sarwing/commits/2024v6v24_1v0_0v2v3vpost10_1v0v8_1v0v6_1v0v6_1v1v11_1v0v2_1v0v1_1v2v0/grdwindinversion/lib/python3.10/site-packages/xarray/core/dataset.py:274: UserWarning: The specified chunks separate the stored chunks along dimension "Latitude" starting at index 1000. This could degrade performance. Instead, consider rechunking after loading.
  warnings.warn(
/home/datawork-cersat-public/cache/project/sarwing/commits/2024v6v24_1v0_0v2v3vpost10_1v0v8_1v0v6_1v0v6_1v1v11_1v0v2_1v0v1_1v2v0/grdwindinversion/lib/python3.10/site-packages/xarray/core/dataset.py:274: UserWarning: The specified chunks separate the stored chunks along dimension "Longitude" starting at index 1000. This could degrade performance. Instead, consider rechunking after loading.
  warnings.warn(
/home/datawork-cersat-public/cache/project/sarwing/commits/2024v6v24_1v0_0v2v3vpost10_1v0v8_1v0v6_1v0v6_1v1v11_1v0v2_1v0v1_1v2v0/grdwindinversion/lib/python3.10/site-packages/xsarsea/windspeed/windspeed.py:82: UserWarning: Unable to check sigma0 pol. Assuming  VH
  warnings.warn(
Traceback (most recent call last):
  File "/home/datawork-cersat-public/cache/project/sarwing/commits/2024v6v24_1v0_0v2v3vpost10_1v0v8_1v0v6_1v0v6_1v1v11_1v0v2_1v0v1_1v2v0/grdwindinversion/bin/SAR_L1-to-L2_wind_processor", line 10, in <module>
    sys.exit(processor_starting_point())
  File "/home/datawork-cersat-public/cache/project/sarwing/commits/2024v6v24_1v0_0v2v3vpost10_1v0v8_1v0v6_1v0v6_1v1v11_1v0v2_1v0v1_1v2v0/grdwindinversion/lib/python3.10/site-packages/grdwindinversion/main.py", line 53, in processor_starting_point
    out_file,outputds = makeL2(input_file, out_folder, config_file, overwrite=args.overwrite,resolution = resolution, generateCSV = args.no_generate_csv)
  File "/home/datawork-cersat-public/cache/project/sarwing/commits/2024v6v24_1v0_0v2v3vpost10_1v0v8_1v0v6_1v0v6_1v1v11_1v0v2_1v0v1_1v2v0/grdwindinversion/lib/python3.10/site-packages/grdwindinversion/inversion.py", line 784, in makeL2
    xr_dataset, encoding = makeL2asOwi(
  File "/home/datawork-cersat-public/cache/project/sarwing/commits/2024v6v24_1v0_0v2v3vpost10_1v0v8_1v0v6_1v0v6_1v1v11_1v0v2_1v0v1_1v2v0/grdwindinversion/lib/python3.10/site-packages/grdwindinversion/inversion.py", line 317, in makeL2asOwi
    xr_dataset = xr_dataset.rename({
  File "/home/datawork-cersat-public/cache/project/sarwing/commits/2024v6v24_1v0_0v2v3vpost10_1v0v8_1v0v6_1v0v6_1v1v11_1v0v2_1v0v1_1v2v0/grdwindinversion/lib/python3.10/site-packages/xarray/core/dataset.py", line 4317, in rename
    return self._rename(name_dict=name_dict, **names)
  File "/home/datawork-cersat-public/cache/project/sarwing/commits/2024v6v24_1v0_0v2v3vpost10_1v0v8_1v0v6_1v0v6_1v1v11_1v0v2_1v0v1_1v2v0/grdwindinversion/lib/python3.10/site-packages/xarray/core/dataset.py", line 4255, in _rename
    raise ValueError(
ValueError: cannot rename 'offboresight' because it is not a variable or dimension in this dataset
vincelhx commented 6 days ago

https://github.com/umr-lops/grdwindinversion/commit/d59bce0da648d810a850ce9e12b22aeaf5e76d7c will solve

vincelhx commented 6 days ago

in this commit i also checked all var attributes + dataset attributes. I just need a test with a RS2 file and i push

Skealz commented 2 days ago

Keep me in touch !

vincelhx commented 1 day ago

https://github.com/umr-lops/grdwindinversion/commit/979e2e33fdb917d23971a7818deec7829eb5301b : it should be ok now

vincelhx commented 1 day ago

close if it works