bopen / xarray-sentinel

Xarray backend to Copernicus Sentinel-1 satellite data products
Apache License 2.0
219 stars 22 forks source link

Add fsspec support to open_dataset #45

Closed alexamici closed 3 years ago

alexamici commented 3 years ago

Related to #15 but will only open metadata until rasterio supports fsspec.

codecov-commenter commented 3 years ago

Codecov Report

Merging #45 (18ddc5f) into main (afcb81f) will decrease coverage by 0.96%. The diff coverage is 86.95%.

Impacted file tree graph

@@            Coverage Diff             @@
##             main      #45      +/-   ##
==========================================
- Coverage   93.12%   92.15%   -0.97%     
==========================================
  Files           7        7              
  Lines         538      548      +10     
  Branches       62       65       +3     
==========================================
+ Hits          501      505       +4     
- Misses         20       23       +3     
- Partials       17       20       +3     
Impacted Files Coverage Δ
xarray_sentinel/sentinel1.py 92.23% <82.85%> (-2.59%) :arrow_down:
tests/test_esa_safe.py 100.00% <100.00%> (ø)
xarray_sentinel/esa_safe.py 81.75% <100.00%> (-0.40%) :arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update afcb81f...18ddc5f. Read the comment docs.

alexamici commented 3 years ago

At the moment all datasets except the bursts ones work with fsspec as well:

>>> from xarray_sentinel import sentinel1
>>> gs_urlpath = "gs://mybucket/S1B_IW_SLC__1SDV_20210401T052622_20210401T052650_026269_032297_EFA4.SAFE"
>>> sentinel1.open_dataset(gs_urlpath, group="IW1/orbit")
<xarray.Dataset>
Dimensions:       (azimuth_time: 17)
Coordinates:
  * azimuth_time  (azimuth_time) datetime64[ns] 2021-04-01T05:25:19 ... 2021-...
Data variables:
    x             (azimuth_time) float64 4.3e+06 4.359e+06 ... 5.187e+06
    y             (azimuth_time) float64 1.454e+06 1.453e+06 ... 1.408e+06
    z             (azimuth_time) float64 5.419e+06 5.372e+06 ... 4.593e+06
    vx            (azimuth_time) float64 5.963e+03 5.914e+03 ... 5.103e+03
    vy            (azimuth_time) float64 -91.12 -116.1 -140.9 ... -454.7 -478.0
    vz            (azimuth_time) float64 -4.695e+03 -4.756e+03 ... -5.602e+03
Attributes:
    reference_system:  Earth Fixed
    Conventions:       CF-1.7
    title:             Orbit information used by the IPF during processing
    comment:           The dataset contains a sets of orbit state vectors tha...
    history:           created by xarray_sentinel-0.2...

I'll add a test and consider merging this before attempting to get bursts to work as well.

alexamici commented 3 years ago

I added a test. I also found that an effort is underway to have rasterio / GDAL play well with fsspec, but it is far from operational, see: https://github.com/mapbox/rasterio/pull/2141

This is not the cleanest of the code, but the PR is big, I'd like have it merged now and clean up later.

Please review.