Implement support for ZARR file subsetting and downloading

[pepijn-devries]

It should be possible to load the ZARR files in R. let's implement this feature...

[dancrear]

Hi @pepijn-devries,

Is your comment about ZARR files related to my inquiry? Can you expand more on what you are talking about here? Is this a way to download layers from BGC_001_028 product?

Thanks, Dan

[pepijn-devries]

Hi @dancrear,

The Copernicus Marine Service providing the ZARR files is not the same as the STAC service, so in that respect it is not related to your earlier inquiry (issue https://github.com/pepijn-devries/CopernicusMarine/issues/36 for the record). Copernicus Marine Services provide a subsetting services that delivers ZARR files, which is related to the service I have implemented as cms_download_subset() (which delivers netcdf files instead). The first produces smaller files and is therefore faster. But because of issue https://github.com/r-spatial/stars/issues/564, I will wait with implementing this feature until that issue has been resolved.

The issue reported here: https://github.com/pepijn-devries/CopernicusMarine/issues/33 is about the currently implemented subsetting feature (cms_download_subset()), which apparently does not work for that particular product. As it is unclear what is going wrong there, I will contact the people at Copernicus Marine about that. As an alternative I wish to implement the 'ZARR-feature', which has the problem stated above.

Hopefully, it's a bit more clear now? I wanted to add some links to some documentation about the services and their API, but I could not find any. Which is why I have to rely on reverse engineering on my part (which is not most efficient, I know).

Cheers,

Pepijn

[dancrear]

Hi,

I really appreciate the response and understand the situation now. Hopefully you hear back from Copernicus Marine soon. Thanks again!

Dan

On Sat, Feb 3, 2024 at 5:10 AM pepijn-devries @.***> wrote:

Hi @dancrear https://github.com/dancrear,

The Copernicus Marine Service providing the ZARR files is not the same as the STAC service, so in that respect it is not related to your earlier inquiry (issue #36 https://github.com/pepijn-devries/CopernicusMarine/issues/36 for the record). Copernicus Marine Services provide a subsetting services that delivers ZARR files, which is related to the service I have implemented as cms_download_subset() (which delivers netcdf files instead). The first produces smaller files and is therefore faster. But because of issue r-spatial/stars#564 https://github.com/r-spatial/stars/issues/564, I will wait with implementing this feature until that issue has been resolved.

The issue reported here: #33 https://github.com/pepijn-devries/CopernicusMarine/issues/33 is about the currently implemented subsetting feature (cms_download_subset()), which apparently does not work for that particular product. As it is unclear what is going wrong there, I will contact the people at Copernicus Marine about that. As an alternative I wish to implement the 'ZARR-feature', which has the problem stated above.

Hopefully, it's a bit more clear now? I wanted to add some links to some documentation about the services and their API, but I could not find any. Which is why I have to rely on reverse engineering on my part (which is not most efficient, I know).

Cheers,

Pepijn

— Reply to this email directly, view it on GitHub https://github.com/pepijn-devries/CopernicusMarine/issues/34#issuecomment-1925317325, or unsubscribe https://github.com/notifications/unsubscribe-auth/AWCCN23OBQOVTUFQRN5CMJTYRYZLPAVCNFSM6AAAAABCNKYRB6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMRVGMYTOMZSGU . You are receiving this because you were mentioned.Message ID: @.***>

-- Dan Crear, PhD

[pepijn-devries]

Hi @dancrear ,

Note that I'm touch now with the Copernicus Service desk, it may take a while to sort things out. I'll keep you posted here...

Cheers,

Pepijn