Get trial annual mosaic from Stef

NLeSC / team-atlas

1 stars 0 forks source link

Get trial annual mosaic from Stef #188

Closed meiertgrootes closed 3 years ago

meiertgrootes commented 3 years ago

This can form the basis of experimentation with the retiling tools and storage

fnattino commented 3 years ago

The titles of the annual mosaic exported by Maaike from GEE to a GCS bucket have been dowloaded to the SURF dCache storage (download has been performed as illustrated in this notebook)

fnattino commented 3 years ago

Tiles are now available at this path: /pnfs/grid.sara.nl/data/iceshelf/disk/S2_composite_2019-11-1_2020-3-1/

fnattino commented 3 years ago

After realizing that there was a problem with the data, tiles have been copied again. To speed up the process I have:

implemented parallel downloading using multiprocessing;
run the download on Spider - faster connection to dCache. The scripts employed on Spider are available here.

fnattino commented 3 years ago

Tiles have been also copied on Cartesius for easier access during learning, to the following path: /projects/0/einf512/S2_composite_2019-11-1_2020-3-1/