search

mapme-initiative / mapme.vegetation

https://mapme-initiative.github.io/mapme.vegetation/
GNU General Public License v3.0
3 stars 2 forks source link

The download function ignores the overwrite attribute and crashes on downloading the thumbnail #3

Open stpCES opened 2 years ago

stpCES commented 2 years ago 
items = download_aws(assets="overview",
+                      bbox = bbox,
+                      after = "2022-09-02",
+                      before = "2022-09-04",
+                      timeframe = "full",
+                      max.cloud = 30, outdir = "C:\\SOM-Togdheer\\R-programs",
+                      overwrite = TRUE,
+                      verbose = T,
+                      query_only = F,
+                      )
STAC version: 1.0.0-beta.2
Querying data for temporal window 2022-09-02/2022-09-04
The following items matched the specified spatiotemporal extent.
###STACItemCollection
- matched feature(s): 4
- features (4 item(s) / 0 not fetched):
  - S2A_38PMR_20220904_0_L2A
  - S2A_38PNR_20220904_0_L2A
  - S2A_38PMS_20220904_0_L2A
  - S2A_38PNS_20220904_0_L2A
- assets: 
thumbnail, overview, info, metadata, visual, B01, B02, B03, B04, B05, B06, B07, B08, B8A, B09, B11, B12, AOT, WVP, SCL
- other field(s): 
type, stac_version, stac_extensions, context, numberMatched, numberReturned, features, links
Starting download...
  |=========================                         |  50%Error: Error while downloading https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/38/P/NR/2022/9/S2A_38PNR_20220904_0_L2A/L2A_PVI.tif. 
Path exists and overwrite is FALSE

It actually downloads the file and afterwards says it already existing and overwrite would be false even though I specified it to be true. I changed the outdir in case it would have something to do with Windows access permissions but it doesn't change anything.

Moreover the function also interrupts when trying to download the all assets because the thumbnail is first and it gives the following error

Starting download...
  |============                                      |  25%Error: Error while downloading https://roda.sentinel-hub.com/sentinel-s2-l1c/tiles/38/P/MR/2022/9/4/0/preview.jpg. 
Unrecognized content encoding type. libcurl understands deflate, gzip content encodings.
goergen95 commented 2 years ago

Thanks! I see the same issue when R's internal downloader is used. I am unsure when I will be able to fix this (at least about 2 weeks). Please consider downloading the aria2c executable for your system, set use_aria = TRUE and point aria_bin towards where you installed the executable. This comes with the additional benefit of parallel downloads thus speeding up the whole process. Also consider if your really need to download all assets. For example, the visual asset is simply a RBG-composite of Band 02, 03, 04 thus this data would be downloaded twice.

stpCES commented 2 years ago

Yes, I started the download using the aria2c now. Let's see if it works for the whole data package.