appelmar / gdalcubes_cpp

Earth observation data cubes from GDAL image collections
MIT License
74 stars 7 forks source link

Sentinel 2 unzipped SAFE not read #32

Closed micha-silver closed 4 years ago

micha-silver commented 4 years ago

I'm trying to read into an image collection a list of directories of (unzipped) Sentinel 2 data. I get an empty collection. However the same procedure run on zipped files works. My workflow begins with the R package sen2r to download the tiles, and this package automatically unzips the the downloaded file into a *.SAFE directory. So I need gdalcubes to read in the SAFE dir.

Here's what happens:

S2_SAFE_list = list.files(path = S2_download_dir, pattern = ".*SAFE$",  full.names = TRUE)
S2_SAFE_list
 [1] "/media/micha/MY_DATA/Data/DWH/Sentinel2/S2A_MSIL2A_20190908T081601_N0213_R121_T36RXV_20190908T110603.SAFE"
 [2] "/media/micha/MY_DATA/Data/DWH/Sentinel2/S2A_MSIL2A_20190918T081601_N0213_R121_T36RXV_20190918T131954.SAFE"
 [3] "/media/micha/MY_DATA/Data/DWH/Sentinel2/S2A_MSIL2A_20190928T081721_N0213_R121_T36RXV_20190928T110727.SAFE"
 [4] "/media/micha/MY_DATA/Data/DWH/Sentinel2/S2A_MSIL2A_20191008T081831_N0213_R121_T36RXV_20191008T110004.SAFE"
 [5] "/media/micha/MY_DATA/Data/DWH/Sentinel2/S2A_MSIL2A_20191018T081941_N0213_R121_T36RXV_20191018T113720.SAFE"
 [6] "/media/micha/MY_DATA/Data/DWH/Sentinel2/S2A_MSIL2A_20191028T082041_N0213_R121_T36RXV_20191028T095551.SAFE"
 [7] "/media/micha/MY_DATA/Data/DWH/Sentinel2/S2A_MSIL2A_20191107T082141_N0213_R121_T36RXV_20191107T093242.SAFE"
 [8] "/media/micha/MY_DATA/Data/DWH/Sentinel2/S2A_MSIL2A_20191117T082221_N0213_R121_T36RXV_20191117T114817.SAFE"
 [9] "/media/micha/MY_DATA/Data/DWH/Sentinel2/S2A_MSIL2A_20191127T082301_N0213_R121_T36RXV_20191127T093536.SAFE"
[10] "/media/micha/MY_DATA/Data/DWH/Sentinel2/S2B_MSIL2A_20190903T081609_N0213_R121_T36RXV_20190903T120314.SAFE"
[11] "/media/micha/MY_DATA/Data/DWH/Sentinel2/S2B_MSIL2A_20190913T081609_N0213_R121_T36RXV_20190913T125058.SAFE"
[12] "/media/micha/MY_DATA/Data/DWH/Sentinel2/S2B_MSIL2A_20190923T081639_N0213_R121_T36RXV_20190923T124312.SAFE"
[13] "/media/micha/MY_DATA/Data/DWH/Sentinel2/S2B_MSIL2A_20191013T081859_N0213_R121_T36RXV_20191013T124529.SAFE"
[14] "/media/micha/MY_DATA/Data/DWH/Sentinel2/S2B_MSIL2A_20191023T082009_N0213_R121_T36RXV_20191023T131005.SAFE"
[15] "/media/micha/MY_DATA/Data/DWH/Sentinel2/S2B_MSIL2A_20191102T082009_N0213_R121_T36RXV_20191102T111125.SAFE"
[16] "/media/micha/MY_DATA/Data/DWH/Sentinel2/S2B_MSIL2A_20191112T082109_N0213_R121_T36RXV_20191112T113452.SAFE"
[17] "/media/micha/MY_DATA/Data/DWH/Sentinel2/S2B_MSIL2A_20191122T082149_N0213_R121_T36RXV_20191122T113933.SAFE"

S2_collection = create_image_collection(files = S2_SAFE_list, format='Sentinel2_L2A', unroll_archives=TRUE, out_file=file.path(GIS_dir, "S2_collection.db"))
0%   10   20   30   40   50   60   70   80   90   100%
[----|----|----|----|----|----|----|----|----|----|
**************************************************|

S2_collection
A GDAL image collection object, referencing 0 images with 0  bands
Images:
[1] name     left     top      bottom   right    datetime srs     
<0 rows> (or 0-length row.names)

Bands:
[1] name        offset      scale       unit        nodata      image_count
<0 rows> (or 0-length row.names)

Again, if I zip the SAFE directories, and make a list of the zipfiles, then create_image_collection() succeeds. Do I need to make any changes to the Sentinel_2A.json ?

Thanks, Micha

appelmar commented 4 years ago

The content of zip files is automatically scanned recursively (if unroll_archives = TRUE), i.e. it will find the .jp2 files in subdirectories. So, instead of the .SAFE directory, you need to pass the paths to the .jp2 files directly, e.g. by replacing the first line with:

S2_SAFE_list = list.files(path = S2_download_dir, pattern = ".jp2$",  full.names = TRUE, recursive = TRUE)

Does this work for you?

micha-silver commented 4 years ago
body p { margin-bottom: 0cm; margin-top: 0pt; } 

Hi Marius
On 02/12/2019 11:28, Marius Appel
  wrote:

  The content of zip files is automatically scanned recursively
    (if unroll_archives = TRUE), i.e. it will find the
    .jp2 files in subdirectories. So, instead of the .SAFE
    directory, you need to pass the paths to the .jp2 files
    directly, e.g. by replacing the first line with:
  S2_SAFE_list = list.files(path = S2_download_dir, pattern = ".jp2$",  full.names = TRUE, recursive = TRUE)

Now I've got it.  Thanks.

  Does this work for you?
  —
    You are receiving this because you authored the thread.
    Reply to this email directly, view it on GitHub, or unsubscribe.
  [

{ "@context": "http://schema.org", "@type": "EmailMessage", "potentialAction": { "@type": "ViewAction", "target": "https://github.com/appelmar/gdalcubes/issues/32?email_source=notifications\u0026email_token=AA5J52GZYMASQNEEJ7CFQD3QWTILDA5CNFSM4JTNMYC2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEFS2PLY#issuecomment-560310191", "url": "https://github.com/appelmar/gdalcubes/issues/32?email_source=notifications\u0026email_token=AA5J52GZYMASQNEEJ7CFQD3QWTILDA5CNFSM4JTNMYC2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEFS2PLY#issuecomment-560310191", "name": "View Issue" }, "description": "View this Issue on GitHub", "publisher": { "@type": "Organization", "name": "GitHub", "url": "https://github.com" } } ]

-- 

Micha Silver Arava Drainage Authority +972-523-665918