Error in lst_todo[[1]] : subscript out of bounds

[t-stratmann]

First of all, thank-you for this great package!!! It has saved me a lot of time! I just have a question. Last week I was able to successfully use the runGdal function and download HDF files, stitch them, and convert to TIFF. Now I am working in a different area and am getting a confusing error. All it says is "Error in lst_todo[[1]] : subscript out of bounds" The error message is not really enough to help me troubleshoot the issue. Any help / insight would be greatly appreciated!

Thank-you!!!

Script and console output below

sessionInfo()

R version 3.4.3 (2017-11-30) Platform: x86_64-w64-mingw32/x64 (64-bit) Running under: Windows 7 x64 (build 7601) Service Pack 1

other attached packages: [1] RCurl_1.95-4.10 bitops_1.0-6 rgdal_1.2-18 MODIS_1.1.2 raster_2.6-7 mapdata_2.2-6 maps_3.2.0 sp_1.2-7

library(MODIS)

transform dates to correct formats (begin, end)

Nov 21, - Dec. 2, 2017

transDate("2017-11-21", "2017-12-02") $begin [1] "2017-11-21"

$end [1] "2017-12-02"

$beginDOY [1] "2017325"

$endDOY [1] "2017336"

get correct extent information

tile <-getTile(extent(c(xmin=32, xmax=42, ymin=-5, ymax=5))) tile An object of class "MODISextent" Slot "tile": [1] "h21v09" "h22v09" "h21v08" "h22v08"

Slot "tileH": [1] 21 22

Slot "tileV": [1] 9 8

Slot "extent": class : Extent xmin : 32 xmax : 42 ymin : -5 ymax : 5

Slot "system": [1] "MODIS"

Slot "target": NULL

MODIS:::checkTools() Checking availability of MRT: 'MRT_HOME' found: C:/Users/.../Documents/MRT 'MRT_DATA_DIR' found: C:/Users/.../Documents/MRT/data MRT enabled, settings are fine! Checking availability of 'FWTools/OSGeo4W' (GDAL with HDF4 support for Windows): OK, GDAL 1.11.2, released 2015/02/10 found!

set relevant options

MODISoptions(localArcPath = "C:/Users/.../Documents/MODIS_ARC", # raw hdf files outDirPath = "C:/Users/.../Documents/MODIS_ARC/PROCESSED", # extracted layers MODISserverOrder = c("LAADS", "LPDAAC")) Setting 'localArcPath' to 'C:/Users/.../Documents/MODIS_ARC/' If you already have downloaded some HDF files, you can use '?orgStruc' to rearrange them. 'outDirPath' does not exist and will be created in 'C:/Users/.../Documents/MODIS_ARC/PROCESSED/'.

STORAGE:

---

localArcPath : C:/Users/.../Documents/MODIS_ARC/ outDirPath : C:/Users/.../Documents/MODIS_ARC/PROCESSED/

DOWNLOAD:

---

MODISserverOrder : LAADS, LPDAAC dlmethod : auto stubbornness : high wait : 0.5 quiet : FALSE

PROCESSING:

---

GDAL : GDAL 1.11.2, released 2015/02/10 MRT : Version 4.1 (March 2011) pixelSize : LAADS outProj : LPDAAC resamplingType : NN dataFormat : GTiff

get latest product collection

cll <- getCollection(product = "MOD13Q1", forceCheck = TRUE) Updating collection from LPDAAC for product: MOD13Q1 cll $MOD13Q1 [1] "006"

download and extract data

runGdal(product = "MOD13Q1", collection = cll, job="giraffe", extent = tile@extent,
begin = "2017325", end = "2017336", SDSstring = "110000000000", outProj = "asIn", >pixelSize="asIn") ######################## outProj = asIn pixelSize = asIn resamplingType = near Output Directory = C:/Users/.../Documents/MODIS_ARC/PROCESSED//giraffe ######################## Local structure is up-to-date. Using offline information! Error in lst_todo[[1]] : subscript out of bounds

[fdetsch]

Hi,

Sorry for the delay. The answer here is rather simple: there exists no file for the specified time period from "2017325" to "2017336", see here:

The file for "2017321" might include some values for this time range since we are dealing with 16-day composites here. However, the file search is performed via the actual names of the 16-day files, which is why runGdal() cannot find any match.

Nonetheless, I admit that the reported error is not very helpful. Therefore, as of a904a09184fc26032ed0d943a878056d74833125, a warning is thrown whenever no file could be found for a particular product and time range. In addition, runGdal() now returns a single NA for this product rather than throwing an error.

> tfs = runGdal(product = "MOD13Q1", collection = cll, job="giraffe", extent = tile@extent,
+         begin = "2017325", end = "2017336", SDSstring = "110000000000")
########################
outProj          =  asIn 
pixelSize        =  asIn 
resamplingType   =  near 
Output Directory =  C:/Users/.../MODIS_ARC/PROCESSED//giraffe 
########################
Local structure is up-to-date. Using offline information!
Warning message:
In runGdal(product = "MOD13Q1", collection = cll, job = "giraffe",  :
  No MOD13Q1 files found for the period from 2017-11-21 to 2017-12-02.

> tfs
$MOD13Q1.006
[1] NA

[t-stratmann]

Thanks for taking the time to look into this and fixing the error message!!! Much appreciated. So the key is to make sure to give dates that match with the 16 day composite dates? I think this used to not matter, since "2017325" to "2017336" fall within the 321 16-day composite, the files in the 321 folder would be downloaded.

[fdetsch]

Yeah, exactly. On second thought, maybe an automated reset of user-defined dates to the 16-day composite periods could indeed be of added value. I'll think about it and then come back to you!

[t-stratmann]

That could be really cool / easy! But a clear statement that one needs to match the MODIS scene dates would definitely be easier. Just extremely thankful your package exists!

[fdetsch]

Starting with 3cac4832279b7c285bba161fd5509cb2af01113f (runGdal()) and 1a928a693ec4e89530beca699633bcf4da71558b (getHdf()), 'begin' is now reset to the start date of the corresponding composite period (e.g. start of 16-day cycle for MOD13Q1). In addition, a warning is printed to the console:

> tfs = runGdal(product = "MOD13A1", collection = cll, job = "giraffe"
+               , extent = tile@extent, begin = "2017325", end = "2017336"
+               , SDSstring = "110000000000")
########################
outProj          =  asIn 
pixelSize        =  asIn 
resamplingType   =  near 
Output Directory =  C:/Users/florianD/Data/MODIS_ARC/PROCESSED/giraffe 
########################
Local structure is up-to-date. Using offline information!
Warning message:
In correctStartDate(tLimits$begin, sel, prodname) :
  Resetting 'begin' to start of corresponding MOD13A1 16-day composite period (2017-11-17).

Can you please install the 'develop' version via devtools::install_github("MatMatt/MODIS", ref = "develop") and verify?

[t-stratmann]

Installed. Tried the "giraffe" example and a different extent and date (also not set to the beginning of the MODIS scene) and it worked! Both times I got a warning, but it downloaded the MODIS scenes which make sense for the dates given:

Warning message: In correctStartDate(tLimits$begin, sel, prodname) : Resetting 'begin' to start of corresponding MOD13A1 16-day composite period (2017-11-17).

Warning message: In correctStartDate(tLimits$begin, sel, prodname) : Resetting 'begin' to start of corresponding MOD13A1 16-day composite period (2018-02-02).

Thank-you very much!

[fdetsch]

Great, then I consider this solved!

[fdetsch]

A small addition as of 9be254a1365ddcd567998eff53284834cad3d24c on branch 'sinusoidal' (heavy development going on lately...): These warning messages can now be disabled through MODISoptions(quiet = TRUE).