fdetsch / MODIS

Download and processing framework for MODIS imagery. The package provides automated access to the global online data archives LP DAAC, LAADS and NSIDC as well as processing capabilities such as file conversion, mosaicking, subsetting and time series.
Other
58 stars 27 forks source link

runMrt() returns nothing #137

Open korandocando opened 7 months ago

korandocando commented 7 months ago

Hello Florian and Matteo @MatMatt ! Here is Corrado. I've been having issues with the complete download of requested images through runGdal(): sometimes it downloads less images than those requested (both in "raw" and "processed" folders), probably due to my slow connection. I found a workaround by using getHdf(), which is able (I guess) to download only those .hdf files that are missing in the "raw" folder. After that, as far as I understand, 'runMtr()' is needed to process these .hdf files into geoTIFFs. Sadly, it returns empty characters (as many as the selected SDS layers), while printing something about "mrtmosaic" and "resample". Here is a reproducible code:

library(raster)
library(terra)
library(MODIS)

## setting modis options
lap <- file.path(tempdir(), "MODIS")
MODISoptions(
  localArcPath = lap, 
  outDirPath = file.path(lap, "PROCESSED"),
  dlmethod = "auto", MODISserverOrder = c("LPDAAC", "LAADS", "NSIDC"))
MODIS:::checkEarthdataLogin()     ## TRUE 
getCollection("MOD10A1F", forceCheck = TRUE) ##"061" ok

## setting the scene
limits <- c(5.9 , 6.2, 44.6 , 44.8)
start  <- "2003-01-04"
end    <- "2003-01-11"

## downloading HDF files
getHdf(product=c('MOD10A1F'), collection = "061",
       extent=extent(limits), 
       begin=start, end=end,
       outProj= "EPSG:4326")

## importing them
hdf_files <- list.files(path= file.path(lap, "MODIS/MOD10A1F.061"), pattern =".hdf$", full.names=T, recursive = T)

## from hdf to GeoTIFF
runMrt(product=c('MOD10A1F'), collection = "061",
       extent=extent(limits),
       begin=start, 
       end=end,
       job="loop", SDSstring = "11100", ## selecting only the first 3 layers
       datum="WGS84")

this last function returns:

Output projection: Sinusoidal 
Output pixel size: asIn 
Resampling method: nn 
Datum conversion: WGS84 
Local structure is up-to-date. Using offline information!
"mrtmosaic" is not recognized as an internal or external command, as an executable program, or as a batch file.
"resample" is not recognized as an internal or external command, as an executable program, or as a batch file.
 "mrtmosaic" is not recognized as an internal or external command, as an executable program, or as a batch file.
"resample" is not recognized as an internal or external command, as an executable program, or as a batch file.

It keeps printing these two lines for each .hdf file in the "raw" folder. Finally, it prints:

 $MOD10A1F.061
$MOD10A1F.061$`2003-01-04`
[1] "character(0)" "character(0)" "character(0)"

$MOD10A1F.061$`2003-01-05`
[1] "character(0)" "character(0)" "character(0)"

$MOD10A1F.061$`2003-01-06`
[1] "character(0)" "character(0)" "character(0)"

$MOD10A1F.061$`2003-01-07`
[1] "character(0)" "character(0)" "character(0)"

$MOD10A1F.061$`2003-01-08`
[1] "character(0)" "character(0)" "character(0)"

$MOD10A1F.061$`2003-01-09`
[1] "character(0)" "character(0)" "character(0)"

$MOD10A1F.061$`2003-01-10`
[1] "character(0)" "character(0)" "character(0)"

$MOD10A1F.061$`2003-01-11`
[1] "character(0)" "character(0)" "character(0)"

And nothing happens, nor in the environment, nor in the folders. I also tried setting different outProj ("GEO" and "UTM"), but nothing changes. Btw, I also tried to process these .hdf files using terra::rast() and terra::sds(), but I find no way to set the proper crs out from them.

A final thought... Would it be efficient to add this same "update" possibility that is implemented in runMrt() directly into runGdal()? So that - when requested - it can download only those dates that are missing from the local archive ("localArcPath" folder). If this feature is already implemented, I would suggest a printout to make the user aware of it (just as runMrt() that prints Local structure is up-to-date. Using offline information!).

Thanks a lot for providing this package, and for keeping working on it! Much much much appreciated!!! Cheers :)

Finally, here is my session info:

R version 4.3.1 (2023-06-16 ucrt)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 19045)

Matrix products: default

locale:
[1] LC_COLLATE=Italian_Italy.utf8  LC_CTYPE=Italian_Italy.utf8    LC_MONETARY=Italian_Italy.utf8 LC_NUMERIC=C                  
[5] LC_TIME=Italian_Italy.utf8    

time zone: Europe/Rome
tzcode source: internal

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
 [1] beepr_1.3            data.table_1.14.10   forcats_1.0.0        stringr_1.5.1        purrr_1.0.2          readr_2.1.5         
 [7] tidyr_1.3.1          tibble_3.2.1         ggplot2_3.4.4        tidyverse_2.0.0      dplyr_1.1.4          lubridate_1.9.3     
[13] exactextractr_0.10.0 sf_1.0-15            rts_1.1-14           xts_0.13.2           zoo_1.8-12           terra_1.7-65        
[19] MODIS_1.2.11.9000    mapdata_2.3.1        maps_3.4.2           units_0.8-5          raster_3.6-26        sp_2.1-2            

loaded via a namespace (and not attached):
 [1] tidyselect_1.2.0   bitops_1.0-7       fastmap_1.1.1      RCurl_1.98-1.14    promises_1.2.1     digest_0.6.34      timechange_0.3.0  
 [8] mime_0.12          lifecycle_1.0.4    ellipsis_0.3.2     ptw_1.9-16         magrittr_2.0.3     compiler_4.3.1     rlang_1.1.3       
[15] tools_4.3.1        utf8_1.2.4         corrplot_0.92      htmlwidgets_1.6.4  pkgbuild_1.4.3     classInt_0.4-10    curl_5.2.0        
[22] pkgload_1.3.4      KernSmooth_2.23-22 miniUI_0.1.1.1     foreign_0.8-86     withr_3.0.0        grid_4.3.1         fansi_1.0.6       
[29] urlchecker_1.0.1   profvis_0.3.8      mapedit_0.6.0      xtable_1.8-4       e1071_1.7-14       colorspace_2.1-0   scales_1.3.0      
[36] cli_3.6.2          generics_0.1.3     remotes_2.4.2.1    rstudioapi_0.15.0  RcppDE_0.1.7       tzdb_0.4.0         sessioninfo_1.2.2 
[43] DBI_1.2.1          cachem_1.0.8       proxy_0.4-27       audio_0.1-11       parallel_4.3.1     vctrs_0.6.5        devtools_2.4.5    
[50] hms_1.1.3          maptools_1.1-6     glue_1.7.0         codetools_0.2-19   stringi_1.8.3      gtable_0.3.4       later_1.3.2       
[57] munsell_0.5.0      pillar_1.9.0       htmltools_0.5.7    R6_2.5.1           shiny_1.8.0        lattice_0.22-5     memoise_2.0.1     
[64] httpuv_1.6.14      class_7.3-22       Rcpp_1.0.12        fs_1.6.3           usethis_2.2.2      pkgconfig_2.0.3