search

ropensci / MODIStsp

An "R" package for automatic download and preprocessing of MODIS Land Products Time Series
https://docs.ropensci.org/MODIStsp
GNU General Public License v3.0
155 stars 51 forks source link

Error - Processing and Reprojecting Terra NDVI files #192

Closed Olive-r closed 4 years ago

Olive-r commented 4 years ago

Hello,

Firtsly, thanks for developing MODIStsp. Until now, I had never had problems using MODIStsp. Recently, I encountered an error. Initially, I thought it was GDAL. But I installed GDAL and the error persists. I have also reinstalled MODIStsp several times.

Please see the error message below (Report).

Thank you in advance.

Oliver.

Report (Main error: Processing and Reprojecting Terra NDVI files)

MODIStsp() GDAL version in use: 3.0.2

[1] "C:\MODIS" [1] "C:\Users\olive\Desktop\MODIS" [Thu Jan 02 03:24:21 2020] MODIStsp --> Starting processing [Thu Jan 02 03:24:21 2020] Accessing http server at: https://e4ftl01.cr.usgs.gov/MOLT/MOD13Q1.006/ [Thu Jan 02 03:24:22 2020] Retrieving list of available Terra Files for Year 2019 [Thu Jan 02 03:24:24 2020] Downloading Terra Files for date: 2019_11_01 : 1 of: 1 [Thu Jan 02 03:24:44 2020] [Thu Jan 02 03:24:43 2020] 1 files for date: 2019.11.01 were successfully downloaded! [Thu Jan 02 03:24:54 2020] Processing and Reprojecting Terra NDVI files for date: 2019_11_01 Error in .local(.Object, ...) :

Además: Warning messages: 1: In system(cmd, intern = TRUE) : comando ejecutado '"C:\Program Files\GRASS GIS 7.8\extrabin\gdalbuildvrt.exe" -sd "1" -srcnodata "-3000" -vrtnodata "32767" "C:\Users\olive\AppData\Local\Temp\RtmpMH3hPW/mstp_temp\file4c98455b1bf3.vrt" "C:\Users\olive\Desktop\MODIS/MOD13Q1.A2019305.h17v05.006.2019323202058.hdf"' tiene estatus 1 2: In system(cmd, intern = TRUE) : comando ejecutado '"C:\Program Files (x86)\TerrSet\GDAL\bin\gdalwarp.exe" -multi -overwrite -nomd -s_srs "+proj=sinu +lon_0=0 +x_0=0 +y_0=0 +a=6371007.181 +b=6371007.181 +units=m +no_defs" -t_srs "+init=epsg:4326 +proj=longlat +datum=WGS84 +no_defs +ellps=WGS84 +towgs84=0,0,0" -ot "Int16" -r "near" -of "GTiff" -wo "INIT_DEST = NO_DATA" -wo "NUM_THREADS=6" -co "COMPRESS=None" "C:\Users\olive\AppData\Local\Temp\RtmpMH3hPW/mstp_temp\file4c98455b1bf3.vrt" "C:\Users\olive\AppData\Local\Temp\RtmpMH3hPW/mstp_temp\file4c98495119e.tif"' tiene estatus 1 Error in .rasterObjectFromFile(x, band = band, objecttype = "RasterLayer", : Cannot create a RasterLayer object from this file. (file does not exist)

lbusett commented 4 years ago

Hi. Its probably an issue related to the gdal 3 update. Could you please try installing the "devel" branch from gdal and see if it solves the problem?

install.packages('ropensci/MODIStsp', ref = 'devel')

Olive-r commented 4 years ago

Hi there !

Thank you for the quick response. I also think the same about GDAL: it is the origin of this issue.

I have tried to install MODIStsp as you say, and now run! However, now, the download is much slower. Anyway, its run!

Thanks for all your hard work and your dedication developing Modistsp.

King regards.

Oliver.

Report.

MODIStsp() MODIStsp would like to save a "MODIStsp_Previous.json" file containing information about its last successfull run in the folder .../your-r-library/MODIStsp/ExtData/.

Do you authorize this?

1: Yes - "MODIStsp_Previous.json" will be saved permanently and updated after after each successfull run of the tool. You will not see this message anymore.

2: No - Previous options will be written to tempdir and will be lost when you exit R. You will see this message at each MODIStsp execution.

Choice (1/2): 2 GDAL version in use: 3.0.2 [Thu Jan 02 21:18:52 2020] MODIStsp --> Starting processing [Thu Jan 02 21:18:52 2020] Accessing http server at: https://e4ftl01.cr.usgs.gov/MOLT/MOD13Q1.006/ [Thu Jan 02 21:18:53 2020] Retrieving list of available Terra Files for Year 2019 [Thu Jan 02 21:18:56 2020] Downloading Terra Files for date: 2019_12_03 : 1 of: 1 [Thu Jan 02 21:26:10 2020] [Thu Jan 02 21:26:10 2020] 1 files for date: 2019.12.03 were successfully downloaded! [Thu Jan 02 21:26:21 2020] Processing and Reprojecting Terra NDVI files for date: 2019_12_03 [Thu Jan 02 21:26:51 2020] Processing and Reprojecting Terra EVI files for date: 2019_12_03 [Thu Jan 02 21:27:21 2020] Processing and Reprojecting Terra b1_Red files for date: 2019_12_03 [Thu Jan 02 21:27:50 2020] Processing and Reprojecting Terra b2_NIR files for date: 2019_12_03 [Thu Jan 02 21:28:08 2020] Computing Terra SAVI for date: 2019_12_03 [Thu Jan 02 21:28:13 2020] Creating Virtual Files and R time series for layer NDVI [Thu Jan 02 21:28:13 2020] Creating Virtual Files and R time series for layer EVI [Thu Jan 02 21:28:13 2020] Creating Virtual Files and R time series for layer SAVI [Thu Jan 02 21:28:13 2020] Total Processing Time: 9.35278362035751 [Thu Jan 02 21:28:13 2020] MODIStsp processed files are in: C:\MODIS [Thu Jan 02 21:28:13 2020] Original downloaded MODIS HDF files are in: C:\MODIS Warning messages: 1: In system(cmd, intern = TRUE) : comando ejecutado '"C:\Program Files\GRASS GIS 7.8\extrabin\gdalbuildvrt.exe" -sd "1" -srcnodata "-3000" -vrtnodata "32767" "C:\Users\olive\AppData\Local\Temp\RtmpIR6yuC/mstp_temp\file283412ea3837.vrt" "C:\MODIS/MOD13Q1.A2019337.h17v05.006.2019354133839.hdf"' tiene estatus 1 2: In system(cmd, intern = TRUE) : comando ejecutado '"C:\Program Files\GRASS GIS 7.8\extrabin\gdalbuildvrt.exe" -sd "2" -srcnodata "-3000" -vrtnodata "32767" "C:\Users\olive\AppData\Local\Temp\RtmpIR6yuC/mstp_temp\file28342dce6085.vrt" "C:\MODIS/MOD13Q1.A2019337.h17v05.006.2019354133839.hdf"' tiene estatus 1 3: In system(cmd, intern = TRUE) : comando ejecutado '"C:\Program Files\GRASS GIS 7.8\extrabin\gdalbuildvrt.exe" -sd "4" -srcnodata "-1000" -vrtnodata "32767" "C:\Users\olive\AppData\Local\Temp\RtmpIR6yuC/mstp_temp\file2834157e3536.vrt" "C:\MODIS/MOD13Q1.A2019337.h17v05.006.2019354133839.hdf"' tiene estatus 1 4: In system(cmd, intern = TRUE) : comando ejecutado '"C:\Program Files\GRASS GIS 7.8\extrabin\gdalbuildvrt.exe" -sd "5" -srcnodata "-1000" -vrtnodata "32767" "C:\Users\olive\AppData\Local\Temp\RtmpIR6yuC/mstp_temp\file283446a15b2a.vrt" "C:\MODIS/MOD13Q1.A2019337.h17v05.006.2019354133839.hdf"' tiene estatus 1

lbusett commented 4 years ago

Hi @Olive-r

Download speed should not be affected. Maybe it was just a temporary slow down.

I however see some warnings in your log that should not probably happen. Could you please save your processing options as a json file from the gui and cut and paste its contents here?

Olive-r commented 4 years ago

Hello Lorenzo,

In relation to downloading, it is true that now is much slower, but may be as you say, probably it is caused by the server. Certainly, I do not know.

Here is the content of json file (https://www.dropbox.com/s/t23bv0iqer5c04f/MODIStsp_2020-01-03.json?dl=0):

{ "sel_prod": "Vegetation Indexes_16Days_250m (M*D13Q1)", "sensor": "Terra", "prod_version": "6", "start_date": "2019-12-01", "end_date": "2019-12-31", "bandsel": [1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], "indexes_bandsel": [0, 0, 0, 1], "quality_bandsel": [0, 0, 0, 0, 0, 0, 0, 0, 0], "start_x": 17, "end_x": 17, "start_y": 5, "end_y": 5, "user": "Oliver82", "password": "Lamaroma82", "use_aria": false, "download_server": "http", "download_range": "full", "proj": "User Defined", "output_proj4": "+init=epsg:4326 +proj=longlat +datum=WGS84 +no_defs +ellps=WGS84 +towgs84=0,0,0", "out_res_sel": "Native", "out_res": "Native", "full_ext": "Select MODIS Tiles", "resampling": "near", "out_format": "GTiff", "ts_format": [], "compress": "None", "nodata_change": "Yes", "scale_val": "Yes", "delete_hdf": "Yes", "reprocess": "Yes", "bbox": [null, null, null, null], "out_folder": "C:\MODIS", "out_folder_mod": "C:\MODIS", "MODIStspVersion": "1.3.9", "custom_indexes": [] }

And here is the R session content:

R version 3.6.2 (2019-12-12) -- "Dark and Stormy Night" Copyright (C) 2019 The R Foundation for Statistical Computing Platform: x86_64-w64-mingw32/x64 (64-bit)

R es un software libre y viene sin GARANTIA ALGUNA. Usted puede redistribuirlo bajo ciertas circunstancias. Escriba 'license()' o 'licence()' para detalles de distribucion.

R es un proyecto colaborativo con muchos contribuyentes. Escriba 'contributors()' para obtener más información y 'citation()' para saber cómo citar R o paquetes de R en publicaciones.

Escriba 'demo()' para demostraciones, 'help()' para el sistema on-line de ayuda, o 'help.start()' para abrir el sistema de ayuda HTML con su navegador. Escriba 'q()' para salir de R.

library(MODIStsp) Registered S3 method overwritten by 'xts': method from as.zoo.xts zoo MODIStsp() MODIStsp would like to save a "MODIStsp_Previous.json" file containing information about its last successfull run in the folder .../your-r-library/MODIStsp/ExtData/.

Do you authorize this?

1: Yes - "MODIStsp_Previous.json" will be saved permanently and updated after after each successfull run of the tool. You will not see this message anymore.

2: No - Previous options will be written to tempdir and will be lost when you exit R. You will see this message at each MODIStsp execution.

Choice (1/2): 2 Loading required package: gWidgetsRGtk2 Loading required package: RGtk2 Loading required package: gWidgets Loading required package: cairoDevice [Fri Jan 03 01:20:44 2020] Welcome to MODIStsp!We will now search for a valid GDAL installation - pleasewait! (this will happen only once) GDAL version in use: 3.0.2 [1] "C:\MODIS" [1] "C:\MODIS" [1] "C:\MODIS\Modis" TRUE [Fri Jan 03 01:24:15 2020] MODIStsp --> Starting processing [Fri Jan 03 01:24:15 2020] Accessing http server at: https://e4ftl01.cr.usgs.gov/MOLT/MOD13Q1.006/ [Fri Jan 03 01:24:17 2020] Retrieving list of available Terra Files for Year 2019 [Fri Jan 03 01:24:22 2020] Downloading Terra Files for date: 2019_12_03 : 1 of: 1 [Fri Jan 03 01:29:34 2020] [Fri Jan 03 01:29:34 2020] 1 files for date: 2019.12.03 were successfully downloaded! [Fri Jan 03 01:29:45 2020] Processing and Reprojecting Terra NDVI files for date: 2019_12_03 [Fri Jan 03 01:30:15 2020] Processing and Reprojecting Terra EVI files for date: 2019_12_03 [Fri Jan 03 01:30:44 2020] Processing and Reprojecting Terra b1_Red files for date: 2019_12_03 [Fri Jan 03 01:31:13 2020] Processing and Reprojecting Terra b2_NIR files for date: 2019_12_03 [Fri Jan 03 01:31:30 2020] Computing Terra SAVI for date: 2019_12_03 [Fri Jan 03 01:31:35 2020] Creating Virtual Files and R time series for layer NDVI [Fri Jan 03 01:31:35 2020] Creating Virtual Files and R time series for layer EVI [Fri Jan 03 01:31:35 2020] Creating Virtual Files and R time series for layer SAVI [Fri Jan 03 01:31:35 2020] Total Processing Time: 7.32985810041428 [Fri Jan 03 01:31:35 2020] MODIStsp processed files are in: C:\MODIS [Fri Jan 03 01:31:35 2020] Original downloaded MODIS HDF files are in: C:\MODIS Warning messages: 1: In system(cmd, intern = TRUE) : comando ejecutado '"C:\Program Files\GRASS GIS 7.8\extrabin\gdalbuildvrt.exe" -sd "1" -srcnodata "-3000" -vrtnodata "32767" "C:\Users\olive\AppData\Local\Temp\Rtmp8m6UBp/mstp_temp\file496c7c224ce9.vrt" "C:\MODIS/MOD13Q1.A2019337.h17v05.006.2019354133839.hdf"' tiene estatus 1 2: In system(cmd, intern = TRUE) : comando ejecutado '"C:\Program Files\GRASS GIS 7.8\extrabin\gdalbuildvrt.exe" -sd "2" -srcnodata "-3000" -vrtnodata "32767" "C:\Users\olive\AppData\Local\Temp\Rtmp8m6UBp/mstp_temp\file496ce3a79b6.vrt" "C:\MODIS/MOD13Q1.A2019337.h17v05.006.2019354133839.hdf"' tiene estatus 1 3: In system(cmd, intern = TRUE) : comando ejecutado '"C:\Program Files\GRASS GIS 7.8\extrabin\gdalbuildvrt.exe" -sd "4" -srcnodata "-1000" -vrtnodata "32767" "C:\Users\olive\AppData\Local\Temp\Rtmp8m6UBp/mstp_temp\file496c2e425635.vrt" "C:\MODIS/MOD13Q1.A2019337.h17v05.006.2019354133839.hdf"' tiene estatus 1 4: In system(cmd, intern = TRUE) : comando ejecutado '"C:\Program Files\GRASS GIS 7.8\extrabin\gdalbuildvrt.exe" -sd "5" -srcnodata "-1000" -vrtnodata "32767" "C:\Users\olive\AppData\Local\Temp\Rtmp8m6UBp/mstp_temp\file496c59b943c1.vrt" "C:\MODIS/MOD13Q1.A2019337.h17v05.006.2019354133839.hdf"' tiene estatus 1

Thanks for all.

Oliver.