search

chrisvwn / Rnightlights

R package to extract data from satellite nightlights.
GNU General Public License v3.0
47 stars 14 forks source link

Stuck at gdalwarp masking to VRT #52

Closed nreguera closed 4 years ago

nreguera commented 4 years ago

Hi @chrisvwn

I have started downloading the data from another machine to speed up the process, and after downloading the first period, it got stuck in the step of masking to VRT and it took several minutes to finish as you can see in this screenshot:

Untitled

I remember that this step (masking to VRT) is very fast in the other machine I am working, it takes a few seconds, so I am wondering why this can happen.

The code I used is this:

library(Rnightlights)`

pkgOptions(downloadMethod = "aria", cropMaskMethod = "gdal", extractMethod = "gdal", deleteTiles = TRUE)
pkgOptions(extractMethod = "rast", numCores=4)`

getCtryNlData(
  ctryCode = "THA", # Country
  admLevel = "admin3", # Geograhical level
  nlType = "VIIRS.M", # Periodicity
  nlPeriods = nlRange("201801", "201812"), # timeframe
  ignoreMissing = FALSE, # ignoring missing values
  nlStats = list("sum", na.rm=TRUE)) # aggregation operation for the radiance values

Also, when it finishes the process, it gives me this error, and no data file is generated:

Untitled2

The tiles are still in the directory after this error.

nreguera commented 4 years ago

Additional info:

nreguera commented 4 years ago

More, I got this:

Untitled
nreguera commented 4 years ago

I found out that the problem are the gdal pkgOptions I took from the example in your code repo. If I don't use them works fine. No idea why.

chrisvwn commented 4 years ago

Do you mean if you do not set pkgOptions(cropMaskMethod='gdal', extractMethod='gdal')? This means that it is not using gdal at all. This is not a bad thing as this is the default i.e. pkgOptions(cropMaskMethod='rast', extractMethod='rast'), Gdal is used as an alternative to the default mode using raster. I suggest using only setting downloadMethod and maybe numThreads which is:

pkgOptions(downloadMethod='aria', numThreads=4)

If you still want to try the gdal option:

I suspect that the problem you are facing with gdal is related to having multiple gdal installations one of which is not working. Could you try uninstalling gdal and then reinstalling it?

nreguera commented 4 years ago

Probably is not installed properly as you said. Anyway I tried with the raster option and it works fine, so I think is better to just let it go this time. Thanks.