cloud masking/ estimation of shadows

[Romaysa]

Sir, hope you do well. the error i get while intractively selecting the pixels where cloud exist.

shadow<- cloudShadowMask(lsatt, cldmsk_final, nc=2) Please select cloud and corresponding shadow pixels (alternating cloud,shadow,cloud,shadow...)

Error in cut.default(me, breaks = mbreax, labels = FALSE, include.lowest = TRUE) : invalid number of intervals In addition: Warning message: In .local(x, size, ...) : size set to the number of cells within "ext": 47591017

[bleutner]

I can't reproduce this error. Please create a reproducible example for me. To get you started: check whether the example in ?cloudMask runs in your setup.

[Romaysa]

Dear Bleutner, honestly i did not get your massage. if you could write me again please. the error is about cut. default and the end is like size set the number of cells within the ext 4759... some thing like that. and if you want i will try to send you the whole Landsat 8 image.

[bleutner]

I need a self contained minimal example so I can reproduce the error on my computer. Please read How to make a great R reproducible example? Instead of including the full Landsat data it is often enough to crop it to a smaller extent before sharing.

Also please report your R version, Operating System and RStoolbox version, e.g. by posting the output of devtools::session_info()

[Romaysa]

Dear Bleutner, I hope this works. #Cloud masking using RStoolbox, ggplot2, raster, packages. I am working on a Landsat 8 image; the aim of this practice is to remove the clouds from this image. ## Here is the MTL file.

mtl<- system.file("MTL.txt") readMeta("MTL.txt") meta <-readMeta("MTL.txt")

R can read the file perfectly. Here is the data

dput(lsatt) new("RasterStack" , filename = "" , layers = list(<S4 object of class structure("RasterLayer", package = "raster")>, <S4 object of class structure("RasterLayer", package = "raster")>, <S4 object of class structure("RasterLayer", package = "raster")>, <S4 object of class structure("RasterLayer", package = "raster")>, <S4 object of class structure("RasterLayer", package = "raster")>, <S4 object of class structure("RasterLayer", package = "raster")>, <S4 object of class structure("RasterLayer", package = "raster")>, <S4 object of class structure("RasterLayer", package = "raster")>, <S4 object of class structure("RasterLayer", package = "raster")>, <S4 object of class structure("RasterLayer", package = "raster")>) , title = character(0) , extent = new("Extent" , xmin = 549585 , xmax = 777315 , ymin = 3396285 , ymax = 3628815 ) , rotated = FALSE , rotation = new(".Rotation" , geotrans = numeric(0) , transfun = function () NULL ) , ncols = 7591L , nrows = 7751L , crs = new("CRS" , projargs = "+proj=utm +zone=41 +datum=WGS84 +units=m +nodefs +ellps=WGS84 +towgs84=0,0,0" ) , history = list() , z = list() ) Define threshold: cloudMask(lsatt, threshold = 0.8, blue= "B2_dn", tir = "B10dn", buffer = NULL, plot= FALSE)

In my example the band for blue is B2 and band for tir is B10. Landsat 8 bands.

cldmsk<- cloudMask(lsatt, blue = 2, tir = 10) ggR(cldmsk, 2, geom_raster = TRUE)

To plot it as ggRGB we have to load the ggplot2 package, which looks fine.

cldmsk_final<- cloudMask(cldmsk, threshold = 0.1, buffer = 5)

ggRGB(lsatt, stretch = "lin") + ggR(cldmsk_final[[1]], ggLayer = TRUE, forceCat = TRUE, geom_raster = TRUE)+ scale_fill_manual(values = "red", na.value = NA)

Here is the warning massage I get after running this code.

Warning message: Removed 484084 rows containing missing values (geom_raster).

Now to estimate the shadow I am getting the error like below.

shadow<- cloudShadowMask(lsatt, cldmsk_final, shiftEstimate = c(-16, -6))

Error

Error in cut.default(me, breaks = mbreax, labels = FALSE, include.lowest = TRUE) : invalid number of intervals In addition: Warning message: In .local(x, size, ...) : size set to the number of cells within "ext": 47591017

i am sure its not a true massage which you want hope it works well even if i make a subset of the image it will be still so big and i can't attach the data. thanks for your time. Regards

[Romaysa]

i forgot to write about the version of the R. R version 3.3.3 (2017-03-06) --

Platform: x86_64-w64-mingw32/x64 (64-bit) Regards

[Romaysa]

Sir, hope things are going well, any progress about the error?

Regards

[bleutner]

This is still not a self contained example. The easier you make it for me to reproduce your error, the quicker I can solve it.

Please give me the Landsat scene ID you are trying to process, then I'll download the dataset myself.

[Romaysa]

Sir, thanks a lot for help. here is the image ID if you could download it.

REQUEST_ID = "0501504161126_00035"
LANDSAT_SCENE_ID = "LC81550382015106LGN00"
FILE_DATE = 2015-04-16T10:04:43Z
STATION_ID = "LGN"
PROCESSING_SOFTWARE_VERSION = "LPGS_2.4.0"

GROUP = PRODUCT_METADATA DATA_TYPE = "L1T" ELEVATION_SOURCE = "GLS2000" OUTPUT_FORMAT = "GEOTIFF" SPACECRAFT_ID = "LANDSAT_8" SENSOR_ID = "OLI_TIRS" WRS_PATH = 155 WRS_ROW = 38 sir, i am not changing or writing another code to do a could mask and shadow masking just using the code you provide but its giving the same error. erro. in cut###

MTL<- system.file("MTL.txt") meta<- readMeta("MTL.txt") lsaatt<- stackMeta("MTL.txt") lsaatt_sref<- radCor(lsaatt, metaData = meta, method = "apref") cloudMask(lsaatt, threshold = 0.8, blue = "B2_dn", tir = "B10_dn", buffer= NULL, plot = F) ggRGB(lsaatt, stretch = "lin") cldmsk<- cloudMask(lsaatt, blue = 2, tir = 10) ggR(cldmsk, 2, geom_raster = T)

cldmsk_final<- cloudMask(cldmsk, threshold = 0.1, buffer = 5) ggRGB(lsaatt, stretch = "lin")+ ggR(cldmsk_final[[1]], ggLayer = T, forceCat = T, geom_raster = T)+scale_fill_manual(values= "red", na.value=NA) shadow<- cloudShadowMask(lsaatt, cldmsk_final, shiftEstimate = c(-16,6))

Warning message: Removed 484084 rows containing missing values (geom_raster).

#######and here is the error it self.

Error in cut.default(me, breaks = mbreax, labels = FALSE, include.lowest = TRUE) : invalid number of intervalsIn addition: Warning message:In .local(x, size, ...) : size set to the number of cells within "ext": 47591017

On Wed, Apr 26, 2017 at 6:51 PM, Benjamin Leutner notifications@github.com wrote:

This is still not a self contained example. The easier you make it for me to reproduce your error, the quicker I can solve it.

Please give me the Landsat scene ID you are trying to process, then I'll download the dataset myself.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/bleutner/RStoolbox/issues/21#issuecomment-297424028, or mute the thread https://github.com/notifications/unsubscribe-auth/AZd3FS8eKSfbEKtuJjapwEhuNKruqc8Iks5rz1L_gaJpZM4M7EnE .

--

Ahmad Shah "Frahmand"

Ph.D Candidate

Geomatics Engineering Department, Yildiz Technical University, Istanbul, Turkey

ahmad.frahmand@gmail.com ahmad.frahmand@gmail.com

University official email: f5013021@std.yildiz.edu.tr

Skype: ah_frahmand_india

Mobile phone: 00905469743785

[bleutner]

The issue was that the quantile argument which is used internally to create a "potential shadow map" was too low. I'll add a check to warn when this happens in the future.

In your case the quantile calculation was off, because there were still many background pixels which were taken as valid pixels.

Thus the easiest fix is to set these pixels to NA, then the default quantile threshold of 0.2 works just fine. Otherwise you'd have to increase the value of the quantile argument.

library(RStoolbox)

meta <- readMeta("LC81550382015106LGN00_MTL.txt")
lsat <- stackMeta(meta)

# Set all pixels to NA, where the thermal band is 0
lsat[lsat[[10]]==0] <- NA

# calculate cloudmask 
cldmsk_final <- cloudMask(lsat, blue = 2, tir = 10, threshold = 0, buffer = 5)

# calculate shadow shift interactively
shadow <- cloudShadowMask(lsat, cldmsk_final,nc = 10, quantile = .2, returnShift = TRUE)

# Note that your shift estimate was too low also. It is in map units, i.e. in meters, not in pixels
shadow$shift
#>     x     y 
#> -2151  1254 

ggRGB(lsat, stretch="lin")+
        ggR(shadow$shadowMap, sat=1, ggLayer = TRUE) + 
        ggR(cldmsk_final,sat=1, hue=.5, ggLayer=TRUE)