r-spatial / dtwSat

Time-Weighted Dynamic Time Warping for satellite image time series analysis
https://www.victor-maus.com/dtwSat/
GNU General Public License v3.0
131 stars 39 forks source link

cluster error #19

Closed JieBNU closed 6 years ago

JieBNU commented 6 years ago

@vwmaus Hi

I tried the dtwSat in Windows Server (64 cores,256GB memory), and used the function twdtwApplyParallel. _beginCluster(n=27) r_twdtw1 <- twdtwApplyParallel(x = rts1, y = temporal_patterns1, weight.fun = logfun1, progress = 'text') endCluster()

When I set the parameter number of nodes (n) to 27, everything is fine. But, when I set n to 28, it returns the error that "Error in twdtwApplyParallel.twdtwRaster(x, y, weight.fun, dist.method, : cluster error"

I am wondering how to release hold of the number of nodes(n)?

vwmaus commented 6 years ago

@JieBNU I will try to reproduce the error here. Can you send me the output of str(rts1) and str(temporal_patterns1)?

JieBNU commented 6 years ago

@vwmaus Sure, I used the code and data that you provided in the dtwSat demo. The code, output of str(rts1) and str(temporal_patterns1) are in the attach files. str(rts1).txt

str(temporal_patterns).txt

demo_dtwSat1.txt

vwmaus commented 6 years ago

@JieBNU the parallel processing splits the raster into chunks by rows. As the data in rts1 has only 27 rows it cannot be divided into 28 processing threads. Please, use the number of rows in the raster as the maximum number of cores beginCluster(n = dim(rts)[2]). Could you also try to set the number of rows beginCluster(n = dim(rts)[2]) in the Linux system #18?

JieBNU commented 6 years ago

@vwmaus Thanks, I used other data and it worked using 60 cores on Windows system.

vwmaus commented 6 years ago

what about the GNU Linux system?