search

ConsBiol-unibern / SDMtune

Performs Variables selection and model tuning for Species Distribution Models (SDMs). It provides also several utilities to display results.
https://consbiol-unibern.github.io/SDMtune/
Other
25 stars 8 forks source link

[BUG] cluster error during predict with large raster stack #7

Closed ghost closed 4 years ago

ghost commented 4 years ago

Describe the bug Predict fails when using parallel = TRUE during a SDMtune::predict. Error message is:
in raster::clusterR(data, predict, args = list(model = model, clamp = clamp, : cluster error

Raster stack being used is large, 14 layers each 9gb each. Doesn't appear to be a memory issue when observing windows task manager - plenty of memory left.

If I turn parallel = FALSE, new error occurs:
in p[-naind, ] <- predv : "number of items to replace is not a multiple of replacement length".

Reducing number of layers to less than 9 seems to fix it.

sgvignali commented 4 years ago

Which version are you using?

Can you please post sessionInfo()

ghost commented 4 years ago

Thanks, version 1.1.0

sgvignali commented 4 years ago

In the next release I deprecated the use of parallel. This is because it improves only for very large datasets. The function used the cluster implemented by the package raster that uses the superseded package snow. Despite that, your second error is strange, but I need more to figure it out, I should be able to reproduce it. Maybe one of the layers has a problem, you could try to start with the 9 working layers and add the remaining one by one, to see which one causes the problem

ghost commented 4 years ago

Thanks, i'll check my rasters.

sgvignali commented 4 years ago

@mrlewie could you check your raster layers? If is not an issue related to the pkg we can close it!