Open AaronNCSU opened 3 years ago
MathieuChailloux
commented
3 years ago Hello,
I have never seen this error before but let's try. Can you launch directly r.cost (GRASS algorithm in processing toolbox) on file _LandUse_ForestDisp_startstart.tif with same parameters ? What is your CRS ? You can send me your data and BioDispersal project file if you want me to look deeper.
Regards, Mathieu
AaronNCSU
commented
3 years ago Mathieu,
After reinstalling GIS I get the same error when running the r.cost tool. Additionally, the following forum suggests I may need to use a UTM CRS, is this true, or should BioDispersal work on this NAD83 projection? My CRS is EPSG:6543 - NAD83(2011)/North Carolina (ftUS). My BioDispersal project file is attached.
Thanks, Aaron BioDispersalPrj2.zip
Algorithm 'r.cost' starting… Input parameters: { '-k' : False, '-n' : True, 'GRASS_MIN_AREA_PARAMETER' : 0.0001, 'GRASS_RASTER_FORMAT_META' : '', 'GRASS_RASTER_FORMAT_OPT' : '', 'GRASS_REGION_CELLSIZE_PARAMETER' : 0, 'GRASS_REGION_PARAMETER' : None, 'GRASS_SNAP_TOLERANCE_PARAMETER' : -1, 'input' : 'C:/Users/aschw/Documents/School/2020Fall/ES400/ProjectWork/Layers/BioDispersal/Subnetworks/LandUse/LandUse_friction.tif', 'max_cost' : 5000, 'memory' : None, 'nearest' : 'TEMPORARY_OUTPUT', 'null_cost' : None, 'outdir' : 'TEMPORARY_OUTPUT', 'output' : 'TEMPORARY_OUTPUT', 'start_coordinates' : None, 'start_points' : None, 'start_raster' : 'C:/Users/aschw/Documents/School/2020Fall/ES400/ProjectWork/Layers/BioDispersal/Subnetworks/LandUse/LandUse_ForestDisp_start.tif', 'stop_coordinates' : None, 'stop_points' : None }
g.proj -c proj4="+proj=lcc +lat_0=33.75 +lon_0=-79 +lat_1=36.1666666666667 +lat_2=34.3333333333333 +x_0=609601.219202438 +y_0=0 +ellps=GRS80 +units=us-ft +no_defs"
r.external input="C:\Users\aschw\Documents\School\2020Fall\ES400\ProjectWork\Layers\BioDispersal\Subnetworks\LandUse\LandUse_friction.tif" band=1 output="rast_5f845723a64ef2" --overwrite -o
r.external input="C:\Users\aschw\Documents\School\2020Fall\ES400\ProjectWork\Layers\BioDispersal\Subnetworks\LandUse\LandUse_ForestDisp_start.tif" band=1 output="rast_5f845723a68d73" --overwrite -o
g.region n=2246083.9808584764 s=-1167021.87599631 e=3575493.583517518 w=-50754.20139412093 res=30.0
r.cost input=rast_5f845723a64ef2 -n start_raster=rast_5f845723a68d73 max_cost=5000 memory=300 output=output7fe04ea0e59b4cd6b2a67e6c8a42946a nearest=nearest7fe04ea0e59b4cd6b2a67e6c8a42946a outdir=outdir7fe04ea0e59b4cd6b2a67e6c8a42946a --overwrite
g.region raster=output7fe04ea0e59b4cd6b2a67e6c8a42946a
r.out.gdal -t -m input="output7fe04ea0e59b4cd6b2a67e6c8a42946a" output="C:\Users\aschw\AppData\Local\Temp\processing_1c3e2079c4aa45c1b81a44d7b14bd98f\f9fd2f6beaf94d1cbe341fef0ff27286\output.tif" format="GTiff" createopt="TFW=YES,COMPRESS=LZW" --overwrite
g.region raster=nearest7fe04ea0e59b4cd6b2a67e6c8a42946a
r.out.gdal -t -m input="nearest7fe04ea0e59b4cd6b2a67e6c8a42946a" output="C:\Users\aschw\AppData\Local\Temp\processing_1c3e2079c4aa45c1b81a44d7b14bd98f\0f4dc0a77b0f402387aa6a4d407177f0\nearest.tif" format="GTiff" createopt="TFW=YES,COMPRESS=LZW" --overwrite
g.region raster=outdir7fe04ea0e59b4cd6b2a67e6c8a42946a
r.out.gdal -t -m input="outdir7fe04ea0e59b4cd6b2a67e6c8a42946a" output="C:\Users\aschw\AppData\Local\Temp\processing_1c3e2079c4aa45c1b81a44d7b14bd98f\9e6da9a0d4fc4b228f195429952e0d21\outdir.tif" format="GTiff" createopt="TFW=YES,COMPRESS=LZW" --overwrite
Starting GRASS GIS...
WARNING: Concurrent mapset locking is not supported on Windows
Cleaning up temporary files...
Executing <C:\Users\aschw\AppData\Local\Temp\processing_1c3e2079c4aa45c1b81a44d7b14bd98f\grassdata\grass_batch_job.cmd> ...
C:\Users\aschw\Documents>chcp 1252 1>NUL
C:\Users\aschw\Documents>g.proj -c proj4="+proj=lcc +lat_0=33.75 +lon_0=-79 +lat_1=36.1666666666667 +lat_2=34.3333333333333 +x_0=609601.219202438 +y_0=0 +ellps=GRS80 +units=us-ft +no_defs"
WARNING: Datum g.region -d should be run in each to update the region from the default
Projection information updated
C:\Users\aschw\Documents>r.external input="C:\Users\aschw\Documents\School\2020Fall\ES400\ProjectWork\Layers\BioDispersal\Subnetworks\LandUse\LandUse_friction.tif" band=1 output="rast_5f845723a64ef2" --overwrite -o
Over-riding projection check
Reading band 1 of 1...
Link to raster map
Loading resulting layers The following layers were not correctly generated.
MathieuChailloux
commented
3 years ago Hello, CRS seems OK with my data. Could you send me input layers of last step ? (LandUse_ForestDisp_start.tif and LandUse_friction.tif) It seems to be linked to ouptut size and available space indeed but if you have 80G available this is strange. Is this a large area ?
AaronNCSU
commented
3 years ago Mathieu, Input layers are attached in the zipped folder, I only had one Subnetwork because my land use data was already into a classified raster format which BioDispersal was using, I believe I went about it right in terms of simply running through the prelim steps and assigning the friction classes to my existing layer classes.
The input layers are about 11G each, so I was assuming the output raster should be the same size. For reference I have tried this at both a cumulative cost of 2500 and 5000 and neither work, but I would like to see the difference in what the 2 of them yield. The study area is the extent of North Carolina in the United States, this is an area of 53,819 sq mi (139,390 km2). Since the problem seems to be related to memory, I will try moving some things to secondary hard drive and see if I can get it to write.
Another question for you; does this last step, Dispersal, essentially just run r.cost and have styling for the layer built in? If so, I have a friend who has an ESRI software license and may be able to use a similar tool there. Though I would still like to figure out how to run this on my own. Files too large for GitHub to attach, hopefully you can use a google drive link: [Here](https://drive.google.com/file/d/197va61ry9lii0dpxg4UCS6egNCP7EHau/view?usp=sharing, https://drive.google.com/file/d/1FVCg4X3sL5FOdrQsgf7t0FkJXoRB-05G/view?usp=sharing)
MathieuChailloux
commented
3 years ago Thanks for the data, I will run my tests.
Last step mainly calls r.cost (some preprocessing to warp layer with same extent/crs, and some postprocessing to remove NoData and apply a singleband pseudocolor style) so you could try with ArcGIS (hard for me to say this as the initial goal was to provide a free and open alternative :p).
AaronNCSU
commented
3 years ago Yes I agree, I love open source software because of this, and developers like you can often bring the software past what paid products can do. This tool was massively helpful in making my analysis easier and bringing an understanding of the workflow for animal dispersal estimation. It seems the problem is related to r.cost though, rather than the plugin, as the same friend with the ESRI suite could not run r.cost (in QGIS) on her machine either.
MathieuChailloux
commented
3 years ago Indeed your files are quite large, too much for my personal computer (currently teleworking). This might be caused by r.cost intermediate layers but nothing sure. You could try with a higher resolution, or on smaller areas (maybe you can divide your study area), or with some super computer :-D
AaronNCSU
commented
3 years ago I'll try cutting down to a smaller area and get back to you if I have any more questions. Thanks for the help so far!
Hello,
I am trying to run the last step of the BioDispersal plugin and have been getting this error. I do not fully understand what mapset locking is but can see that not being able to do it is preventing the plugin from generating a certain raster. This also says my drive is out of memory, but I have about 80 Gb of space remaining, so I do not see how that could be the case.
[2020-10-11 18:09:36.765973] [info] **** BEGIN : Computing dispersal [2020-10-11 18:09:37.376771] [info] Calling processing algorithm 'gdal:rasterize' [2020-10-11 18:09:37.423634] [info] GDAL command: [2020-10-11 18:09:37.423634] [info] GDAL command output: [2020-10-11 18:09:45.521184] [info] Results: {'OUTPUT': 'C:\Users\aschw\Documents\School\2020Fall\ES400\ProjectWork\Layers\BioDispersal\Subnetworks\LandUse\LandUse_ForestDisp_start_start.tif'} [2020-10-11 18:09:45.536808] [info] Call to rasterize successful, performed in 8.160037517547607 seconds [2020-10-11 18:09:45.536808] [info] Calling processing algorithm 'grass7:r.cost' [2020-10-11 18:09:46.465140] [warn] WARNING: Concurrent mapset locking is not supported on Windows [2020-10-11 18:09:46.621402] [warn] WARNING: Datum not recognised by GRASS and no parameters found
[2020-10-11 18:11:22.808937] [warn] WARNING: segment zero_fill(): Unable to write (No space left on device)
[2020-10-11 18:11:22.808937] [warn] WARNING: Could not write segment file
[2020-10-11 18:11:22.808937] [warn] ERROR: Can not create temporary file
[2020-10-11 18:11:22.855815] [warn] ERROR: Raster map not found
[2020-10-11 18:11:22.933946] [warn] ERROR: Raster map or group not found
[2020-10-11 18:11:23.121460] [info] Results: {'output': 'C:\Users\aschw\Documents\School\2020Fall\ES400\ProjectWork\Layers\BioDispersal\Subnetworks\LandUse\LandUse_ForestDisp_start_disp_tmp.tif'}
[2020-10-11 18:11:23.121460] [info] Call to r.cost successful, performed in 97.58465218544006 seconds
[2020-10-11 18:11:23.152712] [info] Calling processing algorithm 'gdal:rastercalculator'
[2020-10-11 18:11:23.152712] [warn] Unable to execute algorithm Could not load source layer for INPUT_A: C:\Users\aschw\Documents\School\2020Fall\ES400\ProjectWork\Layers\BioDispersal\Subnetworks\LandUse\LandUse_ForestDisp_start_disp_tmp.tif not found
[2020-10-11 18:11:23.152712] [warn] Failed to call rastercalculator : Unable to execute algorithm Could not load source layer for INPUT_A: C:\Users\aschw\Documents\School\2020Fall\ES400\ProjectWork\Layers\BioDispersal\Subnetworks\LandUse\LandUse_ForestDisp_start_disp_tmp.tif not found
[2020-10-11 18:11:23.152712] [Unexpected error] : Unable to execute algorithm Could not load source layer for INPUT_A: C:\Users\aschw\Documents\School\2020Fall\ES400\ProjectWork\Layers\BioDispersal\Subnetworks\LandUse\LandUse_ForestDisp_start_disp_tmp.tif not found