search

zoran-cuckovic / QGIS-visibility-analysis

Quantum GIS plugin for visibility analysis
GNU General Public License v3.0
73 stars 18 forks source link

Memory bloat #52

Closed zoran-cuckovic closed 3 years ago

zoran-cuckovic commented 3 years ago

When working with very large DEM rasters, the output of viewshed module gets unrealistically bloated (like 10 Gb or more for a 1Gb DEM). This happens only when the raster is larger than the buffer specified under processing options for the Viewshed plugin. For the time being this issue can be circumvented by enlarging the buffer size (image below). Note that this is live memory footprint - if too large it may choke up the computer.

image

zoran-cuckovic commented 3 years ago

Solved (kind of) in 1.6 version. The problem seems to be related to gdal compression routine for the output raster. I've removed the compression for situations when the algo has to work outside live memory (RAM). This will produce large output files, which can then be compressed manually (using gdal translate in QGIS, for example).

dmamalis commented 3 years ago

We might need to reopen this one. After upgrading to 3.16.4 and 1.6 I am having serious issues with memory. It looks like the algorithm does its job as normal but it takes quite a lot of resources to actually save the result. Let me know if I can provide more quality info.

zoran-cuckovic commented 3 years ago

Hello, The ressources you're refering to, are they live memory or hard disk memory?

dmamalis commented 3 years ago

I was just about to add a bit more info.I am referring to RAM and CPU usage. I am currently working on a local 400MB DSM raster with a 10 Viewpoints. When I try to do Viewshed analysis, memory ramps up to 14-15 GB and freezes the whole OS. When I try Intervisibility Analysis RAM ramps up and QGIS just crashes.