search

zoran-cuckovic / QGIS-visibility-analysis

Quantum GIS plugin for visibility analysis
GNU General Public License v3.0
72 stars 17 forks source link

Viewshed data type Float32 #40

Closed PedroVenancio closed 4 years ago

PedroVenancio commented 4 years ago

Dear Zoran,

I'm conducting a viewshed analysis of about 1700 points and I need the individual viewsheds from each point.

The plugin works like a charm to do it, iterating over each input point.

The problem is that each output has about 1GB, which multiplied by 1700 is a huge amount of hard drive space, that I don't have!

I've tried multiple approachs to overcome this, including programing a crojob/task to delete temporary layers, but it does not work on windows, because temporary layers are blocked until the process finish.

So, my question is, the plugin output is always a Float32 - Thirty two bit floating point, even when it only has pixels with 0 and 1.

Is there any possibility to adapt the output raster type to the content it needs?

Thank you very much!

zoran-cuckovic commented 4 years ago

Hello! The most straightforward way to do this would be to use gdal (Raster > Conversion > Translate). On the other hand, you may reduce the data volume by 4 at maximum (when cast to byte format), which doesn't seem to resolve your problem.... What are you trying to achieve with your analysis ?

PedroVenancio commented 4 years ago

Hi Zoran,

This is the model I'm running

Model

The problem is only a matter of space. The Viewshed output is a Float32, it is only temporary in the model, but it is stored in hard drive.

The second step of the model, that just change the pixel value 0 to null, gives a temporary output which reduces the 1GB raster from the first step, to a 16MB raster.

Looking at the metadata of both, the reason to that difference become evident:

  1. Viewshed output

    Name    OUTPUT1
Path    D:\Testes\Viewshed\OUTPUT1.tif
CRS EPSG:3763 - ETRS89 / Portugal TM06 - Projected
Extent  -120000.0000000000000000,-302000.0000000000000000 : 164000.0000000000000000,278000.0000000000000000
Unit    meters
Width   11360
Height  23200
Data type   Float32 - Thirty two bit floating point
GDAL Driver Description GTiff
GDAL Driver Metadata    GeoTIFF
Dataset Description D:\Testes\Viewshed\OUTPUT1.tif
Compression 
Band 1  
STATISTICS_APPROXIMATE=YES
STATISTICS_MAXIMUM=1
STATISTICS_MEAN=0.23503808487486
STATISTICS_MINIMUM=0
STATISTICS_STDDEV=0.4240226214876
STATISTICS_VALID_PERCENT=0.05287
More information    
AREA_OR_POINT=Area
Dimensions  X: 11360 Y: 23200 Bands: 1
Origin  -120000,278000
Pixel Size  25,-25

  2. r.null output

    Name    output
Path    D:\Testes\Viewshed\output.tif
CRS EPSG:3763 - ETRS89 / Portugal TM06 - Projected
Extent  -120000.0000000000000000,-302000.0000000000000000 : 164000.0000000000000000,278000.0000000000000000
Unit    meters
Width   11360
Height  23200
Data type   Float32 - Thirty two bit floating point
GDAL Driver Description GTiff
GDAL Driver Metadata    GeoTIFF
Dataset Description D:\Testes\Viewshed\output.tif
Compression LZW
Band 1  
STATISTICS_APPROXIMATE=YES
STATISTICS_MAXIMUM=1
STATISTICS_MEAN=1
STATISTICS_MINIMUM=1
STATISTICS_STDDEV=0
STATISTICS_VALID_PERCENT=0.01243
More information    
AREA_OR_POINT=Area
TIFFTAG_SOFTWARE=GRASS GIS 7.8.2 with GDAL 3.0.4
Dimensions  X: 11360 Y: 23200 Bands: 1
Origin  -120000,278000
Pixel Size  25,-25

GRASS applies LZW compression to the output.

Do you think this could also be done in Visibility Analysis plugin?

Thanks Zoran!

zoran-cuckovic commented 4 years ago

I didn't implement particular raster compression routines - these can be done using dedicated tools (gdal translate for instance, available in Processing). You can add this to your model. Considering your approach, you need only one raster at the time, perhaps it is possible to delete/overwrite rasters on the fly (I don't use modeller much...) ?