Add "COG" ( Cloud Optimized GeoTIFF generator) as possible output format in GDAL based tools

[ramotswa]

Feature description. I would like to be able to export a raster layer as a COG - be it via a checkbox option in the dialog or just as a format option.

Additional context I know that via the 'Export Raster' dialog I can get close (or possibly all the way) to COG by specifying TILED and a BLOCK{X,Y}SIZE as 512, but I still get an error when trying to validate the output tiff. I guess there are other create options I can use to fix those errors but it still seems a bit overkill considering a small change to the output format drop down to include 'COG' might be possible?

Errors:

The offset of the main IFD should be 8. It is 6291930 instead
The offset of the first block of the image should be after its IFD

Thanks

[gioman]

@ramotswa COG = https://gdal.org/drivers/raster/cog.html ?

[ramotswa]

Yes indeed.

[gioman]

@ramotswa look at the GDAL "translate" tool in the QGIS Processing toolbox, you have the option to add yourself new GDAL "creation options". For this format it seems you need to add the

COPY_SRC_OVERVIEWS=YES

one. You'll probably will find more infos in the GDAL man page.

[kannes]

That will only work if the file already has overviews which would make this a manual, two-or-more-step process. It would be great if the COG driver was directly selectable so GDAL could do all the magic.

[ramotswa]

Thanks @kannes . @gioman - even with that command it doesn't work as expected. Using input image s3://planet-disaster-data/hurricane-harvey/SkySat_Freeport_s03_20170831T162740Z3.tif Adding as a raster layer via S3. Then using the toolbox GDAL translate with that -co flag gives this command:

gdal_translate -of GTiff -co COPY_SRC_OVERVIEWS=YES /vsis3/planet-disaster-data/hurricane-harvey/SkySat_Freeport_s03_20170831T162740Z3.tif /Users/me/Downloads/skysat_test.tif

But clearly (surely?) that won't work as the -of is still GTiff right?

[kannes]

You can try to build overviews/pyramids for the file first, then try the export as suggested by gioman. That might work.

COG is just a special structure of GeoTIFF which you should get by doing these steps.

[gioman]

@ramotswa @kannes so, as far as I understand we should add "COG" option (to be used as " -of COG") among the list of the possible output formats (in GDAL based tools) but still maintain the .tiff extension for the same output, correct?

[ramotswa]

Hi @gioman - yes that sounds good.

[arkanoid87]

any news on this? I've been manually running GDAL command line to get COGs since 2.x. Exporting them from QGIS would be handy

[jo-chemla]

On our side, we're using a mixture of gdal -of COG and rio-cogeo CLI. Would love to have the COG export option (in addition to standard GTiff option) for non tech-savvy users!

If there are some pointers as to where to implement this feature, would love to give it a try (although I've never built qgis on my own yet)

[aritchie-usgs]

I also would love this. Seems like it could be done with either gdalwarp or gdal_translate. It would save me lots of re-writing files... These are my typical commands, optimized over several years for speed and file size, thanks to many folks posting in many places (I know some or all of these could be exposed options for the cog driver in the QGIS GUI, which would be great):

gdalwarp method gdalwarp -of cog -overwrite -multi --config GDAL_CACHEMAX 9000 -wm 9000 -co BLOCKSIZE=256 -co BIGTIFF=YES -co COMPRESS=DEFLATE -co PREDICTOR=YES -co NUM_THREADS=ALL_CPUS

gdal_translate method gdal_translate -of COG -co BLOCKSIZE=256 -co COMPRESS=DEFLATE -co PREDICTOR=YES -co NUM_THREADS=ALL_CPUS -co BIGTIFF=YES --config GDAL_CACHEMAX 9000

-edit- I just went into the preferences to see if I could make a "create" profile for COG and saw that in QGIS v3.32.0 /settings/options/raster drivers the gdal cog driver is listed with flags rwv and not rw+v - what does it take to get the + in there?

[GO1984]

Any news on this? As a end user i would like this feature too.

[tomkom]

following.

This seems like a great option to add. I believe GDAL should be creating all tifs as cog by default, but I fear the way the output specs are handled in the processing tools forces standard GTIFF.

[kannes]

To the best of my knowledge:

I looked into the code, trying if simply adding/enabling the format was possible.

Unfortunately this is not easy from the QGIS side.

The COG driver of GDAL only provides CreateCopy support, which means it can only be used as target for an an existing GDAL dataset.

QGIS uses Create everywhere instead, probably for its maximum flexibility and for the many features where rasters are created from scratch anyways.

I think the only place where it might be fairly easy to enable COG, is the gdal:translate Processing algorithm.

@rouault, could you spare some thoughts on this? Would it be imaginable that GDAL got Create support for COG (considering the versatility of the format) or would it rather be on QGIS' side to rebuilt its raster creation?

[rouault]

Would it be imaginable that GDAL got Create support for COG (considering the versatility of the format)

No, that's not technically doable. Create() involves random writing. COG creation requires to have access to the full resolution dataset in its whole at the beginning of the process to start being overviews

or would it rather be on QGIS' side to rebuilt its raster creation?

On QGIS side there would be 2 possibilities:

• either QGIS would use Create() into a temporary TIFF and at the end, use CreateCopy() it to COG
• or QGIS would implement the GDALDataset C++ class to wrap a QgsRasterLayer, and particularly override the IReadBlock/IRasterIO methods to forward them to its own API, and use that as the input of the COG driver. This would save the need to create a temporary file, but requires more coding.

[kannes]

Thank you!