COG Generation

[alronlam]

This is from @tm-jc-nacpil:

Input: folder of geotiffs Output: processed cloud optimized geotiffs

Reference implementation here

def create_vrt(images_folder, output_vrt):
    "Create virtual dataset (VRT) of geotiff images in folder"

    logger.info(f"Generating VRT file {output_vrt} from {images_folder}")
    # We use a recursive wildcard (**/*.tif) to search within subdirectories
    buildvrt_command = f'gdalbuildvrt "{output_vrt}" "{images_folder}"/*.tif '
    os.system(buildvrt_command)

    return

def create_cog(input_vrt, output_cog):
    "Create cloud-optimized geotiff (COG) from a virtual dataset (VRT)"

    logger.info(f"Generating COG .tif file {output_cog} from {input_vrt}")
    start = time.time()
    os.system(f'gdal_translate "{input_vrt}" "{output_cog}" -of COG -co BIGTIFF=YES')
    end = time.time()
    elapsed = end - start
    logger.info(f"COG Generation Time: {str(timedelta(seconds=elapsed))}")

    return

Possible improvements we can add:

• add checks for estimated file size (these files are BIG),
• add checks for overlapping rasters (will cause bugs in the output COG)
• implement other GDAL settings like compression, tiling, etc.

Potential Blockers: in the current implementation, we use gdal CLI commands that we pass through os . Not sure if this will be a blocker or anti-pattern for GW since it might rely on something outside of python?

[alronlam]

We should keep GeoWrangler free from infra dependencies as much as possible for general use.

• GCS - Any download/upload from/to GCS should be separate from this. This is straightforward so should be no problem.
• gdal - I think the gdal dependency is fine since it's open-source and free. We just have to include in the instructions that users have to install gdal themselves for this function to work.

@butchtm @jtmiclat what do you think?

[alronlam]

Seems like there's also a way to generate the COG through the python gdal wrapper.

https://geoexamples.com/other/2019/02/08/cog-tutorial.html (see this section: Create a COG using gdal-python).

[alronlam]

Discussed with @tm-jc-nacpil and @jtmiclat, agreed to drop this feature request as this is just calling gdal commands. And that settings actually have to be tweaked depending on your tiffs (currently hardcoded). Better to do this striaght on gdal.