To Reproduce
Steps to reproduce the behavior on HPC: see [path/to]/operationalization_dev/geosys-jobs/jobs/NB_DNR_2013_a_INF_BUIL_2023-05-24T09h38_inference.job
This inference uses a huge image as input which results in the following error when it's time to write the inference to tif.
This results in the following error:
GDAL signalled an error: err_no=1, msg='TIFFAppendToStrip:Maximum TIFF file size exceeded. Use BIGTIFF=YES creation option.'
Expected behavior
We want to be able to write TIFFs larger than 4 gb. This can be done by modifying our create_new_raster_from_base to use the BIGTIFF=YES as suggested here.
Screenshots
Here's an example of an enhance image that couldn't write data beyond 4GB. The valid data is cut passed the 4GB threshold. For the records, this is QC_Sherbrooke_WV03_20220930-017162136010_01_P001-WV03_red-green-blue-nir_clahe25.tif (deleted on HPC)
Description GDL writes tifs at inference and these tifs may become very large:
To Reproduce Steps to reproduce the behavior on HPC: see
[path/to]/operationalization_dev/geosys-jobs/jobs/NB_DNR_2013_a_INF_BUIL_2023-05-24T09h38_inference.jobThis inference uses a huge image as input which results in the following error when it's time to write the inference to tif.This results in the following error:
GDAL signalled an error: err_no=1, msg='TIFFAppendToStrip:Maximum TIFF file size exceeded. Use BIGTIFF=YES creation option.'Expected behavior We want to be able to write TIFFs larger than 4 gb. This can be done by modifying our
create_new_raster_from_baseto use the BIGTIFF=YES as suggested here.Screenshots Here's an example of an enhance image that couldn't write data beyond 4GB. The valid data is cut passed the 4GB threshold. For the records, this is QC_Sherbrooke_WV03_20220930-017162136010_01_P001-WV03_red-green-blue-nir_clahe25.tif (deleted on HPC)