Open lachlanhurst opened 8 years ago
Quick and dirty fix would be to skip over the wmts
directory when you walk over the file system. Although I'm not sure you need to recursively traverse the file system for this.
Relevant log output, you'll note there's three TMS generation requests. First is successful, last two fail.
rsa_1 | INFO o.v.w.c.DataController - messageBox((2250000.0, 2100000.0) - (2400000.0, 2500000.0))
rsa_1 | INFO o.v.w.c.DataController - messageBox((2250000.0, 2500000.0) - (2400000.0, 2967100.0))
rsa_1 | INFO o.v.w.c.DataController - messageBox((2400000.0, 2100000.0) - (2860200.0, 2500000.0))
rsa_1 | INFO o.v.w.c.DataController - messageBox((2400000.0, 2500000.0) - (2860200.0, 2967100.0))
rsa_1 | INFO o.v.n.t.Application - Temporary directory: /var/tmp/ndg/tmpDir475797298009994556
rsa_1 | INFO o.v.n.t.WmtsQueryCreator - Initialising WMTS tile creation for query with task/job id 2c91808b557fd37d015595e78f1d0027
rsa_1 | INFO o.v.n.t.WmtsQueryCreator - Query Job Progress : org.vpac.ndg.storage.model.JobProgress@638b19a3
rsa_1 | INFO o.v.n.t.WmtsQueryCreator - TASK 6, vrtWithColourFile:GraphicsFile(/var/spool/ndg/pickup/2c91808b557fd37d015595e78f1d0027/wmts/2c91808b557fd37d015595e78f1d0027_Colour.vrt) with resolution = null
rsa_1 | INFO o.v.n.t.WmtsQueryCreator - TASK 6, palette:org.vpac.ndg.colour.HashPalette@5f74154c
rsa_1 | INFO o.v.n.t.WmtsQueryCreator - TASK 7, vrtWithColourFile:GraphicsFile(/var/spool/ndg/pickup/2c91808b557fd37d015595e78f1d0027/wmts/2c91808b557fd37d015595e78f1d0027_Colour.vrt) with resolution = null
rsa_1 | INFO o.v.n.t.TaskPipeline - Start of task initialisation.
rsa_1 | INFO o.v.n.t.Application - Task starting: 2c91808b557fd37d015595e8372e0028
rsa_1 | INFO o.v.n.t.TaskPipeline - START OF TASK EXECUTION
rsa_1 | INFO o.v.n.t.OutputDirStatistics - filePath:/var/spool/ndg/pickup/2c91808b557fd37d015595e78f1d0027
rsa_1 | INFO o.v.n.t.OutputDirStatistics - filePath:/var/spool/ndg/pickup/2c91808b557fd37d015595e78f1d0027/71feaf9b-37ac-4eaa-972c-ddfb1683beff_out.nc
rsa_1 | INFO o.v.n.t.OutputDirStatistics - command:[gdalinfo, -noct, -stats, /var/spool/ndg/pickup/2c91808b557fd37d015595e78f1d0027/71feaf9b-37ac-4eaa-972c-ddfb1683beff_out.nc]
rsa_1 | INFO o.v.n.t.OutputDirStatistics - Found No data value of -9999.0
rsa_1 | INFO o.v.n.t.OutputDirStatistics - filePath:/var/spool/ndg/pickup/2c91808b557fd37d015595e78f1d0027/wmts
rsa_1 | INFO o.v.n.t.OutputDirStatistics - filePath:/var/spool/ndg/pickup/2c91808b557fd37d015595e78f1d0027/wmts/2c91808b557fd37d015595e78f1d0027
rsa_1 | INFO o.v.n.t.OutputDirStatistics - filePath:/var/spool/ndg/pickup/2c91808b557fd37d015595e78f1d0027/162eb3f9-1120-479e-a873-d4bf23ce787a_out.nc
rsa_1 | INFO o.v.n.t.OutputDirStatistics - command:[gdalinfo, -noct, -stats, /var/spool/ndg/pickup/2c91808b557fd37d015595e78f1d0027/162eb3f9-1120-479e-a873-d4bf23ce787a_out.nc]
rsa_1 | INFO o.v.n.t.OutputDirStatistics - Found No data value of -9999.0
rsa_1 | INFO o.v.n.t.OutputDirStatistics - Found max of 23.0
rsa_1 | INFO o.v.n.t.OutputDirStatistics - Found min of 12.0
rsa_1 | INFO o.v.n.t.OutputDirStatistics - filePath:/var/spool/ndg/pickup/2c91808b557fd37d015595e78f1d0027/18f1948e-b5a2-4869-83b2-7bf2286f86f7_out.nc
rsa_1 | INFO o.v.n.t.OutputDirStatistics - command:[gdalinfo, -noct, -stats, /var/spool/ndg/pickup/2c91808b557fd37d015595e78f1d0027/18f1948e-b5a2-4869-83b2-7bf2286f86f7_out.nc]
rsa_1 | INFO o.v.n.t.OutputDirStatistics - Found No data value of -9999.0
rsa_1 | INFO o.v.n.t.OutputDirStatistics - Found max of 26.0
rsa_1 | INFO o.v.n.t.OutputDirStatistics - Found min of 12.0
rsa_1 | INFO o.v.n.t.OutputDirStatistics - filePath:/var/spool/ndg/pickup/2c91808b557fd37d015595e78f1d0027/399d0977-2922-471c-bd97-184f39cf5047_out.nc
rsa_1 | INFO o.v.n.t.OutputDirStatistics - command:[gdalinfo, -noct, -stats, /var/spool/ndg/pickup/2c91808b557fd37d015595e78f1d0027/399d0977-2922-471c-bd97-184f39cf5047_out.nc]
rsa_1 | INFO o.v.n.t.OutputDirStatistics - Found No data value of -9999.0
rsa_1 | INFO o.v.n.t.OutputDirStatistics - Found max of 6.0
rsa_1 | INFO o.v.n.t.OutputDirStatistics - Found min of 6.0
rsa_1 | INFO o.v.n.t.VrtBuilder - getPixelType:null
rsa_1 | INFO o.v.n.t.VrtBuilder - command:[gdalbuildvrt, -overwrite, -tr, 66.1459656252646, 66.1459656252646, -te, 1786000.0, 1997264.4991957, 2999976.907120481, 2899958.492083686, /var/spool/ndg/pickup/2c91808b557fd37d015595e78f1d0027/wmts/2c91808b557fd37d015595e78f1d0027_scaled.vrt, /var/spool/ndg/pickup/2c91808b557fd37d015595e78f1d0027/18f1948e-b5a2-4869-83b2-7bf2286f86f7_out.nc.aux.xml, /var/spool/ndg/pickup/2c91808b557fd37d015595e78f1d0027/162eb3f9-1120-479e-a873-d4bf23ce787a_out.nc.aux.xml, /var/spool/ndg/pickup/2c91808b557fd37d015595e78f1d0027/399d0977-2922-471c-bd97-184f39cf5047_out.nc.aux.xml, /var/spool/ndg/pickup/2c91808b557fd37d015595e78f1d0027/71feaf9b-37ac-4eaa-972c-ddfb1683beff_out.nc, /var/spool/ndg/pickup/2c91808b557fd37d015595e78f1d0027/162eb3f9-1120-479e-a873-d4bf23ce787a_out.nc, /var/spool/ndg/pickup/2c91808b557fd37d015595e78f1d0027/18f1948e-b5a2-4869-83b2-7bf2286f86f7_out.nc, /var/spool/ndg/pickup/2c91808b557fd37d015595e78f1d0027/399d0977-2922-471c-bd97-184f39cf5047_out.nc]
rsa_1 | INFO o.v.n.t.FileStatistics - command:[gdalinfo, -noct, -stats, /var/spool/ndg/pickup/2c91808b557fd37d015595e78f1d0027/wmts/2c91808b557fd37d015595e78f1d0027_scaled.vrt]
rsa_1 | INFO o.v.n.t.FileStatistics - gdalinfo -noct -stats /var/spool/ndg/pickup/2c91808b557fd37d015595e78f1d0027/wmts/2c91808b557fd37d015595e78f1d0027_scaled.vrt
rsa_1 | INFO o.v.w.c.DataController - Data getTableByTaskId
rsa_1 | INFO o.v.w.c.DataController - Data getTableByTaskId
rsa_1 | INFO o.v.n.t.FileStatistics - Found No data value of -9999.0
rsa_1 | INFO o.v.n.t.FileStatistics - Found max of 26.0
rsa_1 | INFO o.v.n.t.FileStatistics - Found mean of 18.181842772204
rsa_1 | INFO o.v.n.t.FileStatistics - Found min of 6.0
rsa_1 | INFO o.v.n.t.FileStatistics - Found stddev of 4.3008764015185
rsa_1 | INFO o.v.n.t.Translator - value:6.0
rsa_1 | INFO o.v.n.t.Translator - value:26.0
rsa_1 | INFO o.v.n.t.Translator - value:1.0
rsa_1 | INFO o.v.n.t.Translator - value:255.0
rsa_1 | INFO o.v.n.t.Translator - Translator command[gdal_translate, -of, GTiff, -ot, Byte, -scale, 6.0, 26.0, 1.0, 255.0, --config, GDAL_CACHEMAX, 1000, -co, COMPRESS=LZW, -co, BIGTIFF=IF_SAFER, /var/spool/ndg/pickup/2c91808b557fd37d015595e78f1d0027/wmts/2c91808b557fd37d015595e78f1d0027_scaled.vrt, /var/spool/ndg/pickup/2c91808b557fd37d015595e78f1d0027/wmts/2c91808b557fd37d015595e78f1d0027_byte.tif]
rsa_1 | INFO o.v.n.t.VrtBuilder - command:[gdalbuildvrt, -overwrite, -resolution, highest, /var/spool/ndg/pickup/2c91808b557fd37d015595e78f1d0027/wmts/2c91808b557fd37d015595e78f1d0027_noColour.vrt, /var/spool/ndg/pickup/2c91808b557fd37d015595e78f1d0027/wmts/2c91808b557fd37d015595e78f1d0027_byte.tif]
rsa_1 | INFO o.v.n.t.VrtColouriser - Using palette org.vpac.ndg.colour.HashPalette@5f74154c
rsa_1 | INFO o.v.n.t.Translator - Translator command[gdal_translate, -of, VRT, -expand, rgba, --config, GDAL_CACHEMAX, 1000, /var/spool/ndg/pickup/2c91808b557fd37d015595e78f1d0027/wmts/2c91808b557fd37d015595e78f1d0027_Colour.vrt, /var/spool/ndg/pickup/2c91808b557fd37d015595e78f1d0027/wmts/2c91808b557fd37d015595e78f1d0027_Colour_Expanded.vrt]
rsa_1 | INFO o.v.n.t.TileBuilder - gdal2tile:[gdal2tiles.py, -p, raster, -z, 0-7, /var/spool/ndg/pickup/2c91808b557fd37d015595e78f1d0027/wmts/2c91808b557fd37d015595e78f1d0027_Colour_Expanded.vrt, /var/spool/ndg/pickup/2c91808b557fd37d015595e78f1d0027/wmts/2c91808b557fd37d015595e78f1d0027]
rsa_1 | INFO o.v.w.c.DataController - Data getTableByTaskId
rsa_1 | INFO o.v.n.t.Application - Task finished: 2c91808b557fd37d015595e8372e0028
rsa_1 | INFO o.v.n.t.TaskPipeline - START OF TASK FINALISE
rsa_1 | INFO o.v.n.t.TaskPipeline - END OF TASK
rsa_1 | INFO o.v.n.t.Application - Temporary directory: /var/tmp/ndg/tmpDir4634680853858788138
rsa_1 | INFO o.v.n.t.WmtsQueryCreator - Initialising WMTS tile creation for query with task/job id 2c91808b557fd37d015595e78f1d0027
rsa_1 | INFO o.v.n.t.WmtsQueryCreator - Query Job Progress : org.vpac.ndg.storage.model.JobProgress@26591cc9
rsa_1 | INFO o.v.n.t.WmtsQueryCreator - TASK 6, vrtWithColourFile:GraphicsFile(/var/spool/ndg/pickup/2c91808b557fd37d015595e78f1d0027/wmts/2c91808b557fd37d015595e78f1d0027_Colour.vrt) with resolution = null
rsa_1 | INFO o.v.n.t.WmtsQueryCreator - TASK 6, palette:org.vpac.ndg.colour.HashPalette@665e796b
rsa_1 | INFO o.v.n.t.WmtsQueryCreator - TASK 7, vrtWithColourFile:GraphicsFile(/var/spool/ndg/pickup/2c91808b557fd37d015595e78f1d0027/wmts/2c91808b557fd37d015595e78f1d0027_Colour.vrt) with resolution = null
rsa_1 | INFO o.v.n.t.TaskPipeline - Start of task initialisation.
rsa_1 | INFO o.v.n.t.Application - Task starting: 2c91808b557fd37d015595f15dba0029
rsa_1 | INFO o.v.n.t.TaskPipeline - START OF TASK EXECUTION
rsa_1 | INFO o.v.n.t.OutputDirStatistics - filePath:/var/spool/ndg/pickup/2c91808b557fd37d015595e78f1d0027
rsa_1 | INFO o.v.n.t.OutputDirStatistics - filePath:/var/spool/ndg/pickup/2c91808b557fd37d015595e78f1d0027/18f1948e-b5a2-4869-83b2-7bf2286f86f7_out.nc.aux.xml
rsa_1 | INFO o.v.n.t.OutputDirStatistics - command:[gdalinfo, -noct, -stats, /var/spool/ndg/pickup/2c91808b557fd37d015595e78f1d0027/18f1948e-b5a2-4869-83b2-7bf2286f86f7_out.nc.aux.xml]
rsa_1 | INFO o.v.n.t.OutputDirStatistics - filePath:/var/spool/ndg/pickup/2c91808b557fd37d015595e78f1d0027/162eb3f9-1120-479e-a873-d4bf23ce787a_out.nc.aux.xml
rsa_1 | INFO o.v.n.t.OutputDirStatistics - command:[gdalinfo, -noct, -stats, /var/spool/ndg/pickup/2c91808b557fd37d015595e78f1d0027/162eb3f9-1120-479e-a873-d4bf23ce787a_out.nc.aux.xml]
rsa_1 | INFO o.v.n.t.OutputDirStatistics - filePath:/var/spool/ndg/pickup/2c91808b557fd37d015595e78f1d0027/399d0977-2922-471c-bd97-184f39cf5047_out.nc.aux.xml
rsa_1 | INFO o.v.n.t.OutputDirStatistics - command:[gdalinfo, -noct, -stats, /var/spool/ndg/pickup/2c91808b557fd37d015595e78f1d0027/399d0977-2922-471c-bd97-184f39cf5047_out.nc.aux.xml]
rsa_1 | INFO o.v.n.t.OutputDirStatistics - filePath:/var/spool/ndg/pickup/2c91808b557fd37d015595e78f1d0027/71feaf9b-37ac-4eaa-972c-ddfb1683beff_out.nc
rsa_1 | INFO o.v.n.t.OutputDirStatistics - command:[gdalinfo, -noct, -stats, /var/spool/ndg/pickup/2c91808b557fd37d015595e78f1d0027/71feaf9b-37ac-4eaa-972c-ddfb1683beff_out.nc]
rsa_1 | INFO o.v.n.t.OutputDirStatistics - Found No data value of -9999.0
rsa_1 | INFO o.v.n.t.OutputDirStatistics - filePath:/var/spool/ndg/pickup/2c91808b557fd37d015595e78f1d0027/wmts
rsa_1 | INFO o.v.n.t.OutputDirStatistics - filePath:/var/spool/ndg/pickup/2c91808b557fd37d015595e78f1d0027/wmts/2c91808b557fd37d015595e78f1d0027_Colour.vrt
rsa_1 | INFO o.v.n.t.OutputDirStatistics - command:[gdalinfo, -noct, -stats, /var/spool/ndg/pickup/2c91808b557fd37d015595e78f1d0027/wmts/2c91808b557fd37d015595e78f1d0027_Colour.vrt]
rsa_1 | INFO o.v.n.t.OutputDirStatistics - Found No data value of -9999.0
rsa_1 | INFO o.v.n.t.OutputDirStatistics - Found max of 255.0
rsa_1 | INFO o.v.n.t.OutputDirStatistics - Found min of 0.0
rsa_1 | INFO o.v.n.t.OutputDirStatistics - filePath:/var/spool/ndg/pickup/2c91808b557fd37d015595e78f1d0027/wmts/2c91808b557fd37d015595e78f1d0027
rsa_1 | INFO o.v.n.t.OutputDirStatistics - filePath:/var/spool/ndg/pickup/2c91808b557fd37d015595e78f1d0027/wmts/2c91808b557fd37d015595e78f1d0027_byte.tif
rsa_1 | INFO o.v.n.t.OutputDirStatistics - command:[gdalinfo, -noct, -stats, /var/spool/ndg/pickup/2c91808b557fd37d015595e78f1d0027/wmts/2c91808b557fd37d015595e78f1d0027_byte.tif]
rsa_1 | INFO o.v.n.t.OutputDirStatistics - Found No data value of -9999.0
rsa_1 | INFO o.v.n.t.OutputDirStatistics - Found max of 255.0
rsa_1 | INFO o.v.n.t.OutputDirStatistics - Found min of 0.0
rsa_1 | INFO o.v.n.t.OutputDirStatistics - filePath:/var/spool/ndg/pickup/2c91808b557fd37d015595e78f1d0027/wmts/2c91808b557fd37d015595e78f1d0027_noColour.vrt
rsa_1 | INFO o.v.n.t.OutputDirStatistics - command:[gdalinfo, -noct, -stats, /var/spool/ndg/pickup/2c91808b557fd37d015595e78f1d0027/wmts/2c91808b557fd37d015595e78f1d0027_noColour.vrt]
rsa_1 | INFO o.v.n.t.OutputDirStatistics - Found No data value of -9999.0
rsa_1 | INFO o.v.n.t.OutputDirStatistics - Found max of 255.0
rsa_1 | INFO o.v.n.t.OutputDirStatistics - Found min of 0.0
rsa_1 | INFO o.v.n.t.OutputDirStatistics - filePath:/var/spool/ndg/pickup/2c91808b557fd37d015595e78f1d0027/wmts/2c91808b557fd37d015595e78f1d0027_scaled.vrt
rsa_1 | INFO o.v.n.t.OutputDirStatistics - command:[gdalinfo, -noct, -stats, /var/spool/ndg/pickup/2c91808b557fd37d015595e78f1d0027/wmts/2c91808b557fd37d015595e78f1d0027_scaled.vrt]
rsa_1 | INFO o.v.n.t.OutputDirStatistics - Found No data value of -9999.0
rsa_1 | INFO o.v.n.t.OutputDirStatistics - Found max of 26.0
rsa_1 | INFO o.v.n.t.OutputDirStatistics - Found min of 6.0
rsa_1 | INFO o.v.n.t.OutputDirStatistics - filePath:/var/spool/ndg/pickup/2c91808b557fd37d015595e78f1d0027/wmts/2c91808b557fd37d015595e78f1d0027_Colour_Expanded.vrt
rsa_1 | INFO o.v.n.t.OutputDirStatistics - command:[gdalinfo, -noct, -stats, /var/spool/ndg/pickup/2c91808b557fd37d015595e78f1d0027/wmts/2c91808b557fd37d015595e78f1d0027_Colour_Expanded.vrt]
rsa_1 | INFO o.v.w.c.DataController - Data getTableByTaskId
rsa_1 | INFO o.v.n.t.Application - Temporary directory: /var/tmp/ndg/tmpDir8764667314236510366
rsa_1 | INFO o.v.n.t.WmtsQueryCreator - Initialising WMTS tile creation for query with task/job id 2c91808b557fd37d015595e78f1d0027
rsa_1 | INFO o.v.n.t.WmtsQueryCreator - Query Job Progress : org.vpac.ndg.storage.model.JobProgress@7701d82e
rsa_1 | INFO o.v.n.t.WmtsQueryCreator - TASK 6, vrtWithColourFile:GraphicsFile(/var/spool/ndg/pickup/2c91808b557fd37d015595e78f1d0027/wmts/2c91808b557fd37d015595e78f1d0027_Colour.vrt) with resolution = null
rsa_1 | INFO o.v.n.t.WmtsQueryCreator - TASK 6, palette:org.vpac.ndg.colour.HashPalette@8004654
rsa_1 | INFO o.v.n.t.WmtsQueryCreator - TASK 7, vrtWithColourFile:GraphicsFile(/var/spool/ndg/pickup/2c91808b557fd37d015595e78f1d0027/wmts/2c91808b557fd37d015595e78f1d0027_Colour.vrt) with resolution = null
rsa_1 | INFO o.v.n.t.TaskPipeline - Start of task initialisation.
rsa_1 | INFO o.v.n.t.Application - Task starting: 2c91808b557fd37d015595f507c9002a
rsa_1 | INFO o.v.n.t.TaskPipeline - START OF TASK EXECUTION
rsa_1 | INFO o.v.n.t.OutputDirStatistics - filePath:/var/spool/ndg/pickup/2c91808b557fd37d015595e78f1d0027
rsa_1 | INFO o.v.n.t.OutputDirStatistics - filePath:/var/spool/ndg/pickup/2c91808b557fd37d015595e78f1d0027/18f1948e-b5a2-4869-83b2-7bf2286f86f7_out.nc.aux.xml
rsa_1 | INFO o.v.n.t.OutputDirStatistics - command:[gdalinfo, -noct, -stats, /var/spool/ndg/pickup/2c91808b557fd37d015595e78f1d0027/18f1948e-b5a2-4869-83b2-7bf2286f86f7_out.nc.aux.xml]
rsa_1 | INFO o.v.n.t.OutputDirStatistics - filePath:/var/spool/ndg/pickup/2c91808b557fd37d015595e78f1d0027/162eb3f9-1120-479e-a873-d4bf23ce787a_out.nc.aux.xml
rsa_1 | INFO o.v.n.t.OutputDirStatistics - command:[gdalinfo, -noct, -stats, /var/spool/ndg/pickup/2c91808b557fd37d015595e78f1d0027/162eb3f9-1120-479e-a873-d4bf23ce787a_out.nc.aux.xml]
rsa_1 | INFO o.v.n.t.OutputDirStatistics - filePath:/var/spool/ndg/pickup/2c91808b557fd37d015595e78f1d0027/399d0977-2922-471c-bd97-184f39cf5047_out.nc.aux.xml
rsa_1 | INFO o.v.n.t.OutputDirStatistics - command:[gdalinfo, -noct, -stats, /var/spool/ndg/pickup/2c91808b557fd37d015595e78f1d0027/399d0977-2922-471c-bd97-184f39cf5047_out.nc.aux.xml]
rsa_1 | INFO o.v.n.t.OutputDirStatistics - filePath:/var/spool/ndg/pickup/2c91808b557fd37d015595e78f1d0027/71feaf9b-37ac-4eaa-972c-ddfb1683beff_out.nc
rsa_1 | INFO o.v.n.t.OutputDirStatistics - command:[gdalinfo, -noct, -stats, /var/spool/ndg/pickup/2c91808b557fd37d015595e78f1d0027/71feaf9b-37ac-4eaa-972c-ddfb1683beff_out.nc]
rsa_1 | INFO o.v.n.t.OutputDirStatistics - Found No data value of -9999.0
rsa_1 | INFO o.v.n.t.OutputDirStatistics - filePath:/var/spool/ndg/pickup/2c91808b557fd37d015595e78f1d0027/wmts
rsa_1 | INFO o.v.n.t.OutputDirStatistics - filePath:/var/spool/ndg/pickup/2c91808b557fd37d015595e78f1d0027/wmts/2c91808b557fd37d015595e78f1d0027_byte.tif.aux.xml
rsa_1 | INFO o.v.n.t.OutputDirStatistics - command:[gdalinfo, -noct, -stats, /var/spool/ndg/pickup/2c91808b557fd37d015595e78f1d0027/wmts/2c91808b557fd37d015595e78f1d0027_byte.tif.aux.xml]
rsa_1 | INFO o.v.n.t.OutputDirStatistics - filePath:/var/spool/ndg/pickup/2c91808b557fd37d015595e78f1d0027/wmts/2c91808b557fd37d015595e78f1d0027_Colour.vrt
rsa_1 | INFO o.v.n.t.OutputDirStatistics - command:[gdalinfo, -noct, -stats, /var/spool/ndg/pickup/2c91808b557fd37d015595e78f1d0027/wmts/2c91808b557fd37d015595e78f1d0027_Colour.vrt]
rsa_1 | INFO o.v.n.t.OutputDirStatistics - Found No data value of -9999.0
rsa_1 | INFO o.v.n.t.OutputDirStatistics - Found max of 255.0
rsa_1 | INFO o.v.n.t.OutputDirStatistics - Found min of 0.0
rsa_1 | INFO o.v.n.t.OutputDirStatistics - filePath:/var/spool/ndg/pickup/2c91808b557fd37d015595e78f1d0027/wmts/2c91808b557fd37d015595e78f1d0027
rsa_1 | INFO o.v.n.t.OutputDirStatistics - filePath:/var/spool/ndg/pickup/2c91808b557fd37d015595e78f1d0027/wmts/2c91808b557fd37d015595e78f1d0027_byte.tif
rsa_1 | INFO o.v.n.t.OutputDirStatistics - command:[gdalinfo, -noct, -stats, /var/spool/ndg/pickup/2c91808b557fd37d015595e78f1d0027/wmts/2c91808b557fd37d015595e78f1d0027_byte.tif]
rsa_1 | INFO o.v.n.t.OutputDirStatistics - Found No data value of -9999.0
rsa_1 | INFO o.v.n.t.OutputDirStatistics - Found max of 255.0
rsa_1 | INFO o.v.n.t.OutputDirStatistics - Found min of 0.0
rsa_1 | INFO o.v.n.t.OutputDirStatistics - filePath:/var/spool/ndg/pickup/2c91808b557fd37d015595e78f1d0027/wmts/2c91808b557fd37d015595e78f1d0027_noColour.vrt
rsa_1 | INFO o.v.n.t.OutputDirStatistics - command:[gdalinfo, -noct, -stats, /var/spool/ndg/pickup/2c91808b557fd37d015595e78f1d0027/wmts/2c91808b557fd37d015595e78f1d0027_noColour.vrt]
rsa_1 | INFO o.v.n.t.OutputDirStatistics - Found No data value of -9999.0
rsa_1 | INFO o.v.n.t.OutputDirStatistics - Found max of 255.0
rsa_1 | INFO o.v.n.t.OutputDirStatistics - Found min of 0.0
rsa_1 | INFO o.v.n.t.OutputDirStatistics - filePath:/var/spool/ndg/pickup/2c91808b557fd37d015595e78f1d0027/wmts/2c91808b557fd37d015595e78f1d0027_scaled.vrt
rsa_1 | INFO o.v.n.t.OutputDirStatistics - command:[gdalinfo, -noct, -stats, /var/spool/ndg/pickup/2c91808b557fd37d015595e78f1d0027/wmts/2c91808b557fd37d015595e78f1d0027_scaled.vrt]
rsa_1 | INFO o.v.n.t.OutputDirStatistics - Found No data value of -9999.0
rsa_1 | INFO o.v.n.t.OutputDirStatistics - Found max of 26.0
rsa_1 | INFO o.v.n.t.OutputDirStatistics - Found min of 6.0
rsa_1 | INFO o.v.n.t.OutputDirStatistics - filePath:/var/spool/ndg/pickup/2c91808b557fd37d015595e78f1d0027/wmts/2c91808b557fd37d015595e78f1d0027_Colour_Expanded.vrt
rsa_1 | INFO o.v.n.t.OutputDirStatistics - command:[gdalinfo, -noct, -stats, /var/spool/ndg/pickup/2c91808b557fd37d015595e78f1d0027/wmts/2c91808b557fd37d015595e78f1d0027_Colour_Expanded.vrt]
rsa_1 | INFO o.v.w.c.DataController - Data getTableByTaskId
rsa_1 | INFO o.v.w.c.DataController - Data getTableByTaskId
This problem still exists in the latest build. TMS tile generation can be requested, and at some point fail without notification.
The last GDAL command shown in the logs in this instance was
gdalinfo -noct -stats abcdef_Colour_Expanded.vrt
. GDAL actually 'kind of' seems to fail processing this file but returns 0 as an exit code. To the point by the_Colour_Expanded.vrt
step there is no point at all attempting to extract min and max values as the data has already been converted into the three colour bands that must have a value between 0-255. This command should never need to be run, although there is something else in play... usually we wouldn't expect to find other files in the query output dir at the first stage of the task pipeline.The suspect class seems to be
OutputDirStatistics
which simply loops over all files irrespective of what they are (could be xml, could be anything) and runs withgdalinfo
. There's a catch block that absorbs anyIOExceptions
that may be reported link.FileStatistics
is still included in the pipeline, but I'm not sure if it's been superseded byOutputDirStatistics
. Also, I note the exception it once threw when failing to extract the min/max has been commented out. Was there a reason for this?I'm not aware of the issue behind the switch from
FileStatistics
toOutputDirStatistics
, but I'd suggest looking at howVrtBuilder
obtains a list of files from a query output and do similar inOutputDirStatistics
.Happy to go through this tomorrow sometime @forjin-vpac-innovations .