Issue using the WPS gs:Download for some raster layers

[tdipisa]

While testing the gs:Download process through the new MapStore export tool I found the following issue that has been originally reported in this MapStore issue.

There is a GeoServer error if I try to export the nurc:mosaic raster layer (it seems related to an issue previously reported here).

Request URL: https://gs-stable.geo-solutions.it/geoserver/wps?service=WPS&version=1.0.0&REQUEST=Execute&authkey=41db6295-c6df-4384-ad5b-edea0f4acf5b

Request body:

<?xml version="1.0" encoding="UTF-8"?>
<wps:Execute version="1.0.0"
            service="WPS"
            xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
            xmlns="http://www.opengis.net/wps/1.0.0"
            xmlns:wfs="http://www.opengis.net/wfs"
            xmlns:wps="http://www.opengis.net/wps/1.0.0"
            xmlns:ows="http://www.opengis.net/ows/1.1"
            xmlns:gml="http://www.opengis.net/gml"
            xmlns:ogc="http://www.opengis.net/ogc"
            xmlns:wcs="http://www.opengis.net/wcs/1.1.1"
            xmlns:dwn="http://geoserver.org/wps/download"
            xmlns:xlink="http://www.w3.org/1999/xlink"
            xsi:schemaLocation="http://www.opengis.net/wps/1.0.0 http://schemas.opengis.net/wps/1.0.0/wpsAll.xsd">
    <ows:Identifier>gs:DownloadEstimator</ows:Identifier>
    <wps:DataInputs>
        <wps:Input>
            <ows:Identifier>layerName</ows:Identifier>
            <wps:Data>
                <wps:LiteralData>nurc:mosaic</wps:LiteralData>
            </wps:Data>
        </wps:Input>
        <wps:Input>
            <ows:Identifier>ROI</ows:Identifier>
            <wps:Data>
                <wps:ComplexData mimeType="application/json"><![CDATA[{"type":"Polygon","coordinates":[[[918467.3318746778,5181971.895565239],[918467.3318746778,5462954.411541547],[1505503.7091048313,5462954.411541547],[1505503.7091048313,5181971.895565239],[918467.3318746778,5181971.895565239]]]}]]></wps:ComplexData>
            </wps:Data>
        </wps:Input>
        <wps:Input>
            <ows:Identifier>RoiCRS</ows:Identifier>
            <wps:Data>
                <wps:LiteralData>EPSG:3857</wps:LiteralData>
            </wps:Data>
        </wps:Input>
    </wps:DataInputs>
</wps:Execute>

GeoServer response:

<?xml version="1.0" encoding="UTF-8"?>
<wps:ExecuteResponse xmlns:xs="http://www.w3.org/2001/XMLSchema"
                    xmlns:ows="http://www.opengis.net/ows/1.1"
                    xmlns:wps="http://www.opengis.net/wps/1.0.0"
                    xmlns:xlink="http://www.w3.org/1999/xlink"
                    xml:lang="en"
                    service="WPS"
                    serviceInstance="https://gs-stable.geo-solutions.it/geoserver/ows?authkey=41db6295-c6df-4384-ad5b-edea0f4acf5b&amp;"
                    version="1.0.0">
    <wps:Process wps:processVersion="1.0.0">
        <ows:Identifier>gs:DownloadEstimator</ows:Identifier>
        <ows:Title>Estimator Process</ows:Title>
        <ows:Abstract>Checks if the input file does not exceed the limits</ows:Abstract>
    </wps:Process>
    <wps:Status creationTime="2020-11-26T08:38:08.416Z">
        <wps:ProcessFailed>
            <ows:ExceptionReport version="1.1.0">
                <ows:Exception exceptionCode="NoApplicableCode">
                    <ows:ExceptionText>Process failed during execution
                        java.lang.IllegalArgumentException: Illegal argument: &amp;quot;type=null&amp;quot;.
                        Illegal argument: &amp;quot;type=null&amp;quot;.</ows:ExceptionText>
                </ows:Exception>
            </ows:ExceptionReport>
        </wps:ProcessFailed>
    </wps:Status>
</wps:ExecuteResponse>

Using the layer gs:ETOPO_Shadow everything seems to work fine (with the exception of the problem reported here).

[nmco]

@taba90 let's dot he usual work-flow:

• initial investigation and estimate, max 2 hours, if we need more let's ask the client
• try to reproduce the issue, check that we have everything we need

The project and project code are:

ALMAVIVA	C183-ALMAVIVA-2020-GeoNode-DEV

[taba90]

@nmco as reported from Jose in the linked issue it seems a problem with the CoverageInfo definition of the nurc:mosaic layer.

• I've replicated the issue performing the request reported on the nurc:mosaic layer as it is in the geoserver data_dir
• The coverage.xml is missing the dimensionType required by the download estimation which will receive a null object here causing the error reported.
• Reloading the featureType from the layer page will cause the missing dimensionType to be computed and fixes the issue.

[nmco]

@jemacchi creates this JIRA ticket, which is very vague.

We have two situations here:

• @tdipisa is testing the downland process using GeoServer release data directory, which has the coverages definitions broke:
  • @taba90 let's do a PR fixing GeoSevrer release data directory, I guess you can use the JIRA @jemacchi created.
• It's not clear to me how the GeoServer data directory end-up being corrupted, but looks like the coverage band details? functionality breaks backwards compatibility?
  • @taba90, @aaime and @dromagnoli any feedback on this?

[aaime]

Not much right now, but a note, the layer in question has been added in the data directory easily 15 years ago, and might have a configuration that is at least odd by today's standards. That said, the process should not be failing because of it... if the size information is not available from the coverage band details, it can be gathered from GridCoverage2DReader.getImageLayout().getSampleModel() (and likely more reliable than information cached in the config).

[nmco]

Thank you for the feedback @aaime, two bits are not clear to me (I don't know much about this APIs):

• So this is only happening with this specific very old layer? In that case, yes let's just update it.
• The information the process is trying to retrieve, the size of the raster? (@taba90 always add your investigation code pointers), can be obtained in two ways:
  • From the bands details, may not be available for old layers or from the image layout, which is more reliable and usually available.

If I got he above right, I would suggests that we update the download the process to get the size form the image layout if bands details are not available, what's your estimate for this @taba90 ?

[aaime]

It happens for that specific very old layer for sure... and probably every other raster layer having a similar age (data dirs keep on getting upgraded and migrated, there are some very old ones out there). It's probably a small percentage of layers out there, but quite likely there are others around.

[taba90]

@nmco as pointed by @aaime the reader.getImageLayout().getSampleModel() allows to retrieve the size information. The doubt I have is regarding mosaic data. In that case the ImageLayout is always null for the underlying RasterManager. It try to build it from the MosaicConfigurationBeanto colorModel and SampleModel, but thos attributes are allways null. The only way to retrieve the missing size information looks to me that would be to invoke the read method of the CoverageReader by I guess it is not a desired option.

[aaime]

@taba90 weird, the sample image on disk shoudl contain also a image layout, right @dromagnoli ?

[dromagnoli]

Yep @aaime . the rasterManager is indeed loading the default ImageLayout from that sample image. However, as you said, that dataset is probably there since 15 years ago when no sample_image was around. the nurc_mosaic directory is indeed missing the sample_image

[aaime]

Another possibility is just to log a warning and don't try to apply size limits to data that does not want to help :-D

[nmco]

Thank you @aaime, @dromagnoli and @taba90 for the feedback.

I would suggests that we fix the GeoServer data directory and then if this situation happen again, let's see what we can do code wise.

[nmco]

@taba90 was the directory fixed? if not let's do it its a quick thing and will avoid nay future misunderstanding.