dropped pixels in pgraster

[knutole]

Getting quite a lot of dropped pixels with pgraster RasterColorizer. Cause not known. Suspecting it's a bug in either RasterColorizer or pgraster, as different pixels are dropped on different zoom-levels.

Different zoom-level (same area):

The cartocss:

    // raster debug
        var defaultCartocss = '#layer {'
        defaultCartocss += 'raster-opacity: 0.8; '; 
        defaultCartocss += 'raster-scaling: gaussian; '; 
        defaultCartocss += 'raster-colorizer-default-mode: linear; '; 
        defaultCartocss += 'raster-colorizer-default-color: transparent; '; 
        defaultCartocss += 'raster-colorizer-epsilon: 0; '; 
        defaultCartocss += 'raster-colorizer-stops: '; 
        defaultCartocss += '  stop(20, rgba(0,0,0,0)) '; 
        defaultCartocss += '  stop(50, #dddddd) '; 
        defaultCartocss += '  stop(100, #cccccc) '; 
        defaultCartocss += '  stop(150, #0085ff) '; 
        defaultCartocss += '  stop(200, #0078ff) '; 
        defaultCartocss += '  stop(240, #0078ff); '; 
        // defaultCartocss += '  stop(241, red, exact); '; 
        defaultCartocss += 'raster-comp-op: color-dodge;';
        defaultCartocss += ' }';

<!DOCTYPE Map[]>
<Map>

<Style name="layer" filter-mode="first">
  <Rule>
    <RasterSymbolizer opacity="0.8" scaling="gaussian" default-mode="linear" default-color="rgba(0, 0, 0, 0)" epsilon="0" comp-op="color-dodge" >
    <stop value="20" color="rgba(0, 0, 0, 0)" />
    <stop value="50" color="#dddddd" />
    <stop value="100" color="#cccccc" />
    <stop value="150" color="#0085ff" />
    <stop value="200" color="#0078ff" />
    <stop value="240" color="#0078ff" /></RasterSymbolizer>
  </Rule>
</Style>
<Layer name="layer"
  srs="+proj=merc +a=6378137 +b=6378137 +lat_ts=0.0 +lon_0=0.0 +x_0=0.0 +y_0=0 +k=1.0 +units=m +nadgrids=@null +wktext +no_defs +over">
    <StyleName>layer</StyleName>  </Layer>

</Map>

[knutole]

Have checked the pixel values, they are fine (ie. they all have normal values between 20 and 200).

[strk]

pgraster plugin has no knowledge of RasterColorizer, it just provides the raster to the upstream code which decides what to do with it.

The only thing affecting PgRaster fetching of the data is the "band" parameter, which determines which band is being sent to upper levels. In the "snow" case there's a single band so it should always be used for band=0 or band=1

[strk]

See if removing the "prescale-rasters" parameter changes anything

[strk]

See TODO block here: https://github.com/mapnik/mapnik/blob/v3.0.10/plugins/input/pgraster/pgraster_datasource.cpp#L938-L946

[knutole]

Dropped pixels persists even without gdalwarp on import, and without pre-tiling on import (ie. raster2pgsql -I -C $1 $2 | PGPASSWORD=$PGPASSWORD psql --host=$PGHOST --username=$PGUSERNAME $3 )

Problem persists:

[knutole]

For reference, prescale_rasters and clip_rasters also have no effect. Neither does preunion_rasters.

[knutole]

Curiously, it's not the same pixels that are dropped on different zoom levels. Which suggests that this is not an issue with actual pixel values?

[knutole]

Interesting to note that the NODATA pixels are transparent and with gaussian blur (as expected), but the dropped pixels DO NOT have gaussian blur. In other words, they don't seem to be rendered as a pixel value per se at all.

Compare with gaussian off:

[knutole]

Problem only arises with use of RasterColorizer/RasterSymbolizer:

(Ie. the only code-change is removing postgis_settings.band = 1)

[strk]

So a probable reason for this is a bug in pgraster handling of "nodata" for data rasters (when band=x is passed and x!=0). The temptative fix is here: https://github.com/mapnik/mapnik/pull/3382

The PR doesn't have an automated test as there's no core automated test for raster, unfortunately.

The fun behavior would be due to the NODATA value taking the value of the latest pixel being read for each input raster, which for tiled rasters would be different for every tile and for prescaled rasters would be different for every zoom level.

[knutole]

This was fixed by @strk in https://github.com/mapnik/mapnik/pull/3382.

Currently master on Mapnik won't build on Ubuntu 14.04 (see https://github.com/mapnik/mapnik/issues/3384), so we're using "systemapic" branch of knutole/mapnik -> https://github.com/knutole/mapnik/commits/systemapic. (Should try to build mapnik:master as soon as build issues are resolved, see https://github.com/systemapic/docker-systemapic/issues/67.)

Need to install node-mapnik with npm install --build-from-source mapnik.