[RFC] Correct mipmapping in FillStyleExtension

[felixpalmer]

Target Use Case

The FillStyleExtension hard codes the texture it uses to use LINEAR filtering, which means mipmaps are not used when rendering. When the view is zoomed out, artifacts appear in the form of Moire patterns as the sampling rate is too low to capture the texture. This can be easily seen in the example on the FillStyleExtension doc page.

Changing the texture to use LINEAR_MIPMAP_LINEAR filtering helps and when the maps is zoomed out the pattern fades into a single color, as expected.

However, doing so introduces another artifact which is harder to fix. Because the extension uses a texture atlas to support multiple patterns it cannot use REPEAT texture wrapping, and instead emulates this in the shader using mod() functions. These work fine with the LINEAR filtering, but not with LINEAR_MIPMAP_LINEAR because the incorrect mipmap is selected at the periodic boundary of the mod() function. This happens because the the mipmap level is selected based on the derivative of the texCoords variable and this shoots up every time mod clips it from a value of e.g. 1.0 -> 0.0.

To reproduce

Below is a test app which shows how the code can be patched to allow correct filtering, but only with a single texture. This cropped image should be used:

import {Deck} from '@deck.gl/core';
import {GeoJsonLayer} from '@deck.gl/layers';
import {FillStyleExtension} from '@deck.gl/extensions';
import GL from '@luma.gl/constants';
import {Texture2D} from '@luma.gl/webgl';

// source: Natural Earth http://www.naturalearthdata.com/ via geojson.xyz
const COUNTRIES =
  'https://d2ad6b4ur7yvpq.cloudfront.net/naturalearth-3.3.0/ne_50m_admin_0_scale_rank.geojson'; //eslint-disable-line

const INITIAL_VIEW_STATE = {latitude: 51.47, longitude: 0.45, zoom: 4};

class CustomFillStyleExtension extends FillStyleExtension {
  getShaders(t, extension) {
    const shaders = super.getShaders(t, extension);
    const fillStyle = shaders && shaders.modules.find(m => m.name === 'fill-pattern');
    if (fillStyle) {
      fillStyle.inject['fs:DECKGL_FILTER_COLOR'] = `
    if (fill_patternEnabled) {
      float scale = FILL_UV_SCALE * fill_patternPlacement.z;
      // Modify shader to remove mod() calls which introduce artifacts
      vec2 patternUV = fill_uv / scale;

      vec2 texCoords = fill_patternBounds.xy + fill_patternBounds.zw * patternUV;

      vec4 patternColor = texture2D(fill_patternTexture, texCoords);
      color.a *= patternColor.a;
      if (!fill_patternMask) {
        color.rgb = patternColor.rgb;
      }
    }
    `;
    }
    return shaders;
  }
}

function onLoad() {
  // Cannot access deckRenderer before onLoad
  const {gl} = deck.deckRenderer;
  const texture = new Texture2D(gl, {
    data: 'dots.png',
    parameters: {
      [GL.TEXTURE_MIN_FILTER]: GL.LINEAR_MIPMAP_LINEAR, // Change to LINEAR to see Moire pattern issue
      [GL.TEXTURE_MAG_FILTER]: GL.LINEAR,
      [GL.TEXTURE_WRAP_S]: GL.REPEAT,
      [GL.TEXTURE_WRAP_T]: GL.REPEAT
    }
  });

  deck.setProps({
    layers: [
      new GeoJsonLayer({
        id: 'base-map',
        data: COUNTRIES,
        // Styles
        stroked: true,
        filled: true,
        lineWidthMinPixels: 2,
        opacity: 1,
        getLineColor: [60, 60, 60],
        getFillColor: [255, 0, 0],

        // Fill style
        fillPatternAtlas: texture,
        fillPatternMapping: {
          // Will only work if this corresponds to the entire image
          // In other words, there should only be one image in the atlas
          full: {x: 0, y: 0, width: 120, height: 120, mask: true}
        },
        getFillPattern: f => 'full',
        getFillPatternOffset: [0, 0],
        getFillPatternScale: 100,
        extensions: [new CustomFillStyleExtension({pattern: true})] // Use normal extension to see mipmap artifacts
      })
    ]
  });
}

export const deck = new Deck({
  initialViewState: INITIAL_VIEW_STATE,
  controller: true,
  onLoad
});

Proposal

The rendering of patterns is not usable in its current form, except for a narrow range of zoom levels. To fix it a number of options are available:

• Modify the extension as above so that the correct filtering is used, at the cost of only a single pattern being supported
• Support four patterns by placing them in different color channels of the texture - this would mean that the pattern would always be treated as a transparency mask
• Instead of using a single texture, use a WebGL2 TextureArray which would provide correct wrapping behavior. However this is not supported in luma.gl and (naturally) WebGL1.

My preference is for option 2, as it would cover a lot of use cases, be WebGL1 compatible and isn't too onerous to implement.

[felixpalmer]

https://github.com/visgl/deck.gl/pull/7569 provides an easier way to pass parameters to texture

[Pessimistress]

I personally like the TextureArray approach, though I don't know how it can handle various pattern dimensions. I'm less worried about the WebGL2 requirement since it's supported in most mainstream browsers now.

Using color channels just seems so limited, I'm not sure it's worth the effort.

For the time being, I think we can really benefit from a write-up in the FillStyleExtension documentation, similar to this section. There are several mitigations that we can offer before this issue is addressed in code:

• If only a single pattern is needed, add textureParameters to the layer to use mipmap and wrapping
• If using multiple patterns: create a bleeding buffer around each pattern in the atlas to reduce the border artifact: