QEP 63: selective masking

[mhugo]

QGIS Enhancement 63: Symbol clipping

Date 2016/06/06

Author Hugo Mercier (@mhugo)

Contact hugo dot mercier at oslandia dot com

maintainer @mhugo

Version QGIS 3.0

Summary

This QEP is about the need for an advanced labeling and symbology use case where the user wants a special kind of "buffer" around a label or a marker symbol that is not opaque, but "cuts out" some other symbols. This is meant to improve the readibility by avoiding the proximity mix between small elements of the same color.

Example : https://hub.qgis.org/attachments/download/7610

Here the "Wollishofen" label interacts with some underlying layers. In particular, there is no black symbols under the label that is drawn. It could be point or linear symbol layers as well as some lines that are part of a symbol layer (the outline of the red road line for instance).

The functionality must also be available for symbol markers, not only for labels. https://hub.qgis.org/attachments/download/7611

GUI changes

For clipping buffers around labels, a new option will be available, with the same properties already existing for the "regular" buffer. Then the user will choose which symbol layers this buffer around the label should "erase".

Regarding clipping buffers around markers, a new marker symbol layer will be defined with options to define a clipping buffer: its shape (ellipse or rectangle) and the list of symbol layers that have to be "erased" underneath. A later option could allow to define its shape as a buffer around a sub point marker.

Possible implementations

Deferred rendering

The main technical difficulty to face is that QGIS draws the map layer by layer, from the bottom to the top. With this new feature, some elements would have to be clipped by a region that could not be determined at the time of drawing. The rendering process has then to be modified somehow so that two passes are made.

The main idea of this approach is to defer the drawing by stacking all drawing commands (thanks to a painter "proxy") and manipulating the drawing command stack before the actual rendering. Some new drawing commands (in a QgsPainter class derived from QPainter ?) would allow to set a "clipping" region on some other symbol layers. After the first pass where all the regular drawing commands plus the extra clipping commands will be resolved in a second pass so that the stack only contains regular drawing commands. These commands will then be send to the original painting device.

Inspiration: https://majewsky.wordpress.com/2010/08/05/the-color-changing-qpaintdeviceproxy/ Proof of concept in Python : https://gist.github.com/mhugo/fbfa5d15b95787d20f5e8f7863441036 Using QPicture to store painting commands : https://gist.github.com/mhugo/f809970c952f318cb2abc57af1a2446d

Pros:

• High level of control about what layers could be "masked"

Cons:

• Complex implementation
• This technique would certainly cancel any parallel rendering for layers that are "clipped" by a symbol or a label of another layer (since it adds dependencies during the rendering)

  Over painting approach

The idea here is to render the map in a first pass normally, ignoring any "clipping buffer" around labels or markers. Then for each clipping buffer type, the whole map will be rendered again with some symbol levels deactivated. These renderings will be clipped by the clipping buffers and (opaque) painted over the background canvas.

Pros:

• May be easier to design and understand

Cons:

• Need one more rendering for each clipping buffer type (performance issue)

  Symbol layer selection

When defining a clipping buffer (around a label or a marker), only symbol layers beneath the buffer can be cut. It simplifies the ergonomy and avoid cycles in dependencies between symbol layers.

[LKajan]

This would be very welcomed feature!

This was shortly discussed in original blending mode pull request (475). Composition operations destination_out etc. might be used if rendering order of layers and labels could be changed. Did you @nyalldawson have something in your mind back then?

In my opinion a user should be able to set which symbol levels of each layers these halos should mask.

How would this work for exported pdf? These halos should be defined as clipping masks and to be applied to relevant layers.

Here's a Mapnik blog post about "smart halos" using composition operations: https://github.com/mapnik/mapnik.github.com/blob/master/_posts/blog/2012-04-20-smart-halos.md

I think I could arrange some financial support for implementation of this feature.

[mhugo]

@LKajan Hi. Thank you very much for these pointers, it helps ! I remember having a discussion with @nyalldawson when implementing the "inverted polygon fill" about the possibility to use blend modes for that. It is not easy to do within the QGIS rendering process. And moreover QT has limited support for them when exporting to PDF for instance. It forces rasterization, which is not what we want here ... But anyway, it could be worth considering the use of such blend modes again (maybe things have evolved in recent QT5 versions)

[mhugo]

Added a link to a (proof of concept) deferred rendering pipeline in Python: https://gist.github.com/mhugo/fbfa5d15b95787d20f5e8f7863441036

[nyalldawson]

@mhugo have you had a look how QPicture stores paint operations? does it take a similar approach?

[mhugo]

@nyalldawson Thanks, I was not aware of this class. Indeed, this is very similar to what I did with my Python experiment. It seems to me that this may have a performance issue because QPaintEngine operations are serialized and replayed to a QPainter of higher level of abstraction. For text rendering for instance, it seems to mean glyph positions are computed twice.

[mhugo]

Added a simplified version of the proof of concept in Python, based on QPicture: https://gist.github.com/mhugo/f809970c952f318cb2abc57af1a2446d Removed the "pixel based" approach. It seems too hard to have it robust.

But I now realize the deferred approach would need a lot of extra work for anything that calls QPainter::drawImage() or equivalent: shape burst fills, bitmap effects between features and layers, etc. Anything that "flatten" or rasterize painting operations would have to be deferred as well ...

So I think it only lets the "over painting" approach remaining: simpler to implement and understand, but probably slower, since the drawing would have to be done multiple times.

@wonder-sk @nyalldawson I would be happy to have your opinion on that :)

[mhugo]

Hmmm wait, shape burst fills are ok, but the problem remains with effects between features

[rduivenvoorde]

@mhugo cool! Thanks for this QEP. Somebody was asking me for this, and I was pointed from this message on the dev list to this QEP: https://lists.osgeo.org/pipermail/qgis-developer/2016-October/thread.html#44985

FYI: I tried your simplified poc, and does that not miss the point of ONLY hiding the label-text color. In that example it hides ALL colors, while I think the subtality is in the fact that in your examples in the starting post, and the examples below, the buffer is only(!) hiding the label text color. So I think in your poc example:

The black should actually not be clipped, isn't it (because we have yellow as label color).

Below also some examples: only black is clipped, other colors are not touched:

Though here it looks like the clipping is also done on halftransparent pixels of the same color?

Doing such things would probably have to use blending modes isn't it?

[mhugo]

Hi @rduivenvoorde, thanks for your interest !

The idea here is to let the user select what symbol layers have to be "clipped" (or "buffered" ?). If you only want the symbol layer that has the same color as the label text, then you could select it. In my poc code, you can play with what is part of the different symbol layers but moving the line "painter.setRenderingLayer("L1")"

For instance, if you want to simulate the clipping of only the yellow part and not the black part, this can be done :

QT Painter do have some special composition modes (source and destination out) which allow to "punch out" a rendering, but only between a background layer and a foreground layer. It is then not only directly usable for QGIS, unless the rendering process is reorganised in a way that is described in this QEP.

[wonder-sk]

Hi @mhugo coming back to this finally... I would say overpainting is the only option :-) the deferred rendering looks like too much of a hassle, fragile, and probably also much slower.

For overpainting approach, it would be good to sketch out in more details how things would work.

A relatively simple solution could be if the masking would be done on layer level - i.e. for some layers we would keep have a mask (either an image - or a list of shapes), and before the final compositing of layer images to the final map, we would erase layer images according to their mask (or re-render the layer with clipping if we wanted to avoid rasterization). This would mean that symbols composed of multiple symbol layers would need to go to separate layers, but that's probably not a big deal when using such advanced technique (although the separation into multiple layers could be done internally).

[andreasneumann]

Hi @wonder-sk

these variable depth buffers or symbol clipping really need to work more fine-grained. Not just on layers, but on symbol levels. The cartographer needs to be able to pick which symbol levels within a layer have to be clipped/overpainted and which not.

I can't judge on a technical level, if clipping or overpainting works better. Overpainting may have issues with anti-aliasing maybe? Not sure ...

Like Richard, I was also hoping that selective blend modes would help. But apparently no-one yet has found a solution with blend modes.

I would really like to see this enhancement in QGIS 3.x! It would mean a major improvement for the quality of maps!

[mhugo]

Hi @wonder-sk, thanks for your comments.

and probably also much slower.

Instinctively, the "overpainting" technique seems to me slower than the deferred technique, since it involves multiple renderings (one more for each combination of symbol layers that are clipped). Of course, a deferred rendering is slower than a "direct" rendering, but it does not get much slower if you add more symbols to clip.

A relatively simple solution could be if the masking would be done on layer level - i.e. for some layers we would keep have a mask (either an image - or a list of shapes), and before the final compositing of layer images to the final map, we would erase layer images according to their mask

Yes I have something like that in mind. Keeping an image rather than a list of shapes could be an option to have a faster result, but of lower quality (anti aliasing issues), if that makes sense for the end user.

@andreasneumann I think @wonder-sk's remarks do not prevent to define masks on symbol layers :)

[andreasneumann]

BTW: british cartographers (Ordnance survey) call this "selective masking". Perhaps we should rename this QEP accordingly?

[mhugo]

@andreasneumann I like the name and if it comes from native speakers, that should be better :)

@wonder-sk after some more thinking, I think your proposed approach has a limitation on layers with symbol layers: you cannot split them independently into different sub layers and recompose them afterwards (except if symbol levels are enabled, see past discussion here: http://osgeo-org.1560.x6.nabble.com/Symbol-levels-td5135164.html). So it means they cannot be "erased" in the general case, they have to be rendered again with a clipping polygon for each symbol layer.

But I agree the deferred technique is a bit too complex and fragile.

I will update the QEP soon.

[mhugo]

For all followers of this QEP, we are running a co-funding campaign to make it happen. Please find more information on the dedicated page.

[mhugo]

A very first PR is here https://github.com/qgis/QGIS/pull/30747

[Sidapo]

This has been implemented in QGIS 3.12 and I love it! Thank you, it works great.