Render layers as groups

[nyalldawson]

QGIS Enhancement: Render layers as groups

Date 2021/10/07

Author Nyall Dawson @nyalldawson

Contact nyall.dawson@gmail.com

maintainer nyall.dawson@gmail.com

Version QGIS 3.24

Summary

While QGIS supports grouping layers within the layer tree as a means of structuring projects, these groups have no impact on how the component layers are rendered. This proposal concerns adding an option for layer groups to "Render as group", which would cause all component layers to be rendered as a single flattened object during map renders.

Rendering as a flattened group opens new possibilities for map styling, including:

• Opacity can be set at a group level. Features from child layers which are obscured by other group children would remain obscured, and the opacity would apply to the "whole of group" only. The image below demonstrates the difference, with the above image showing two layers being rendered at 50% opacity (underlying features are visible, but semi-masked by the 50% red feature on top). The second image shows the result of rendering the layer as a group, where parts of the blue underlying child layer is completely obscured by the red layer on top and THEN the result is rendered at 50% opacity. (taken from https://github.com/mapbox/mapbox-gl-js/issues/11054 -- but noting that this does not imply QGIS taking a stand either way on the meta issue from that ticket :rofl: ).

Here's another image demonstrating the resultant group render of the QGIS test data using a 50% transparent group layer:

• Blend modes (like multiply, overlay, etc) can be applied on a group level. Just like opacity, setting a blend mode for an entire group would cause the contained layers to first be composited before rendering the flattened result using the desired blending mode. Using this approach would result in child layers obscuring each other completely, before rendering the result using a blend mode which reveals and modifies layers sitting below the group layer.

• The scope of blend modes for CHILD layers from a group will be restricted to only affecting OTHER CHILD LAYERS from that group, and not other layers sitting below the whole group. E.g. a multiply mode applied to the top layer in a group would only multiply blend that layer with the other child layers from the group, before compositing the whole result WITHOUT the blend mode on top of other underlying map layers. While interesting in itself, this would also allow us to expose additional blending modes which are supported by Qt but not currently exposed in QGIS. Specifically, the blending modes which mask layers are not available in current QGIS versions because these modes affect ALL content rendered below the the layer without exceptions, including even things like the map background color! By offering a means to limit the scope of blending modes so that they only apply to other group children, these masking modes suddenly have practical use! Some examples:

Using a "destination out" blending mode for the airplane layer on the top of a group causes the content from the rest of the group to be "cut out" (or "masked") where the airplane point symbols are: (to be precise the opacity of the airplane layer is inverted and then applied to the content from the underlying layers):

And here's the same group when the blend mode for the airplane layer is set to "destination in" (the opacity of the airplane layer is applied directly to the underlying layers, i.e. their content is clipped to the airplane shapes):

Note that this works regardless of the layer type. Here's an example where "destination in" blend mode is applied to the airplane child layer from a group which consists of the airplane layer and a xyz basemap, with the group layer drawn on top of a standard raster layer (the aerial image):

Or let's get a bit more creative! Here's a group consisting of two child layers of the same polygon feature source. The bottom layer is drawn using a normal line pattern fill, and then the top child layer is drawn using a shapeburst fill which transitions from opaque at the polygon exteriors to transparent inside the polygon. I then set the top layer (the shapeburst) to the "destination in" blend mode, so that ONLY the opacity from the shapeburst is transferred to the line pattern fill. As a result the line pattern fill fades smoothly from the exterior to the interior of the polygon.

Here's a similar approach used to mask out a linear gradient fill from top left of the polygon to bottom right:

Effects like this are totally impossible to achieve in current QGIS versions.

• Layer effects could be applied to a group, instead of individual layers. Just like opacity and blend modes applied to a group, this would cause the layer effect to only be applied to a flattened render of the child layers. So e.g. a drop shadow effect applied to the group would not be visible for obscured child layers.

This has previously been discussed here: http://lists.osgeo.org/pipermail/qgis-developer/2014-March/031884.html

Proposed Solution

QgsGroupLayer

A new QgsMapLayer subclass for grouped layers would be created, called QgsGroupLayer. This subclass will implement all the required pure virtual methods from QgsMapLayer, and contain in addition methods for setting and retrieving the group's child layers:

/**
 * Sets the child \a layers contained by the group.
 *
 * This method does not take ownership of the layers, but rather assigns them to the group. Layers should be already added to
 * the parent QgsProject wherever appropriate.
 *
 * \see childLayers()
*/
void setChildLayers( const QList< QgsMapLayer * > &layers );

/**
 * Returns the child layers contained by the group.
 *
 * \see setChildLayers()
 */
QList< QgsMapLayer * > childLayers();

As noted above, setting the child layers for a group does NOT transfer ownership of these layers. Rather, ownership is retained by the parent QgsProject or other data structure.

Internally, QgsGroupLayer stores the list of child layers as a QList< QgsMapLayerRef > list of weak pointers. (i.e. they will be automatically nulled if a layer is deleted). This list will be written to xml as a list of the layer ID strings. When a layer is restored the list will be re-populated using the stored layer IDs, and the QgsMapLayerRef objects resolved to matching map layers in an override of QgsMapLayer::resolveReferences. E.g.:

void QgsGroupLayer::resolveReferences( QgsProject *project )
{
  QgsMapLayer::resolveReferences( project );
  for ( int i = 0; i < mChildren.size(); ++i )
  {
    mChildren[i].resolve( project );

    if ( mChildren[i].layer )
    {
      connect( mChildren[i].layer, &QgsMapLayer::repaintRequested, this, &QgsMapLayer::triggerRepaint, Qt::UniqueConnection );
    }
  }
}

QgsGroupLayerRenderer

A new QgsMapLayerRenderer subclass for QgsGroupLayerRenderer will be created. When a QgsGroupLayerRenderer is constructed, the following preparation steps will occur (on the main thread) for each child layer in the group:

• The extent of the render context operation will be transformed to the child layer's CRS, and temporarily assigned to the QgsGroupLayerRenderer's render context
• A layer renderer will be created for the child layer by calling childLayer->createMapRenderer( context ). These child layer renderers will be stored in a std::vector< std::unique_ptr< QgsMapLayerRenderer > > member for the group layer renderer.
• The child layer composition mode and opacity will also be stored in container members for the group layer renderer

When the QgsGroupLayerRenderer is asked to render() (e.g. on a background thread), the following steps will occur:

• If the group layer itself has a paint effect, then QgsPaintEffect::begin will be called on this effect
• For each stored QgsMapLayerRenderer (corresponding to each group child):
  • the transformed child layer extent will be set temporarily on the render context
  • Where required, a temporary QImage will be constructed for rendering the child layer onto (e.g. required if the child layer itself has a layer-wide opacity set), and the render context will be assigned to paint onto this temporary image
  • QgsMapLayerRenderer::render will be called for the child layer renderer to do the actual child layer rendering
  • If a temporary image was used, this will be drawn onto destination using the desired child layer opacity and blending mode
• If the group layer has a paint effect, then this effect will be finalised

When rendering a map which contains group layers, ONLY the group layer should be added to the QgsMapSettings layers -- it is not necessary to also add the group children (as this would cause them to be drawn twice. Once as part of the group, and once as an individual layer!).

Impact on multi-threaded rendering

As described above, group children will always be rendered sequentially in order to correctly apply interactions between the child layer's contents. This could potentially negatively affect the rendering speed of group layers containing many children (when compared to rendering these children as individual map layers). However, the group-based rendering will be completely optional and non-default, so any performance impact will be entirely optional.

Initially, rendering of group children will NOT utilise previous cached renders of layers. This is a potential optimisation for future development however, as it may be possible in certain circumstances to utilise a previously cached child layer render when rendering a group. (Noting that the existing caching logic will still apply to the WHOLE group itself -- ie. if NONE of the group's children require a redraw then the whole group layer will just be composited from a cached copy and not re-rendered).

Interaction with layer tree and layer order panel

By default, groups created in the layer tree will remain as structural only groups and will not affect rendering operations. A user must right click a group, and from a new "group properties" dialog opt in to "Render layers as a group". (Users will also be able to set group level properties such as the group opacity, blend mode and paint effect from this dialog).

As soon as a layer tree group is set to "render layers as a group", a corresponding QgsGroupLayer will be constructed and added to the project. These are NOT user visible nor will be shown as new entities in the layer tree. Logic will be added so that the layers set for map canvases will include the group layers (and not their individual children).

When a group is set to "render layers as a group", then ONLY the group will be shown in the "layer order" panel list. Group children will NOT be visible in this order list, as their ordering is determined by the placement of the group layer.

Additional composition modes

Logic will be added so that the "masking" composition modes which are currently not exposed in QGIS will be available ONLY for layers which are children of groups. Specifically, the follow modes will added:

• SourceIn: "The output is the source, where the alpha is reduced by that of the destination."
• DestinationIn: "The output is the destination, where the alpha is reduced by that of the source. This mode is the inverse of CompositionMode_SourceIn."
• SourceOut: "The output is the source, where the alpha is reduced by the inverse of destination."
• DestinationOut: "The output is the destination, where the alpha is reduced by the inverse of the source. This mode is the inverse of CompositionMode_SourceOut."
• SourceAtop: "The source pixel is blended on top of the destination, with the alpha of the source pixel reduced by the alpha of the destination pixel."
• DestinationAtop: "The destination pixel is blended on top of the source, with the alpha of the destination pixel is reduced by the alpha of the destination pixel. This mode is the inverse of CompositionMode_SourceAtop."
• Xor: "The source, whose alpha is reduced with the inverse of the destination alpha, is merged with the destination, whose alpha is reduced by the inverse of the source alpha. CompositionMode_Xor is not the same as the bitwise Xor."

See the following image from the Qt docs for a visual demonstration of these modes (not that some modes will remain un-exposed, e.g. "clear", which has little discernible use!!, and source over/destination over which would instead be achieved through rearranging the order of map layers)

Affected Files

New classes for QgsGroupLayer and QgsGroupLayerRenderer will be added. Logic will be added to the layer tree to handle the creation/destruction of QgsGroupLayers, and new UI classes for group layer properties will be created.

Performance Implications

Noted above

Further Considerations/Improvements

(optional)

Backwards Compatibility

(required)

Issue Tracking ID(s)

https://github.com/qgis/QGIS/issues/24860 https://github.com/qgis/QGIS/issues/19648

Votes

(required)

[roya0045]

I might have overlooked this point, but what will happen in the legend. Will it be a single symbol for the group or will items still be displayed individually? I guess the first option will be hard to implement and the second will cause some confusion.

[esnyder-rve]

I would think that this would have no impact on the legend, and that the legend will still render normally as individual items. In the case where a layer is masking out another (the airplane being used to hide an aerial to show osm underneath, the legend item may have to be manually removed from the legend unless there's a relatively easy/efficient way to 100% know that it shouldn't be in the legend.

[nyalldawson]

Will it be a single symbol for the group or will items still be displayed individually?

Displayed individually.

[3nids]

Small question regarding identifying features: will it continue to work per layer or will it need to be implemented?

[3nids]

sorry to come late.

the naming of QgsGroupLayer vs QgsLayerTreeGroup makes it a bit difficult in the layer tree part of the code. Have you thought about something less generic like QgsRenderingLayerGroup or it's meant to be generic ?

[nyalldawson]

@3nids I'd be happy with QgsGroupRenderingLayer, if you want to do the rename

[3nids]

ok, let's do that when you have your work merged.