search

qgis / QGIS-Enhancement-Proposals

QEP's (QGIS Enhancement Proposals) are used in the process of creating and discussing new enhancements for QGIS
118 stars 37 forks source link

QGIS 3D #105

Closed wonder-sk closed 4 years ago

wonder-sk commented 7 years ago

QGIS Enhancement: QGIS 3D

Date 2017/09/26

Author Martin Dobias (@wonder-sk)

Contact wonder dot sk at gmail dot com

maintainer @wonder-sk

Version QGIS 3.0

Summary

This QEP describes the initial implementation of 3D map view in QGIS.

Recommended Reading

Before we dive into details, here are pointers to some resources I have found useful while working on the 3D support:

Learn OpenGL website (https://learnopengl.com/) is a great resource not just to learn OpenGL API, but also to understand concepts behind modern 3D rendering, terminology, math and techniques. No need to spend too much time learning the API itself as we do not use OpenGL API directly in QGIS (all that is abstracted away by Qt 3D framework), but it is useful to get an overview. Understanding the 3D rendering pipeline then helps in understanding why 3D frameworks like Qt 3D or OpenSceneGraph are designed the way they are. Basic knowledge of vertex/fragment shader programming is useful when tinkering with materials.

Next we need to understand basics of Qt 3D framework:

There is of course much more to learn, but that is a minimum to get started. Here is an overview to start with: https://doc.qt.io/qt-5/qt3d-overview.html At this point (Sep 2017) the documentation of classes in Qt 3D is not as detailed as the rest of the Qt documentation, hopefully this will improve over time. There are however various good examples that exercise new APIs: https://doc.qt.io/qt-5/qt3d-examples.html

One more resource I have found particularly useful was the book "3D Engine Design for Virtual Globes" by P. Cozzi and K. Ring: http://www.virtualglobebook.com/ - the book discusses a range of topics and problems one encounters when dealing with 3D display of geographic data, including techniques for terrain generation, out-of-core rendering to support large amount of data and numerical precision issues.

Framework Overview

Let's start by looking at the simplified class diagram... everyone loves fancy diagrams and it is easier to understand how are the main classes related to each other:

class diagram

There are two main groups of classes:

The 3D Map Canvas class (Qgs3DMapCanvas) acts as a glue between Qt 3D, configuration and implementation classes. It creates a surface with 3D view (based on OpenGL) and based on 3D Map Settings (Qgs3DMapSettings) it creates 3D Map Scene (Qgs3DMapScene) that in turn creates and updates 3D entities rendered in the 3D view.

The 3D Map Settings class (Qgs3DMapSettings) is somehow similar to QgsMapSettings used for configuration of 2D map rendering. It encapsulates all scene parameters such as background color, world CRS and origin point, terrain generator, map layers for terrain texture, extra 3D renderers and more.

The 3D Map Scene class (Qgs3DMapScene) is root entity of the 3D scene - all other 3D entities in the scene are its children. Scene class updates contents of the 3D scene according to changes to 3D Map Settings and according to changes of camera's position.

Terrain

There are multiple methods how terrain 3D entity is generated. Because of that we have a terrain generator interface (QgsTerrainGenerator) that is implemented by each such method. As of now we support:

Generated terrain mesh gets textured with map image rendered with usual 2D map rendering. This is handled by QgsTerrainTextureImage and QgsTerrainTextureGenerator classes.

Terrain is organized into a hierarchical tree of terrain tiles. As soon as user moves camera closer to a terrain tile, it will be replaced by terrain tiles with more details (both in terms of mesh quality and map texture). A section below explains in more details how this mechanism works.

3D Renderers for Map Layers

There are two ways how to incorporate data from map layers in a 3D map view. The first one is that a map layer will be rendered using the usual 2D map renderer to an image that will be draped on top of terrain. The second and more interesting option is to assign a 3D renderer to map layers: 3D renderers are first class citizens in 3D map view and thus map layers may be rendered as solid 3D objects or they can be even used to create some special effects or animations.

Each map layer object (QgsMapLayer) can have a 3D renderer (QgsAbstract3DRenderer subclass) assigned. Type and properties of 3D renderers determine how the layer will appear in the 3D scene. Currently we have only support for vector layers: there is QgsVectorLayer3DRenderer implementation that renders all features of a vector layer with given 3D symbol (QgsAbstract3DSymbol subclass). This is pretty much the same as with 2D map rendering: renderer classes decide what features are rendered and symbol classes decide how the features will look like. In the initial stage we have three 3D symbol implementations - one for each geometry type:

The existing symbols all use Phong shading model as their material. The model consists of ambient, diffuse and specular color components that are combined together based on camera and light positions.

Line and polygon symbols support several methods how altitude / elevation is determined for their vertices. Altitude clamping tells how to incorporate terrain elevation, altitude binding decides whether elevation should be assigned from centroid or for each vertex separately.

There is a registry of all available 3D renderers (Qgs3DRendererRegistry). The registry is useful when reading project files in order to be able to pass reading of renderer data to its custom routines.

Compared to 2D renderers, with 3D renderers there is no such virtual function such as render() that would be called for every frame. Instead, 3D renderers have a virtual method createEntity() that is called from the 3D scene object when the scene is created or updated. It is responsibility of the renderer's 3D entity to stay up to date with any data changes or to handle level of detail.

Paged Level of Detail / Chunked Entities

Chunked 3D entities allow QGIS 3D to scale to larger amounts of data. Instead of loading all vertex and texture data to GPU memory and rendering them every frame, we try to be smart. The naive approach quickly leads to exhaustion of GPU memory or very slow rendering performance.

Data are organized in a hierarchical data structure (quadtree). The principle is the same as with raster map tiles in web maps: each map tile has four child map tiles (as if we cut the tile with a knife horizontally and vertically in its center). If a tile does not have enough details for the current view, its four children with more details are shown instead. The structure has a maximum depth assigned, so the recursive splitting of tiles stops at some point.

When rendering a scene with a chunked entity, instead of rendering all chunks that are available, we only render so called active chunks. Whenever camera position is changed, we walk the hierarchy until we find tiles with the right resolution for a given view. This could potentially still be a lot of tiles to render, so we also apply frustum culling based on each chunk's bounding box - this will avoid rendering chunks that are not going to be visible anyway (e.g. they are behind the camera or too far away). After that we get a list of active chunks that will be rendered in the subsequent frames.

In order to keep memory requirements reasonable, we only keep around a small number of chunks. As the camera moves and a new set of active chunks is being selected, chunked entity may request more chunks to be loaded in order to display them later. For that we keep a queue of chunks requested for loading and in a background thread these chunks get loaded. Once a chunk is loaded and ready, it can be used among the active chunks.

To prevent the number of loaded chunks growing rapidly as user moves camera around, we keep also a replacement queue. This is a queue of chunk that are currently loaded and they are sorted according to the time they were last used. Whenever the maximum number of loaded chunks is reached and more chunks need to be loaded, the least recently used chunks get unloaded to free up some memory. They may be loaded again as soon as they are needed in the current view.

Chunk quadtree node (QgsChunkNode) is the key class for the implementation. Each node has a 3D bounding box, geometry/texture error, pointers to parent/children and state. Chunked 3D entity class (QgsChunkedEntity) has root node of the quadtree and drives all the background loading/unloading/updating of chunks. Active chunks are child entities of the chunked entity. Background jobs are executed per one chunk node in subclasses of QgsChunkQueueJob class. Jobs are either chunk loaders (load chunk data into memory and prepare entity for inclusion in 3D scene) and chunk updaters (modify chunks as needed - e.g. update terrain textures when list of map layers changes).

Rendering using chunked entity is currently supported only for terrain (vector layer 3D renderer should support that in the future as well). For example, DEM terrain generator provides chunk loader implementation that reads out values from raster layer, constructs mesh, gets map image rendered and finally combines everything to one terrain tile entity (a chunk).

Source Code Changes

3D support is currently optional and it needs to be explicitly enabled during CMake configuration using WITH_3D=TRUE

The bulk of the implementation is in src/3d directory in the source tree. The 3D support forms a new shared library called qgis_3d which only depends on qgis_core.

3D map canvas widget (Qgs3DMapCanvas) along with configuration widgets are currently not exposed in API - they can be found in src/app/3d directory.

There is a little bit of support code in qgis_core library: interface for 3D renderers and registry of 3D renderers - these need to be in qgis_core library in order to make it possible to load project files or map layer style files with 3D renderers.

The 3D library contains a small support library called poly2tri. It is used for tessellation (conversion of polygons into a set of triangles). The library is licensed under the terms of 3-clause BSD license. The original source code is hosted here: https://github.com/jhasse/poly2tri

Dependencies

This work introduces a new dependency on Qt 3D libraries (Qt3DCore, Qt3DRender, Qt3DInput, Qt3DLogic). These libraries have been included with Qt5 since the release of Qt 5.7, but it has been found that there are some critical bugs in Qt 5.7.x that prevent usage of that version with QGIS 3D. Minimal version is therefore Qt 5.8, however it is recommended to build with Qt 5.9 which adds further improvements and bug fixes, along with long-term maintenance promise from Qt Company.

GUI Changes

There is a new menu item to open 3D map view in a dockable widget (similar to how it is possible to open extra 2D map view): menu View > New 3D Map View. The dockable widget with 3D canvas has a button to open global 3D view configuration for terrain generation and other parameters.

Map layers can get their 3D renderers configured in a new tab in the styling panel.

Camera Control

Map control in the 3D view is meant to follow similar existing applications:

FAQ

Can it replace the main canvas view when enabled?

As of now the 3D map canvas is meant as an additional view for the data - the 2D map canvas will stay as the central widget in QGIS window. In the future we may decide to add a switch between 2D and 3D map canvas for the central area in QGIS.

Can it fetch DEM tiles like globe does?

There is some initial work done towards this (using quantized-mesh-1.0 format), however that is just a proof of concept and at the time of QEP writing it is not enabled. The other issue with fetching of DEM tiles is that we may need to do some extra steps to comply with the licensing terms of the data - e.g. request permission or show attribution when the DEM data are displayed. To be further investigated. From my research there are two possible data sources:

Can it work with CityGML and similar 3D formats?

Yes. For CityGML one may possibly need to do some preprocessing with ogr2ogr, but there are people already testing QGIS 3D with CityGML data. For other 3D formats, it depends on format really. If you have Shapefiles or Geopackages with 3D geometries, these are known to work fine. For file formats with a 3D model of an object (e.g. a building or a tree), these can be used with 3D point renderer of vector layer where each point is rendered with the given model (Qt 3D includes "assimp" library that supports a wide range of common 3D formats like .obj, .stl and others).

Can you print it in composer?

Not right now. This has been deferred to a later time - at the time of writing the QEP there is a new print layout engine in works that will be more extensible than the existing print composer in QGIS 2.x, so it should be possible to support custom print layout items like 3D view.

Can you put the 3D view into full screen?

There is not a dedicated functionality for that now. In theory one can make the 3D map widget floating and then extend it to most of the screen. Probably if a 2D/3D map switch is introduced for the main canvas, we would get that feature for free.

Can we control it like the new multi canvas windows (follow main windows extents etc)?

There are currently fewer ways how to control the 3D map views compared to extra 2D map views. For example following of main canvas extent is not there yet.

Can we animate stuff?

Yes... eventually :-) The 3D map view renders at 60 frames per second (less if the map scene gets too complex or the GPU is simply not good enough), so there is plenty of scope for animations. Right now all the existing 3D entities are static. If the animation only needs to change transform of some entities (i.e. translation/rotation/scaling), that should be fairly easy. It gets more complicated if more elaborate animations are needed. We could do a research where would we want to apply animations but in general there is nothing from the technical perspective that would not allow use to make things animated.

Can we use custom textures for different faces?

Maybe in the future. Currently all faces use a material defined by three color components. For texturing we need two things: 1. texture image (obviously), 2. mapping of texture coordinates of vertices. The assignment of texture coordinates may get a bit tricky as there are possibly many options what users want to do, for example:

Each of these use cases needs to be addressed separately.

Can we make triangular / pyramid roofs?

Yes - if your data has polygons with 3D coordinates that should be possible.

Can we drive the extrusion parameters from data in the vector attribute table?

Yes - for polygon symbols it is possible to set data-defined extrusion height and vertical distance from the ground.

Can we use a library of common symbols (tree, house etc)?

There is no integrated library of 3D symbols now, but there are various libraries of free 3D models on the web - they can be used with 3D point renderer.

Will it replace globe?

Eventually yes. Maybe in QGIS 3.2 when the new 3D framework gets more stable and feature complete and we will be confident that it is a good replacement of the Globe plugin.

Will it replace 2.5D renderer?

No, the 2.5D renderer is still very useful for 2D map rendering - it is complementary to full 3D view.

nyalldawson commented 7 years ago

@wonder-sk

Just out of curiousity - is it planned to expose the 3d library to python too? or is the api considered private here?

(no strong feelings either way, I'm just interested to know your thoughts about this)

timlinux commented 7 years ago

What will we do with globe when 3D lands?

nyalldawson commented 7 years ago

What will we do with globe when 3D lands?

I'd say when this matches the globe feature set it should be removed (3.2?). We don't want to be maintaining two separate 3d frameworks.

timlinux commented 7 years ago

Can you also make a note of how the user interaction will work - remembering the keybindings for tilt etc is not discoverable. Perhaps it needs some quick help overlay to get the user started.

wonder-sk commented 7 years ago

Check out the new FAQ section :-)

giohappy commented 7 years ago

Thx @wonder-sk! Altitude clamping is something I missed the couple of times I used Cesium. Don't if they introduced it in the meantime. I saw it in osgEarth and your description seems to mimic it's approach. Does campling apply to the geometry vertices or do you some tasseletion based on the terrain resolution? In the first case the usual effect is seeing geometries hidden beneath the terrain if the geometry is coarse. I remember I employed fragment shaders to overcome vertex clamping limits, but it was long time ago...

wonder-sk commented 7 years ago

@timlinux Added a small section on how to control the camera

@nyalldawson For now I would prefer to keep the API private until the code gets more stable - at least for one release or two. There are also few missing pieces that prevent people from making their own 3D renderers in Python anyway (e.g. to define config widget for such renderer). And we would need another newe dependency - PyQt3D - seems like it is being distributed separately.

@giohappy Yes the clamping follows osgEarth approach. The alignment with terrain still needs some work... vertices are clamped based on one terrain resolution right now, but this should be adaptive because as soon as you zoom in/out and terrain tiles get more/fewer details, currently you may get some artifacts.

rduivenvoorde commented 7 years ago

@wonder-sk not sure if I'm over-asking, but I was wondering if in future this could also work with non-ground-based geometries: like pointclouds (Lidar?) or 'flying geometries' like I'm working with now: modelled clouds? OR is that just a totally other kind of '3D' ?

Other Question would be: now we have the attributes-based height of geometries, are there any 'true' 3D public datasets available in which every vertex has an actual x,y,z ? Ah, found some public stuff: fgdb: https://www.pdok.nl/nl/producten/pdok-downloads/basisregistratie-topografie/3d-kaart-nl (take Noord-Holland to have Amsterdam) gdb: https://www.arcgis.com/home/webmap/viewer.html?webmap=f10e34635c7742d19737ffc3a2ced119 (pick eg http://services.arcgisonline.nl/3D_kaart_NL/25gn1.zip which has some tall buildings)..

I will try those...

wonder-sk commented 7 years ago

@rduivenvoorde It is a bit broad question, but in general the answer is that you can have 3D objects rendered anywhere. For example you can already create a layer with planes and get them rendered high above the ground.

Lidar data will need extra work to get them supported in QGIS - we will probably need to add PDAL library as a dependency, create a new "point cloud" layer type, add some basic 2D rendering for it and then finally add support for 3D rendering (to get results like from POTree JS library).

For modeled clouds - I assume here you mean semi-transparent raster grids (or a 3D raster grid) that would be rendered in a particular altitude - that should be also possible, but it will again require additional work to make it possible.

For 'true' 3D public datasets, there are some out there, but I have not researched much what is available - typically I come across some datasets when people send them to me because they have some problems :-) Actually it would be very useful to have a curated list of nice datasets to start playing with 3D stuff - any volunteers? ;-) In such a list I think it would be also good to distinguish between 2.5D datasets (where coordinates have just extra Z coordinate, but they are still mostly planar) and 3D datasets (where you can have e.g. vertical polygons for walls).

rduivenvoorde commented 7 years ago

@wonder-sk happy to pull this, and create some list somewhere (wiki? website? trainingmaterial?). Will ask around on the lists. BUT: how do I distinguish these different datasets. Wondering about the buildings gdb above, I see: 3D Multi Polygon in ogrinfo, but in QGIS Geometry | Polygon (MultiPolygon25D)

And copy pasting a 'simple' square building gives me (see below)

Which to me looks like 'True 3D', BUT I only see 2 different Z-values.... How do I distinguish between your different 'types' of 3D?

wkt_geom OBJECTID TOP10_ID HOOGTEKLASSE STATUS HOOGTENIVEAU HOOGTE TYPE_GEBOUW NEDERLANDSE_NAAM FRIESE_NAAM VISUALISATIE_CODE MultiPolygonZ (((122892.75099999830126762 483575.99900000169873238 0.64999999999417923, 122889.67900000140070915 483570.02899999916553497 0.64999999999417923, 122894.28700000047683716 483567.31599999964237213 0.64999999999417923, 122897.44900000095367432 483573.46599999815225601 0.64999999999417923, 122892.75099999830126762 483575.99900000169873238 0.64999999999417923)),((122897.44900000095367432 483573.46599999815225601 0.64999999999417923, 122897.44900000095367432 483573.46599999815225601 5.83000000000174623, 122892.75099999830126762 483575.99900000169873238 5.83000000000174623, 122892.75099999830126762 483575.99900000169873238 0.64999999999417923, 122897.44900000095367432 483573.46599999815225601 0.64999999999417923)),((122894.28700000047683716 483567.31599999964237213 0.64999999999417923, 122894.28700000047683716 483567.31599999964237213 5.83000000000174623, 122897.44900000095367432 483573.46599999815225601 5.83000000000174623, 122897.44900000095367432 483573.46599999815225601 0.64999999999417923, 122894.28700000047683716 483567.31599999964237213 0.64999999999417923)),((122889.67900000140070915 483570.02899999916553497 0.64999999999417923, 122889.67900000140070915 483570.02899999916553497 5.83000000000174623, 122894.28700000047683716 483567.31599999964237213 5.83000000000174623, 122894.28700000047683716 483567.31599999964237213 0.64999999999417923, 122889.67900000140070915 483570.02899999916553497 0.64999999999417923)),((122892.75099999830126762 483575.99900000169873238 0.64999999999417923, 122892.75099999830126762 483575.99900000169873238 5.83000000000174623, 122889.67900000140070915 483570.02899999916553497 5.83000000000174623, 122889.67900000140070915 483570.02899999916553497 0.64999999999417923, 122892.75099999830126762 483575.99900000169873238 0.64999999999417923)),((122892.75099999830126762 483575.99900000169873238 5.83000000000174623, 122897.44900000095367432 483573.46599999815225601 5.83000000000174623, 122894.28700000047683716 483567.31599999964237213 5.83000000000174623, 122889.67900000140070915 483570.02899999916553497 5.83000000000174623, 122892.75099999830126762 483575.99900000169873238 5.83000000000174623))) 2177 102456676 laagbouw in gebruik 0 5.18 overig 13000

wonder-sk commented 7 years ago

@rduivenvoorde great!

I would say the 2.5D / 3D difference can be best spotted in the 3D map view when you enable 3D renderer for the layer - whether the layer is still shown as a plane (and needs to be extruded) or whether it will be shown as 3D objects without extrusion. My categorization 2.5D / 3D is quite rough though and imprecise though and applies mainly to buildings :-) With buildings the main difference is that with true 3D geometries you can have nicer shapes of buildings - e.g. including real roofs:

image

(from Rotterdam dataset - http://rotterdamopendata.nl/dataset/rotterdam-3d-bestanden)

anitagraser commented 7 years ago

"tilt camera by pressing Shift and dragging the mouse with left mouse button pressed" doesn't seem to do anything for me in OSGeo4W dev.

saberraz commented 7 years ago

@anitagraser I think it should be the middle button (the wheel) instead.

anitagraser commented 7 years ago

@saberraz no middle button on my notebook ;-)

wonder-sk commented 7 years ago

@anitagraser best to raise tickets for any bugs you encounter. I have seen this particular issue before - it seems to work only when the 3D view is not docked.

anitagraser commented 7 years ago

@wonder-sk Thanks, will file a ticket: https://issues.qgis.org/issues/17337

nyalldawson commented 6 years ago

@NathanW2 can we mark this done/accepted?

nyalldawson commented 6 years ago

@nathanw2 I just realised I've got permission to change these too :)