Support for CityGML 3.0 Dynamizers

[joemureithi]

I would like to inquire about working with CityGML 3.0 datasets that contain Dynamizers. Are they currently supported, or will they be included in a future release? I attempted an import using the examples provided in the official CityGML Encodings using the v0.6.1-beta version; however, the Dynamizers were missing from the import summary and the database. Thank you.

[clausnagel]

Hi @joemureithi, the CityGML 3.0 modules Dynamizer, Pointcloud and Versioning are not on the roadmap for version 1.0 of citydb-tool. Support may be added in a future version.

[thomashkolbe]

Hi @clausnagel, this is unfortunate, because especially dynamizers and versioning are a decisive improvement of CityGML 3.0 for urban digital twins. E.g., Joseph is using Dynamizers for the automated generation of dynamic 3D/4D web visualizations and we intended to use the 3DCityDB exactly to manage this...

Can we have an online meeting to discuss our options on this in mid January?

Best regards, Thomas

[thomashkolbe]

Hi @clausnagel, am I correct, if I assume that "only" the proper mappings in the sub folder citydb-io-citygml are missing? I assume that citygml4j allows to read and write Dynamizers, Versioning, and Pointcloud features?

[clausnagel]

@thomashkolbe, citygml4j has support for all three modules. But for the database, I think more is needed.

Dynamizer: Dynamizers have a <dyn:attributeRef> property encoded as XPath that links the dynamizer to a property. If we just import and export the XPath expression as-is (i.e., as uninterpreted string), adding support should be fairly simple. However, things get more tricky if this XPath shall be automatically resolved to a FK that points to the referenced attribute row in property. In this case, the XPath must be resolved during import and the current importer code must be extended, because the FK can only be set after the referenced attribute has been imported. Secondly, the XPath expressions must be dynamically reconstructed during export, of course, which adds to the complexity.

Although the solution using uninterpreted strings sounds straight-forward, there is one pitfall. XPath expressions are context-sensitive. For example, assume we have a Building which contains sub-features having dynamizers. And also assume that the <dyn:attributeRef> properties of those dynamizers use XPath expressions that use the Building as root node (like in the examples here). Now, if we have imported the Building using uninterpreted strings and simply export it again, everything should be fine. But assume we just want to export a single BuildingRoom of the building. If this contains a dynamizer, its <dyn:attributeRef> would still use the Building as root node but the building is not part of the export and hence the XPath cannot be correctly resolved.

Pointcloud: I see some challenges with point clouds. If we import point clouds as MultiPoint into geometry_data, this could quickly blow up the table. A point cloud can grow very large very quickly. Thus, it could quickly contain more points than the predefined MultiPoint geometry types can even store. So, we would have to use a specialized geometry type for point clouds. But this has not been examined so far and is not reflected in the current schema.

If the point cloud is stored as separate LAS/LAZ file and the CityGML file only contains a reference to this file, then importing this file reference is pointless in case the reference is just local and not some global URL. So, in case of local references, we would need some mechanism to upload and store the point cloud. But again, LAS/LAZ files can become huge. Should we really store them in the database? Or is there a better alternative? Needs more discussion, I guess.

Versioning: Support for the versioning module requires a concept first. For example, I don't think that versioning information should be stored in the current tables but rather requires dedicated tables for versions and transitions. How to automatically populate the tables and ensure that they are always consistent? How do versions impact export operations and how to automatically identify the correct set of features to be exported (e.g., assume a version just adds/removes sub-features)? How do versions impact the current delete operation? Should there be dedicated database operations for creating versions and transactions? How to implement Git-flow like operations on versions such as merging two versions? Is there something like a "main" version to which all features belong that are not explicitly assigned to a specific version? If so, then this "main" version is not reflected in the current developments. I think we shouldn't start with implementing but with working on these questions first.

[benediktschwab]

Concerning the implementation of point clouds: The extension pgPointcloud is another option for storing and querying point clouds on the PostgreSQL database. It provides a dedicated point cloud class named PC_Patch, whereby the schema document is the same as used by PDAL (see also the pgpointcloud reader and writer). Moreover, it provides compression functionalities as well as a list of interoperability functions with the PostGIS extension.

[clausnagel]

Thanks @beneschwab for pointing to this. It's a good example for what I meant with using a "specialized geometry type for point clouds". We would have to examine what it needs to use this extension (changes to the schema, which tools can we use to support it, can this be integrated into the citydb-tool and what if not, can we have an abstraction layer that would work for different database types like PostgreSQL and Oracle, etc.).

[joemureithi]

@thomashkolbe, citygml4j has support for all three modules. But for the database, I think more is needed.

Dynamizer: Dynamizers have a <dyn:attributeRef> property encoded as XPath that links the dynamizer to a property. If we just import and export the XPath expression as-is (i.e., as uninterpreted string), adding support should be fairly simple. However, things get more tricky if this XPath shall be automatically resolved to a FK that points to the referenced attribute row in property. In this case, the XPath must be resolved during import and the current importer code must be extended, because the FK can only be set after the referenced attribute has been imported. Secondly, the XPath expressions must be dynamically reconstructed during export, of course, which adds to the complexity.

Although the solution using uninterpreted strings sounds straight-forward, there is one pitfall. XPath expressions are context-sensitive. For example, assume we have a Building which contains sub-features having dynamizers. And also assume that the <dyn:attributeRef> properties of those dynamizers use XPath expressions that use the Building as root node (like in the examples here). Now, if we have imported the Building using uninterpreted strings and simply export it again, everything should be fine. But assume we just want to export a single BuildingRoom of the building. If this contains a dynamizer, its <dyn:attributeRef> would still use the Building as root node but the building is not part of the export and hence the XPath cannot be correctly resolved.

Hi @clausnagel, thanks for the response and the detailed explanation of the challenges. As you have pointed out, importing and exporting XPath expressions as uninterpreted strings would be limiting. For instance, if one directly adds Dynamizers to an imported dataset in a database, exporting to CityGML would require dynamic generation of the XPath expressions.

[thomashkolbe]

Thanks @beneschwab for pointing to this. It's a good example for what I meant with using a "specialized geometry type for point clouds". We would have to examine what it needs to use this extension (changes to the schema, which tools can we use to support it, can this be integrated into the citydb-tool and what if not, can we have an abstraction layer that would work for different database types like PostgreSQL and Oracle, etc.).

In the first step, we don't need to implement a native pointcloud support in the 3DCityDB. Many people will just make use of the pointcloud capabilities of CityGML 3.0 to link to external files, which do not necessarily need to be read into the database. And if somebody represents the pointcloud using a MultiPoint geometry (because the number of points is not too large), we can simply use the already existing mapping to a MultiPoint geometry in the geometry table. This implementation would already allow to read and export again such CityGML files without losing any information.

[thomashkolbe]

Regarding versioning: yes, the version objects of course should be stored as regular features in the object table. Because in this way, the management of versions and version transitions can be completely realised using a WFS or an OGC Feature API. Reading and maintaining the versioning information is again important, when somebody feeds a dataset with version data into the 3DCityDB and wants to analyse what has changed between the different versions.

The question how to deal with extra database procedures for e.g. managing versions or delete scripts needs to be discussed, though. Nevertheless, it could be a two-step procedure and handling the versioning module in delete scripts is something that can then be developed once the support for storing versions and version transitions is available in the database.

[clausnagel]

I wouldn't immediately agree that we should use the same tables for versioning information. But maybe you can propose a PR for the required changes?