Realize timestamp based versioned CityGML files and populate database

EBO as lead

General description

This issue can be divided in two parts:

[x] Populate the 3DCityDB database from a citygml with distinct time stamps (for being able to test the temporal functionnality additions on the server and client). Assignees: VJA
- edit a citygml file to add temporal information (using creationDate and terminationDate)
- import with 3DCItyDB importer
[ ] Create versionned CityGML files (with 2009 and 2012 data of Lyon for example and using change detection process of FPE) and provide means to import them in 3DCityDB when Need10 will be fulfilled.
- Check in 3DUse whether we can write timestamp based versioned CityGML files.
- Make an example with Lyon for 2009, 2012 and 2015.

What must be done:

Create a new executable (named ExtractTemporalBuildings ?) that doesn't depend on QT (consider CityGMLCut as example).
Extract and modify the code of the Change Detection plugin. In particular :
- Convert MainWindow::slotChangeDetection() to make it a CLI main()
  - Modify CompareTiles() so that the considered Building (rooftops) are not reduced to their geometry footprint, but also keep track of the associated building ID
  - The output should be a table of BuildingIds valid along the whole temporal period (say 2099, 2012, 2015) together with their respective computed Year_Of_Construction and Year_Of_Demolition.

Implementation notes:

Input data

We shall consider Lyon first burrough as the territory as given through three snapshots: 2009, 2012 and 2015. The respective datasets can be obtained with

wget https://download.data.grandlyon.com/files/grandlyon/localisation/bati3d/LYON_1ER_2009.zip
wget https://download.data.grandlyon.com/files/grandlyon/localisation/bati3d/LYON_1ER_2012.zip
wget https://download.data.grandlyon.com/files/grandlyon/localisation/bati3d/LYON_1ER_2015.zip
unzip LYON_1ER_2009.zip
unzip LYON_1ER_2012.zip
unzip LYON_1ER_2015.zip

Constraints and design notes (theoretical level)

it happens that the considered datasets contain no time related information (that is the within the CItyGML encoding). For example all the CityGML fields creationDate, terminationDate of the Building objects are empty. Nevertheless those datasets are provided with a vintage acting as (dataset) metadata and stating: this CityGML is the snapshot of the City for year YYYY.
one key aspect of our need is the visual coherence provided to the user when browsing temporal data with the time-slider. For example assume the visual interface offers a time-slider encoded with a yearly time step. Although, strictly speaking, the provided data only caries the information that a considered building existed say at the 2009 and 2012 snapshots, not displaying such a building when the time-slider is positioned at 2010 or 2011 would clearly surprise the end-user. This is because the end-user assumes and expects some historical continuity and the flickering effect of displaying data only at the available time of the snapshots would not produce the visual coherence that historical continuity expects.
Hence we must convert vintage CityGML metadata (a date) into CityGML object duration (time interval). For example is building Build_2 is present in both 2009 and 2012 snapshots we will assume (and state) that Build_2 existed during the 2009 - 2012 period (i.e. with a duration of [2009, 2012].
Alas for the given data we have the following additional technical difficulties
- D1: CityGML object identifiers are not unique along a set of snapshots but only within a given snapshot. For example the very same building Build_2 can appear with a GMLId (CityGML Identifier) of UUID_xx in the 2009 snapshot but UUID_yy in the 2012 snapshot.
- D2: some buildings (CityGML objects) get logically subdivided (as opposed to geometrically subdivided) from one snapshot to the other. For example consider building Build_2 appearing in snapshot 2009 with Geom_2geometry and assume Build_2 is made of two juxtaposed (building wings). Then building Build_2 representation might be subdivided into two (apparently separate) Buildings Build_2_wing_1 and Build_2_wing_2 representation in 2012 snapshot. A direct consequence of the above described D1 and D2 technical difficulties is that neither the GMLId nor the geometry of a building can be used to identify a building. Additionally notice that considering the geometry of a building to be an identifier requires the technical ability to compare two building geometries in order to decide whether they are identical or different (up to some geometrical approximation criteria) .
In order to solve the above difficulties, we shall consider that the pair (Building geometry, GMLId) as a valid building identifier. We shall assume that we have the technical ability to compare two building geometries which enables us to solves the
- D1 difficulty because we can state that when the same (building) geometry Geom_2 appears in two snapshots with different GMLId they are in fact the same building (that is the two triplets (2009, Geom_2, UUID_xx) and (2012, Geom_2, UUID_yy) represent the same building say (Geom_2, UUID_yy) that existed during the period [2009,20012].
  - D2 difficulty because we can now infer through geometrical comparisons that Geom_2 (of 2009 snapshot) is the geometrical sum/union of Build_2_wing_1 and Build_2_wing_2 (of 2012 snapshot). This means that the associated triplets (2009, Geom_2, UUID_xx), (2012, Build_2_wing_1 , UUID_yy) and (2012, Build_2_wing_2 , UUID_zz) are in a subdivision relationship. Technical note: this correspondence relationship is exactly what the "Managing Versions and History Within Semantic 3D City Models for the Next Generation of CityGML" article calls a transaction (refer to figure 4 for a quick glance). In the remaining we shall use this transaction terminology to designate the evolution between to CityGML objects along time.

We can now describe a possible strategy to "introduce/restore" some historical continuity between vintage CityGML metadata (a date) to obtain CityGML object duration (time interval) (in order to achve visual coherence when browsing temporal data). This strategy consists in combining the punctual vintage metadata date information with the geometrical similitude/difference of CityGML objects in order to infer historical continuity. In particular we can now infer transactions out of snapshot data. Such inference can be summarized by the following table

`t1` e.g. 2009	`t2` e.g. 2012	Assertions	Transaction name
`(Geom_a, * ) empty`	`(Geom_a, UUID_a)`		Insert
`(Geom_a, UUID_a)`	`(Geom_a, * ) empty`		Delete
`(Geom_a, UUID_a)`	`(Geom_a, UUID_a)`		Maintain
`(Geom_a, UUID_a)`	`(Geom_a, UUID_b)`	`UUID_a != UUID_b`	Re-ID
`(Geom_a, UUID_a)`	`(Geom_b, UUID_b)` and `(Geom_b, UUID_b)`	`Geom_a = Geom_b + Geom_c`	Subdivide

Note: for the time being we set aside the following cases

`t1` e.g. 2009	`t2` e.g. 2012	Assertions	Transaction name
`(Geom_a, UUID_a)`	`(Geom_b, UUID_b)`	`Geom_b` included in `Geom_a`
`(Geom_a, UUID_a)`	`(Geom_b, UUID_b)` and `(Geom_b, UUID_b)`	`Geom_b + Geom_c` included in `Geom_a`

From time stamps to duration (encoding time uncertainties?)

Assume that we extracted, with the help of the above described geometric comparisons, the following table

BuildingId	2009	2012	2015
Build_1	X
Build_2	X	X
Build_3		X	X
Build_4	X	X	X
Build_5		X

For each building, we now need to convert such a vintage based table into duration encoded at the CityObject level (i.e. CityGML fields creationDate, terminationDate).

Initial design

The purpose for establishing the following set of rules (use case) is for example for having an illustrative visual when browsing the data with the slider.

We apply the following rule for setting the respective dates of the buildings:

When a building is present in the first year of the considered set of years (e.g. 2009) then its Year_Of_Construction shall be -infinity (e.g. Build_1, Build_2 and Build_4).
When a building appears (i.e. it as was not present at the previous timestamps and this appearance is not the first year of the considered set of dates for which the above rules applies) at year YYYY then the buildingYear_Of_Construction shall be 01/01/YYYY (e.g. Build_3 will have a Year_Of_Construction of 01/01/2012.
When a building is present in the last year of the considered set of years (e.g. 2015) then its Year_Of_Demolition shall be +infinity (e.g. Build_3 and Build_4).
When a building does not appear anymore (i.e. it was previously present) at year yyyy and this year is not the date of the set of considered set of years (for which rule 3 appears) then the buildingYear_Of_Demolition shall be 01/01/yyyy.

Motivation for introducing a distinction between the first year of the considered set of years (rule 1) from the following years (rule 2):

we don't want to invent an information
indeed we know that the building is present on this first year but we don't know if it created this year or if it was created before (which we might discover when handled over new data)

If we know apply the above rules to the initial timestamps table we obtain the following table:

BuildingId	Year_Of_Construction	Year_Of_Demolition
Build_1	-infinity	01/01/2012
Build_2	-infinity	01/01/2015
Build_3	01/01/2012	31/12/2015
Build_4	-infinity	31/12/2015
Build_5	-infinity	+infinity

Building Build_5 illustratres a paradoxical (and undersirable?) side effect of rule 1 together with 4 since Build_5 appears within a time interval of [ -infinity, +infinity] (it existed at some point) when we know that it existed in 2012 (and this information is lost).

Furthermore we make the assumption that the same building (a building with the same ID) exists in continuity of time i.e. in the considered example, we can not have

| BuildingId   | 2009 | 2012 | 2015 |
| ----------- |:-----:|:-----:|:-----:|
| Build_6       |    X    |          |     X   |

Second design: Year+ and Year-

Implementation notes

When working on UDV-Server::extract_city_data.py we came out with the following implementation good practice when working with dates (refer to StackOverflow's "how to represent end of time in adatabase and sql and date ranges : dont use null):

When a date in the source database is infinity (respectively -infinity) the associated semantics is to the end of time (respecitvely from the begining of time).
When a date in the source database is null we consider that the associated semantics is "unknown" (or unset) and in turn interpret it as beginning of time (respectively end of time) for year_of_construction (respectively year_of_demolition).
Note: using infinity/-infinity instead of null to carry the semantics of "end of/beginning of time" instead of null not only clarifies the situation but also makes the SQL queries simpler an clearer.
When computing the minimum and maximum dates of the source database (in order to establish practical time boundaries, think for example of setting upper and lower bounds to the temporal slider) we drop (disregard) dates that are either null or +/-infinity. In the created materialized view we substitute to those source values (either null or +/-infinity) the corresponding computed minimum (respectively maximum) to a null or -infinity of a year_of_construction (respectively to a null or infinity of a year_of_demolition). The associated semantics are that before the minimum (known or finite) date means from-ever and after the maximum (know or finite) means forever. Again, if we consider the temporal slider usage of such mapped dates the lower time boundary represents the situation at the origin of time whereas the upper time boundary represents final version of the city as expressed by the city data.

VCityTeam / UD-SV