INSPIRE-MIF / 2017.2

Repository for action 2017.2 on alternative encodings
6 stars 11 forks source link

Addresses GeoJSON examples (different strategies) #48

Closed alexanderkotsev closed 5 years ago

alexanderkotsev commented 5 years ago

Short description

Our objective was to investigate different strategies for conversion of existing INSPIRE GML to GeoJSON with the idea to test the usability of the end product with popular GIS tools (desktop and web clients). We discovered 'Addresses' data through the INSPIRE geoportal and converted it to GeoJSON.

Example instance

Example instances are available here.

Sources

Spanish dataset served through a WFS, discovered through the INSPIRE Geoportal.

Underlying (INSPIRE) conceptual model

Source data validates against the 'Addresses' schema.

Purpose & use

The GeoJSON conversion (with different approaches, see below) was developed ad hoc to evaluate the usability of the data set in different tools, and compare it with the original GML.

Specific technical problems investigated:

Used simplification rules

We tried different approaches, which resulted in several GeoJSON files:

Additional information

Our source data was projected in EPSG:25830.

Issues with the encoding

qgis_attributes

Properties of the feature will in such case be shared by all geometries, i.e. we would need to come up with a means to identify which properties describe which geometry. Note: we did not observe such an issue with our test case, but indicate it here for discussion.

Note that QGIS handles (reads/writes) the crs member and data can be reprojected, while this is not the case for the web application http://geojson.io (which only assumes CRS84 and thus does not correctly read and display the data).

  "crs":{  
      "type":"name",
      "properties":{  
         "name":"urn:ogc:def:crs:EPSG::25830"
      }
   }

In addition, the inclusion of the CRS member does not go against RFC 7946, as the GeoJSON specification states that:

... where all involved parties have a prior arrangement, alternative coordinate reference systems can be used without risk of data being misinterpreted.

If we want to go that way, we should figure out a way to come up with an arrangement on the use of CRS.

What do you consider to be the best aspects/characteristics of this example?

The flattened examples are usable (out-of-the-box) within different tools. We tested with QGIS and http://geojson.io.

To summarise, from a usability point of view, the most user-friendly example is Result 4 (or Result 5, which is a further simplification of Result 4). Additional work should be done in order to decide if and how to remove the voidable or unnecessary attributes, and come up with better names of keys for a flat structure.

nmtoken commented 5 years ago

The GeoJSON conversion (with different approaches, see below) was developed ad hoc to evaluate the usability of the data set in different tools, and compare it with the original GML. Specific technical problems investigated: Use CRS different from EPSG:4326

This seems to suggest that the CRS of GeoJSON is EPSG:4326, it's not. The CRS of GeoJSON is CRS:84, the difference is the order of the axes EPSG:4326 is lat/lon, CRS:84 is lon/lat

nmtoken commented 5 years ago

Note example of GeoJSON coming from a WFS (works in http://geojson.io) coordinates definitely in lon/lat order.

{
"type": "FeatureCollection",
"name": "ENERGY_STATUS_2016",
"crs": { "type": "name", "properties": { "name": "urn:ogc:def:crs:OGC:1.3:CRS84" } },
"features": [
{ "type": "Feature", "properties": { "id": 20, "fid": 20, "Site_Name": "Biomass Gasification Gulu", "Site_Name2": "PH Industrial Farm", "River_Name": "N\/A", "Gen_Type": "Biofuel", "Sub_Type_1": "Biogas", "Sub_Type_2": "", "Grid_type": "On-Grid", "Status_1": "At Planning Stage", "Status_2": "", "StatusDesc": "", "StatusERA1": "ERA License", "StatusERA2": "License Effective: 20\/04\/2014; Valid for 21 years", "Year_start": "2100\/12\/31", "Year_end": "", "Const_Time": "", "Plant_Life": "", "No_Units": 0, "Unit_Size": "", "Inst_MW": 0.0, "Pot_MW": 1.0, "Head_m": 0.0, "Flow_ls": 0.0, "Flow_de_ls": 0, "Developer": "PH Industrial Farms Ltd", "Contractor": "", "Operator": "", "Funding": "", "Village": "", "District14": "AMURU", "Latitude": 3.205062, "Longitude": 32.276451, "Notes": "", "References": "ERA Sept 2014" }, "geometry": { "type": "Point", "coordinates": [ 32.276451, 3.205062 ] } },
{ "type": "Feature", "properties": { "id": 52, "fid": 52, "Site_Name": "Palaro-Achwa", "Site_Name2": "", "River_Name": "Awicpalaro", "Gen_Type": "Hydro", "Sub_Type_1": "Mini Hydro", "Sub_Type_2": "0.1-1 MW", "Grid_type": "", "Status_1": "Potential", "Status_2": "", "StatusDesc": "", "StatusERA1": "", "StatusERA2": "", "Year_start": "2100\/12\/31", "Year_end": "", "Const_Time": "", "Plant_Life": "", "No_Units": 0, "Unit_Size": "", "Inst_MW": 0.0, "Pot_MW": 0.5, "Head_m": 0.0, "Flow_ls": 0.0, "Flow_de_ls": 0, "Developer": "", "Contractor": "", "Operator": "", "Funding": "", "Village": "", "District14": "GULU", "Latitude": 3.2706, "Longitude": 32.377728, "Notes": "", "References": "" }, "geometry": { "type": "Point", "coordinates": [ 32.377728, 3.2706 ] } },
{ "type": "Feature", "properties": { "id": 144, "fid": 144, "Site_Name": "Achwa III", "Site_Name2": "", "River_Name": "Achwa", "Gen_Type": "Hydro", "Sub_Type_1": "Small Hydro", "Sub_Type_2": "1-50 MW", "Grid_type": "On-Grid", "Status_1": "Potential", "Status_2": "", "StatusDesc": "Feasibility Studies completed", "StatusERA1": "", "StatusERA2": "", "Year_start": "2100\/12\/31", "Year_end": "", "Const_Time": "", "Plant_Life": "", "No_Units": 0, "Unit_Size": "", "Inst_MW": 0.0, "Pot_MW": 9.0, "Head_m": 0.0, "Flow_ls": 0.0, "Flow_de_ls": 0, "Developer": "Maji Power Limited", "Contractor": "", "Operator": "", "Funding": "", "Village": "", "District14": "PADER", "Latitude": 3.043458, "Longitude": 32.528857, "Notes": "", "References": "Maji Power 2015;" }, "geometry": { "type": "Point", "coordinates": [ 32.528857, 3.043458 ] } },
{ "type": "Feature", "properties": { "id": 141, "fid": 141, "Site_Name": "Agago II", "Site_Name2": "Agago Falls", "River_Name": "Agago", "Gen_Type": "Hydro", "Sub_Type_1": "Small Hydro", "Sub_Type_2": "1-50 MW", "Grid_type": "", "Status_1": "Potential", "Status_2": "", "StatusDesc": "Pre-Feasibility Study completed, can't be developed if Agago I will be developed.", "StatusERA1": "", "StatusERA2": "", "Year_start": "2100\/12\/31", "Year_end": "", "Const_Time": "", "Plant_Life": "", "No_Units": 0, "Unit_Size": "", "Inst_MW": 0.0, "Pot_MW": 4.8, "Head_m": 46.0, "Flow_ls": 10637.0, "Flow_de_ls": 0, "Developer": "", "Contractor": "", "Operator": "", "Funding": "", "Village": "", "District14": "PADER", "Latitude": 2.897067, "Longitude": 32.646332, "Notes": "Can't be developed if Agago I will be developed as water from Agago I will be diverted to river Achwa", "References": "" }, "geometry": { "type": "Point", "coordinates": [ 32.646332, 2.897067 ] } },
{ "type": "Feature", "properties": { "id": 142, "fid": 142, "Site_Name": "Achwa II", "Site_Name2": "", "River_Name": "Achwa", "Gen_Type": "Hydro", "Sub_Type_1": "Small Hydro", "Sub_Type_2": "1-50 MW", "Grid_type": "On-Grid", "Status_1": "At Planning Stage", "Status_2": "Construction is yet to commence", "StatusDesc": "Feasibility Studies completed", "StatusERA1": "ERA License", "StatusERA2": "License Effective: 24\/08\/2014; Valid for 40 years", "Year_start": "2100\/12\/31", "Year_end": "", "Const_Time": "", "Plant_Life": "", "No_Units": 0, "Unit_Size": "", "Inst_MW": 0.0, "Pot_MW": 41.0, "Head_m": 51.65, "Flow_ls": 80918.0, "Flow_de_ls": 0, "Developer": "Maji Power Limited", "Contractor": "", "Operator": "", "Funding": "", "Village": "", "District14": "PADER", "Latitude": 3.146505, "Longitude": 32.517709, "Notes": "", "References": "Maji Power 2015; ERA 2013" }, "geometry": { "type": "Point", "coordinates": [ 32.517709, 3.146505 ] } },
{ "type": "Feature", "properties": { "id": 143, "fid": 143, "Site_Name": "Achwa I", "Site_Name2": "", "River_Name": "Achwa", "Gen_Type": "Hydro", "Sub_Type_1": "Small Hydro", "Sub_Type_2": "1-50 MW", "Grid_type": "On-Grid", "Status_1": "At Planning Stage", "Status_2": "Construction is yet to commence", "StatusDesc": "Feasibility Studies completed", "StatusERA1": "ERA License", "StatusERA2": "License Effective: 24\/08\/2014; Valid for 40 years", "Year_start": "2100\/12\/31", "Year_end": "", "Const_Time": "", "Plant_Life": "", "No_Units": 0, "Unit_Size": "", "Inst_MW": 0.0, "Pot_MW": 42.0, "Head_m": 54.5, "Flow_ls": 78557.0, "Flow_de_ls": 0, "Developer": "Maji Power Limited", "Contractor": "", "Operator": "", "Funding": "", "Village": "", "District14": "PADER", "Latitude": 3.129327, "Longitude": 32.51624, "Notes": "", "References": "Maji Power 2015; ERA 2013" }, "geometry": { "type": "Point", "coordinates": [ 32.51624, 3.129327 ] } },
{ "type": "Feature", "properties": { "id": 145, "fid": 145, "Site_Name": "Achwa IV", "Site_Name2": "", "River_Name": "Achwa", "Gen_Type": "Hydro", "Sub_Type_1": "Small Hydro", "Sub_Type_2": "1-50 MW", "Grid_type": "On-Grid", "Status_1": "Potential", "Status_2": "", "StatusDesc": "Feasibility Studies completed", "StatusERA1": "", "StatusERA2": "", "Year_start": "2100\/12\/31", "Year_end": "", "Const_Time": "", "Plant_Life": "", "No_Units": 0, "Unit_Size": "", "Inst_MW": 0.0, "Pot_MW": 10.0, "Head_m": 0.0, "Flow_ls": 0.0, "Flow_de_ls": 0, "Developer": "Maji Power Limited", "Contractor": "", "Operator": "", "Funding": "", "Village": "", "District14": "PADER", "Latitude": 2.877048, "Longitude": 32.636274, "Notes": "", "References": "Maji Power 2015;" }, "geometry": { "type": "Point", "coordinates": [ 32.636274, 2.877048 ] } },
{ "type": "Feature", "properties": { "id": 146, "fid": 146, "Site_Name": "Agago I", "Site_Name2": "", "River_Name": "Agago", "Gen_Type": "Hydro", "Sub_Type_1": "Small Hydro", "Sub_Type_2": "1-50 MW", "Grid_type": "", "Status_1": "Potential", "Status_2": "", "StatusDesc": "Feasibility Studies completed", "StatusERA1": "ERA Study Permit expired", "StatusERA2": "Permit Granted: 15\/09\/2011; Permit expired: 23\/04\/2014", "Year_start": "2100\/12\/31", "Year_end": "", "Const_Time": "", "Plant_Life": "", "No_Units": 0, "Unit_Size": "", "Inst_MW": 0.0, "Pot_MW": 4.9, "Head_m": 62.0, "Flow_ls": 8056.0, "Flow_de_ls": 0, "Developer": "", "Contractor": "", "Operator": "", "Funding": "", "Village": "", "District14": "PADER", "Latitude": 2.86035, "Longitude": 32.662998, "Notes": "", "References": "ERA 2013" }, "geometry": { "type": "Point", "coordinates": [ 32.662998, 2.86035 ] } }
]
}

Conversely, outputting a file using coordinates in EPSG:4326 lat/long order like below (plots on the map in http://geojson.io, but in the wrong location), data is from Uganda, not Algeria.

{
"type": "FeatureCollection",
"name": "ENERGY_STATUS_2016",
"crs": { "type": "name", "properties": { "name": "urn:ogc:def:crs:EPSG::4326" } },
"features": [
{ "type": "Feature", "properties": { "id": 20, "fid": 20, "Site_Name": "Biomass Gasification Gulu", "Site_Name2": "PH Industrial Farm", "River_Name": "N\/A", "Gen_Type": "Biofuel", "Sub_Type_1": "Biogas", "Sub_Type_2": "", "Grid_type": "On-Grid", "Status_1": "At Planning Stage", "Status_2": "", "StatusDesc": "", "StatusERA1": "ERA License", "StatusERA2": "License Effective: 20\/04\/2014; Valid for 21 years", "Year_start": "2100\/12\/31", "Year_end": "", "Const_Time": "", "Plant_Life": "", "No_Units": 0, "Unit_Size": "", "Inst_MW": 0.0, "Pot_MW": 1.0, "Head_m": 0.0, "Flow_ls": 0.0, "Flow_de_ls": 0, "Developer": "PH Industrial Farms Ltd", "Contractor": "", "Operator": "", "Funding": "", "Village": "", "District14": "AMURU", "Latitude": 3.205062, "Longitude": 32.276451, "Notes": "", "References": "ERA Sept 2014" }, "geometry": { "type": "Point", "coordinates": [ 3.205062,32.276451 ] } },
{ "type": "Feature", "properties": { "id": 52, "fid": 52, "Site_Name": "Palaro-Achwa", "Site_Name2": "", "River_Name": "Awicpalaro", "Gen_Type": "Hydro", "Sub_Type_1": "Mini Hydro", "Sub_Type_2": "0.1-1 MW", "Grid_type": "", "Status_1": "Potential", "Status_2": "", "StatusDesc": "", "StatusERA1": "", "StatusERA2": "", "Year_start": "2100\/12\/31", "Year_end": "", "Const_Time": "", "Plant_Life": "", "No_Units": 0, "Unit_Size": "", "Inst_MW": 0.0, "Pot_MW": 0.5, "Head_m": 0.0, "Flow_ls": 0.0, "Flow_de_ls": 0, "Developer": "", "Contractor": "", "Operator": "", "Funding": "", "Village": "", "District14": "GULU", "Latitude": 3.2706, "Longitude": 32.377728, "Notes": "", "References": "" }, "geometry": { "type": "Point", "coordinates": [ 3.2706,32.377728 ] } },
{ "type": "Feature", "properties": { "id": 144, "fid": 144, "Site_Name": "Achwa III", "Site_Name2": "", "River_Name": "Achwa", "Gen_Type": "Hydro", "Sub_Type_1": "Small Hydro", "Sub_Type_2": "1-50 MW", "Grid_type": "On-Grid", "Status_1": "Potential", "Status_2": "", "StatusDesc": "Feasibility Studies completed", "StatusERA1": "", "StatusERA2": "", "Year_start": "2100\/12\/31", "Year_end": "", "Const_Time": "", "Plant_Life": "", "No_Units": 0, "Unit_Size": "", "Inst_MW": 0.0, "Pot_MW": 9.0, "Head_m": 0.0, "Flow_ls": 0.0, "Flow_de_ls": 0, "Developer": "Maji Power Limited", "Contractor": "", "Operator": "", "Funding": "", "Village": "", "District14": "PADER", "Latitude": 3.043458, "Longitude": 32.528857, "Notes": "", "References": "Maji Power 2015;" }, "geometry": { "type": "Point", "coordinates": [ 3.043458,32.528857 ] } },
{ "type": "Feature", "properties": { "id": 141, "fid": 141, "Site_Name": "Agago II", "Site_Name2": "Agago Falls", "River_Name": "Agago", "Gen_Type": "Hydro", "Sub_Type_1": "Small Hydro", "Sub_Type_2": "1-50 MW", "Grid_type": "", "Status_1": "Potential", "Status_2": "", "StatusDesc": "Pre-Feasibility Study completed, can't be developed if Agago I will be developed.", "StatusERA1": "", "StatusERA2": "", "Year_start": "2100\/12\/31", "Year_end": "", "Const_Time": "", "Plant_Life": "", "No_Units": 0, "Unit_Size": "", "Inst_MW": 0.0, "Pot_MW": 4.8, "Head_m": 46.0, "Flow_ls": 10637.0, "Flow_de_ls": 0, "Developer": "", "Contractor": "", "Operator": "", "Funding": "", "Village": "", "District14": "PADER", "Latitude": 2.897067, "Longitude": 32.646332, "Notes": "Can't be developed if Agago I will be developed as water from Agago I will be diverted to river Achwa", "References": "" }, "geometry": { "type": "Point", "coordinates": [ 2.897067,32.646332] } }
]
}
alexanderkotsev commented 5 years ago

Thanks @nmtoken for indicating this. You are right. A minor detail with a huge impact ;). Probably most of us who have mapped European data have at least once seen them placed around the Horn of Africa.

Note that when writing GeoJSON in QGIS indicating EPSG:4326 as the CRS (what we actually did) what is actually written in the GeoJSON is "urn:ogc:def:crs:OGC:1.3:CRS84". In order to avoid confusion, we will change the overview of the issue.

MarieLambois commented 5 years ago

Short description

I have tried to create different GeoJSON approach based on different use case. I did not take the use case view it in a GIS tool as it has been already addressed by @alexanderkotsev above.

Example instance

Example instances are available here.

Sources

IGNF dataset. Reduced to 1 address rigth now. A larger one is available for testing if required.

Underlying (INSPIRE) conceptual model

Source data validates against the 'Addresses' schema.

Purpose & use

The GeoJSON conversion (with different approaches, see below) was developed automatically to evaluate the conversion tool availability and ad hoc to evaluate the usability of the data set in different use cases.

Uses cases: -Vizualise the address in a webapp -Use the data in a Geolocator

Used simplification rules

Result AddressOnly. Preserves the link structure of the source GML, all address components present in different files. Result AllIncluded: All address componenent are included in a single feature

Issues with the encoding

The original address in GML is 163 lines (all features together).

Encode links:

If components are not included in the main address element, there is a need to be able to encode links properly or to nest features. Though JSON-LD Best practice 11 states

Nest referenced inline objects When multiple related entity descriptions are provided inline, related entities SHOULD be nested.

For bi-directionnal relations it could be linked as well: { "@context": "http://schema.org", "id": "http://www.wikidata.org/entity/Q76", "type": "Person", "name": "Barack Obama", "givenName": "Barack", "familyName": "Obama", "jobTitle": "44th President of the United States", "spouse": { "id": "http://www.wikidata.org/entity/Q13133", "type": "Person", "name": "Michelle Obama", "spouse": "http://www.wikidata.org/entity/Q76" } }

It would require that features have resolvable URI identifiers.

Encode codelists:

I think it is important to keep the link to the codelists and not just add the value. I decided to include the full URI of the code. Another option would be to use JSON-LD mechanisms: { "@context": ["http://schema.org", { "gender": {"@id": "schema:gender", "@type": "@vocab"} }], "id": "http://www.wikidata.org/entity/Q76", "type": "Person", "name": "Barack Obama", "givenName": "Barack", "familyName": "Obama", "jobTitle": "44th President of the United States", "gender": "Male" }

Label element:

UNGGIM Europe says in Good Practice 11: It may be helpful to provide the address semantics in the form of a simple text string as an additional attribute. Free text addresses are simple, readily accessible and human readable within the local context.

In a Geocoding use case that would help display the result address in a simple string.

What do you consider to be the best aspects/characteristics of this example?

For non geometrical feature, it seems more reasonable to have the All-in-One addresses components.

AD-FR-Sample.zip

cportele commented 5 years ago

We have looked into Address features in an INSPIRE Download service in North-Rhine Westphalia, too. We have set up two WFS 3.0 instances using ldproxy on top of the INSPIRE Download service. These are experimental services. The first one merges the address components into the Address feature.

Here is an Address feature:

{
  "type" : "Feature",
  "id" : "Address_DENW000004886132",
  "geometry" : {
    "type" : "Point",
    "coordinates" : [ 7.146147334378, 50.752576691793 ]
  },
  "properties" : {
    "streetNumber" : "40",
    "beginLifespanVersion" : "2018-10-26T12:46:08.409Z",
    "components" : [ {
      "type" : "AdministrativeUnit",
      "name" : "Bonn",
      "level" : "http://inspire.ec.europa.eu/codelist/AdministrativeHierarchyLevel/6thOrder"
    }, {
      "type" : "Thoroughfare",
      "name" : "Willi-Graf-Ring"
    }, {
      "type" : "AdministrativeUnit",
      "name" : "Bonn",
      "level" : "http://inspire.ec.europa.eu/codelist/AdministrativeHierarchyLevel/4thOrder"
    }, {
      "type" : "AdministrativeUnit",
      "name" : "Nordrhein-Westfalen",
      "level" : "http://inspire.ec.europa.eu/codelist/AdministrativeHierarchyLevel/2ndOrder"
    }, {
      "type" : "AdministrativeUnit",
      "name" : "Köln",
      "level" : "http://inspire.ec.europa.eu/codelist/AdministrativeHierarchyLevel/3rdOrder"
    } ]
  }
}

The following screenshot displays how this address looks in the HTML representation provided by the WFS 3.0 API.

screenshot 2018-12-17 at 13 28 06

The second WFS 3.0 instance uses the features as modelled in the INSPIRE application schema.

An Address feature:

{
  "type" : "Feature",
  "id" : "Address_DENW000004886132",
  "geometry" : {
    "type" : "Point",
    "coordinates" : [ 7.146147334378, 50.752576691793 ]
  },
  "properties" : {
    "streetNumber" : "40",
    "beginLifespanVersion" : "2018-10-26T12:46:08.409Z",
    "component" : [ 
      "http://example.com/ad-links/collections/adminunitname/items/AdminUnitName_05314000",
      "http://example.com/ad-links/collections/thoroughfarename/items/ThoroughfareName_0531400003690",
      "http://example.com/ad-links/collections/adminunitname/items/AdminUnitName_05314", 
      "http://example.com/ad-links/collections/adminunitname/items/AdminUnitName_05", 
      "http://example.com/ad-links/collections/adminunitname/items/AdminUnitName_053" 
    ]
  }
}

A ThoroughfareName feature:

{
  "type" : "Feature",
  "id" : "ThoroughfareName_0531400003690",
  "geometry" : null,
  "properties" : {
    "inspireId" : {
      "localId" : "ThoroughfareName_0531400003690",
      "namespace" : "https://registry.gdi-de.de/id/de.nw.inspire.ad.alkis"
    },
    "beginLifespanVersion" : "2018-10-26T12:46:08.409Z",
    "name" : {
      "name" : {
        "language" : "deu",
        "sourceOfName" : "DLKM",
        "spelling" : {
          "text" : "Willi-Graf-Ring",
          "script" : "Latn"
        }
      }
    }
  }
}

A AdminUnitName feature:

{
  "type" : "Feature",
  "id" : "AdminUnitName_05314",
  "geometry" : null,
  "properties" : {
    "inspireId" : {
      "localId" : "AdminUnitName_S_05314",
      "namespace" : "https://registry.gdi-de.de/id/de.nw.inspire.ad.alkis"
    },
    "alternativeIdentifier" : "05314",
    "beginLifespanVersion" : "2018-10-26T12:46:08.409Z",
    "name" : {
      "language" : "deu",
      "sourceOfName" : "DLKM",
      "spelling" : {
        "text" : "Bonn",
        "script" : "Latn"
      }
    }
  }
}