Implement SEP transformation to NGSI-LD

[FranckCo]

Implement the transformation task that formats the SEP Census QB DSD and CSV file as NGSI-LD

[FranckCo]

Specification of RDF format for loading in the Context Broker expected at the end of November 2021

[flopezag]

A EBNF LALR(1) grammar has been created. The python parser code is finished.

We are working in the creation of the corresponding Data Model related to DCAT-AP and statDCAT-AP in order to create the more appropriate data to be reused in the project.

There are on the other side some missing information regarding the definition of the Dataset in SDMX that can be included in the corresponding DCAT-AP Catalogue information.

BTW, there are some inconsistencies in the SDMX content about SEP that need to be reviewed.

[flopezag]

The parser is finished, some issues found in the RDF file, almost closed the service description

[flopezag]

The IoTAgent is finished and check against the Orion-LD instance the file contain in the /example folder: sep-dsd-1.ttl.

Considerations

• Implemented EBNF LALR(1) grammar
• Creation of an application to allow run locally or through a server
• Minor error control
• Parse the content into an extension of statDCAT-AP
• Generation of the NGSI-LD payload based on statDCAT-AP v1.0.0 plus extension to cover Dimensions, Atrtibutes, CodeList, and ConceptSchema.
• Generated incubated Data Model statDACT-AP

Pending

• We are not generating the content regarding the different values of the skis:Concept and range of values (although they are parser in the agent).
• If there is only a language, by default we assign the language English (en)
• We have detected some missing content in the file (e.g. obsValue, unitMeassure)
• Documentation need to be developed.

Testing

• Locally through the example sep-dsd1.ttl example and through the server sending the data to Orion-LD instance.
• There is a docker-compose.yml file to facilitate the creation of an Orion-LD instance for testing.
• Additionally, we have created a Postman Collection to import into Postman and check the operations against the IoTAgent and Orion-LD

[romaintailhurat]

• [ ] Translate the DSD (outside the pipeline)
• [ ] Transfom the data (step of the pipeline)
• [ ] Push to a broker (both metadata and data)