Create a mappings plugin

[TBoonX]

For GUI see https://github.com/stream-project/WP3R/issues/6

[simeonackermann]

Decide, if we want a: 1) (CKAN) plugin with basic functionality or 2) a standalone service/app (may use just parts)

Basic Requirements/Functionality

• view relational data schema (tables, columns)
• allow to create/edit mappings (may also import) -> simple text editor or graphical
• visualize mappings
• allow mapping export -> eg R2RML

Existing standalone apps for mapping ontologies:

• Map-On, Web-App, PHP based, Demo (guest@guestuser.com/guestuser) -> SQL only!
• Karma, Web-App, Java based
• SQuaRE, Not public available? Unclear license, Demo
• RMLEditor, unclear license, Demo (Community-Edition)

[simeonackermann]

After some research/testing with existing Tools we came the following conclusions:

• Karma is quite extensive with very good editing mapping tools. But the visualization of mappings with horizontal tables is quite too wide and not handy. Furthermore its RAM/CPU usage for heavy data is very high (> 8GB)
• RMLEditor looks quite good and may fits our requirements. But the unclear license situation may contradicts the usage in the STREAM project. TODO: May ask for RMLEditor team for license?

New solution:

• use (simple) text based human readable mapping language YARRRML with web based GUI similar to Matey. Further advantages: simple implementation, good performance even for huge data, not too sensitive for Drag-And-Drop errors. Disadvantages: no (graph-based) overview/visualization of the mapping

[simeonackermann]

The main idea of the app like this:

[simeonackermann]

This way it should be also possible to integrate the YARRRML editor directly into CKAN

[TBoonX]

This issues is about WP 4.5 thus I add something for the included subtask: See https://github.com/stream-project/WP4R/issues/22

Basic Requirements/Functionality

• automatically creation of facets from mappings (could be a tiny external service)
• a search mechanism which uses the facets to allow users to explore the data (extra search mechanism additional to the CKAN search)

Software for extracting facets:

• semanticFacets could be adopted to use our sparql endpoint

Remark: A semantic search could be realized with RDForm.

[simeonackermann]

First prototype of mapping app Madams Editor.

[simeonackermann]

Updated general idea

[TBoonX]

@All: I did add all related partners to the issue and will now provide the current situation and current tasks:

For WP3.3 we have a plan for the GUI and how to integrate it into LDM. This is highly related to this issue because the topic is similar and the mappings (RML RDF) should be stored and managed via the SPARQL endpoint. Thus there should be an integration of both somewhen.

This issue is about using the SPARQL endpoint and the LDM to do: https://github.com/stream-project/WP4R/issues/21#issuecomment-727850229 An addition is that at the start we could use manually created facets which will be stored in a graph.

Here are the tasks I identified:

• [ ] Define the graph(s) in which RML is stored (collaboration with WP3.3 needed)
• [ ] Create a service which does use sematicFacets (or something similar) to extract facets from RML and store it in the graph on request.
• [ ] Create a CKAN extension which does load the facets and provide a search on it - could use RDForm

The button for the mapping status and the start of the mapping process should be integrated in the facet based search results. (already there on LDM: https://github.com/AKSW/ckanext-dataretrieval )

Here is an old file about facet based search: https://docs.google.com/document/d/1ULMVNJtNTjy3WGCS2zNEix9wdxq7Ki2Y

We now need to decide who will work on what and if this is enough for this WP.

[mospring]

I worked with YARRRML and found that it has some limitations. While it is possible to include a condition, the available conditions are very simple. For example, if A and B should be linked only if A has a certain value, it is not possible. There is also no checking for typos (of the used resources). Otherwise it is simple and works well.

[TBoonX]

@mospring Thanks for your remarks. I would like to add that RML has also just the equal comparison (aka joinCondition) and that we could enhance the YARRRML parser to also have more functions for conditions. Important is that the mapping (RML or YARRRML) should not be used for data validation. We use SHACL in workpackage 2 for this.

Besides the limitation of the mapping languages, do you have remarks on the facet extraction or how to use the facets in the LDM?

[TBoonX]

I close this task because it is outdated - federated search over data is the current plan. For this we have to create the corresponding issues when the plan and out paper is ready.