search

stream-project / WP4R

Work Package 4 (Leitung:Infai)
1 stars 0 forks source link

Create CKAN data retrieval plugin for on-demand requests #18

Closed TBoonX closed 3 years ago

TBoonX commented 3 years ago

in order to realize this task, the APIs from NOMAD and DSMS need to fulfill a few requirements: Please make sure that the API provides a file format which could be used by RMLMapper

We outlined the usage of RMLMapper in https://docs.google.com/document/d/1W91izUVv-d9YCol9BuaXLbB6u3pKPhPoiU331gQ2k2o/edit

TBoonX commented 3 years ago

High work overlapping with https://github.com/stream-project/WP3R/issues/5 The service which calls the data repository, retrieves, maps and saves the data (as RDF) is already there (working for NOMAD). Thus this issue is about connecting it with the LDM.

TBoonX commented 3 years ago

Atm the data retrieval + mapping is started via a button click. Thus have to be restricted. How this should be restricted and if we need a queue have to be discussed.

TBoonX commented 3 years ago

@MatthiasW999 @yoavnash What is the plan about a DSMS API route to retrieve data? Either raw or preprocessed. With the new dcat:Distribution entries the source files are available. Will this be the case for all the datasets you want to provide? Do you have plans to provide preprocessed data?

TBoonX commented 3 years ago

If we will have visualizations inside the LDM the requirements of this have to be taken into account as well.

yoavnash commented 3 years ago

I would say it would be helpful to discuss this at a meeting. I'm a bit unsure of what is currently required.

TBoonX commented 3 years ago

In general there is no task for the IWM to provide data with an API. It seems that the IWM has to decide if they still want spend time on providing data in raw or preprocessed form for the visualization and mapping. Depending on what we will decide about the visualization part, a theoretical solution for the DSMS described in a report could be sufficient. However the IWM needs to create mapping files (YARRRML) for which data is needed: WP 3.1. Thus I recommend that we all discuss the visualization part in Hannover and that you from IWM then discuss and decide on if and what way you want to provide the data.

TBoonX commented 3 years ago

This issue is outdated. TIB, FHI and IWM will realize a federated search over data via (on the fly execution of) mappings. We will create new tasks for it when the plan (and the current paper) is ready.