search

simphony / simphony-osp

A framework that aims to achieve interoperability between software such as simulation engines, databases and data repositories using a knowledge graph as the common language.
https://simphony.readthedocs.io
Other
16 stars 12 forks source link

Data source parser/reader #620

Open yoavnash opened 3 years ago

yoavnash commented 3 years ago

So far we had wrappers for simulations and for data sink (to store data). We still don't have a wrapper for extracting data from an existing data repo. Such a wrapper would be beneficial for demos and various applications.

Suggestions:

Could be related to #267 and #290

pablo-de-andres commented 3 years ago

Do we just want to extract data? Because then I would not use a wrapper, since both simulations and data storage can be used for communication in both directions which I think is part of what makes wrappers wrappers

yoavnash commented 3 years ago

Good point. It can still be beneficial though to have a "read-only" wrapper, for direct access to data in an OSP-core script.

pablo-de-andres commented 3 years ago

I think would define another component at the same level as wrappers. A parser/reader, if you will. And we could divide between data repos, excel files and other formats...

yoavnash commented 3 years ago

That sounds indeed better. Great suggestion.

pablo-de-andres commented 3 years ago

It might make for a nice symmetric design if we think of readers, writers and wrappers

MBueschelberger commented 3 years ago

It might make for a nice symmetric design if we think of readers, writers and wrappers

Definitely agree.

In case of the GMSHWrapper, I followed a separation between engine, (which takes the role of a reader and writer) and the actual wrapper by two different classes.

According to my experience with this module from the FORCE-project, if there is a syntactic third-party-tool already available, the reader and writer part might be quite easy. The semantic interpretation through a wrapper and the related ontology is much more timeconsuming, as we all now...

yoavnash commented 3 years ago

Could be related to mapping data from tables to RDF. Related technologies:

kysrpex commented 3 years ago

I am labelling this also as breaking change as we might want to introduce a new API or do changes to the wrapper API for this.

kysrpex commented 3 years ago

Related #651.