IATA-Cargo / ONE-Record

This repository contains the documentation & specs for the ONE Record standard.
https://onerecord.iata.org
MIT License
102 stars 52 forks source link

Explore alternate ways to generate and display data model #236

Closed gxclark closed 3 months ago

gxclark commented 5 months ago

While the main modeling activity is using OWL, the primary published data model does not seem to be in a standardized format, such as ... IDEF1X, Martin/ER/IE, UML ... https://en.wikipedia.org/wiki/IDEF1X https://en.wikipedia.org/wiki/Entity%E2%80%93relationship_model https://en.wikipedia.org/wiki/Class_diagram

A while back, ~2020, I "reverse-engineered" ... (mostly manual) the PDF and XLSX files into an ER diagram in Oracle Data Modeler, see image below. This is now out of date, but is there a way to be able to produce this type of improved artifact as part of the standard model development process?

image

NiclasScheiber commented 5 months ago

Very good, I support this. I would also suggest to create a mkdocs page akin to the api https://iata-cargo.github.io/ONE-Record/ to document the concepts and most complex elements in written form.

lambertciata commented 5 months ago

Hi @gxclark thanks for the issue. This is something we always had in mind but also always struggled with because:

  1. Our main objective is to maintain the ontology
  2. The gap between the ontology and any vizualisation (apart from vOWL) can only be manual, and it is becoming quite big.

We have for instance a detailed view of the ontology here: https://onerecord.iata.org/ns/cargo/index-en.html This is automatically updated with the ontology and in my opinion is much more usable than what others can propose (e.g. https://vocabulary.uncefact.org/classes from UNCEFACT).

I am not against maintaining a classical data model view but till today I'm still struggling to find a proper, collaborative, simple as possible tool.

With regards to @NiclasScheiber comment above, let's assess what can be created as documentation indeed.

gxclark commented 5 months ago

Hello @lambertciata @NiclasScheiber , Thanks for your ideas and consideration on this. I've refreshed my searches for a good way to bridge the gap between the relational database world and the semantic web / RDF triple-store / graph world.
I'm looking now at: https://github.com/jimjonesbr/rdf_fdw I'll attempt a prototype with the .TTL file that is published here over the next few weeks, and report back here.

lambertciata commented 5 months ago

Hello!

We are also making progress on our side. We had first identified dbdiagram (https://dbdiagram.io/) and we are focusing on:

  1. Working on translating the TTL into DBML (https://dbml.dbdiagram.io/home)
  2. Using https://github.com/sqlhabit/sql_schema_visualizer as a basis to then display DBML in a way that suits our needs

We will share a MVP in the next couple of weeks to get feedback.

lambertciata commented 4 months ago

Hi,

As promised, the visualizer we've been working on is accessible here: https://aloccid-iata.github.io/ontology_visualizer/

It is an unofficial version that is not yet on IATA's repository and we are open to any feedback to make it better.

There are 2 views: o ONE Record Full Datamodel that is the equivalent view of the previous pdf version. We intentionally limit what is displayed to keep it usable o ONE Record Conceptual Datamodel that is a simpler view with main objects.

gxclark commented 3 months ago

This is fantastic, thanks @lambertciata ! I've had no chance to experiment with RDF. Appreciate your effort to make this more accessible.

gxclark commented 3 months ago

This enables a way to visualize the data model as tables, columns and keys ... it is not quite as useful as a standard relational data model (for example, no definitions), but it is better than the older PDF.

lambertciata commented 3 months ago

Well, we do have the definitions, we added a feature when you hold shift key and go over a property the description is shown :)