Open VladimirAlexiev opened 2 months ago
This issue is important since it will impact the test suite: a great way to create invalid models to test each SHACL, is by using a small difference to a base valid model.
The right (only?) way to represent models is with named graphs.
The non-standard RDF/XML addition parseType=statements
in fact represents a graph, but it has no name and cannot be referenced from any other place.
Models and differential models can be represented something like this:
# this goes into the default graph
<base-model> a md:Model;
md:Model.created ... ;
md:Model.updated ... ;
# other Header data
<difference-model> a dm:DifferenceModel;
dm:DifferenceModel.baseModel <base-model>; #### NEW
dm:DifferenceModel.preconditions # TODO; # don't know what this is, gotta read the spec
dm:DifferenceModel.reverseDifferences <differences-to-delete>;
dm:DifferenceModel.forwardDifferences<differences-to-insert>.
<base-model> {
<transmissionLine> a TransmissionLine;
voltage 220
}
<differences-to-delete> {
<transmissionLine> voltage 220
}
<differences-to-insert> {
<transmissionLine> voltage 330
}
Currently preconditions, reverseDifferences, forwardDifferences
use rdf:Statement
(or rdf:Statements
) but RDF Reification is not a scalable nor good way to capture a bunch of triples.
Named graphs are better.
@Sveino do you agree?
@Sveino by email:
@Sveino @griddigit-ci
md:Model
with dcat:Dataset
. Can you show a mapping between the properties?
dm:DependsOn
shouldn't be mapped to dct:relation
(which is very generic) nor even dct:references
(which is for bibliographic citations: "resource that is referenced, cited, or otherwise pointed to"). Maybe use dct:hasPart
DifferenceModel
(reverseDifferences, forwardDifferences
)?PowerSystemProject
should be subclassed from Model (Dataset). Yes, it comes with a set of statements that describe what is being built. But say you want to change the commissioned
date or some other aspect: how will you do that?You can find the mapping of the header here: https://github.com/3lbits/CIM4NoUtility/issues/277 It might need to be revised to be inline with the latest Metadata document.
@Sveino I've reread https://github.com/3lbits/CIM4NoUtility/issues/277 . Since it's closed, I'll post more stuff here:
modelAuthoritySet
to dct:isVersionOf
: but we need to discuss it, so I better understand what is the meaning and significance of MAS.
This is well described in rdf-improvement
sections ## Represent Models as Named Graphs
and ### Custom CIM XML Parser
.
But I'll leave it open since @Sveino and @griddigit-ci we need to discuss the representation of Models:
you cannot use md/dm
in XML but dcat/dct
in JSON-LD: you have to use the same ontology terms!
Yes, this is a discussion point. @VladimirAlexiev do you know RDF Package (I think this is how it was called) to exchange RDF differences. It was something that was implemented in Jena for that
If you search at LOV for "patch" https://lov.linkeddata.es/dataset/lov/terms?q=patch and "diff" you find some stuff.
PREFIX rdfg: <http://www.w3.org/2004/03/trix/rdfg-1/>
rdfg:Graph rdf:type rdfs:Class;
rdfs:comment "An RDF graph (with intensional semantics).";
rdfs:label "Graph" .
rdfg:subGraphOf rdf:type rdf:Property;
rdfs:comment "The graph associated with the subject is a subgraph of a graph equivalent\nto that associated with the object.";
rdfs:domain rdfg:Graph;
rdfs:label "subgraph of";
rdfs:range rdfg:Graph .
@prefix pat: <http://purl.org/hpi/patchr#>
. Both links are broken but you can download v0.6 from the timeline at the bottom of the page.
@prefix guo: <http://webr3.org/owl/guo#>
<...> a guo:UpdateInstruction ;
guo:target_graph <http://dbpedia.org/>;
guo:target_subject dbp:Oregon ;
guo : insert [
dbo:language dbp:English_language
]
where=condition, deletes=reverseDifferences, inserts=forwardDifferences
patches
a single Resource... we can say that resource is the md:Model being changedlog:Formula
. If you look in the example, that is N3 ("anonymous" graphs interspersed in triples) not Turtle. I really like N3 Logic, but N3 is not supported by most RDF tools, eg jena riot doesn't support itriot.bat --validate solid-patch-eg.n3
:: [line: 5, col: 17] Not implemented (formulae, graph literals)
On the implementation side, there is https://afs.github.io/rdf-delta/rdf-patch.html but it's a line-oriented format to replicate repos, not an RDF format. ("afs" is the maker of Jena)
If we decide to use solid:
the example in https://github.com/Sveino/Inst4CIM-KG/tree/develop/rdf-improved#custom-cim-xml-parser will become:
<urn:uuid:05edbf91-231f-4386-97c0-d4cb498d0afc> { # model graph
# model metadata
<urn:uuid:05edbf91-231f-4386-97c0-d4cb498d0afc>
a dm:DifferenceModel,
rdfg:Graph.
solid:InsertDeletePatch ; <<<<<<<<
solid:deletes <urn:uri:27c8a164-c656-4712-994a-0ab7cec4fd34> ; <<<<<<
solid:inserts <urn:uri:63528ef9-48ff-469b-a58e-ba274f2a10bb> ; <<<<<<
solid:patches <urn:uuid:52a409c9-72d8-4b5f-bf72-9a22ec9353f7> ; # base model <<<<<<
md:Model.DependentOn <urn:uuid:0cd6ada4-b6dc-4a36-a98c-877a39168cd3> ;
md:Model.created "2021-11-19T23:16:27Z" .
<urn:uri:27c8a164-c656-4712-994a-0ab7cec4fd34> a rdfg:Graph.
<urn:uri:63528ef9-48ff-469b-a58e-ba274f2a10bb> a rdfg:Graph.
}
<urn:uri:27c8a164-c656-4712-994a-0ab7cec4fd34> { # reverseDifferences
<http://fullgrid.eu/CGMES/3.0#87478acb-cd1f-40a6-b4a7-59ec99f8b063> cim:IdentifiedObject.description "SET_PNT_1" .
<http://fullgrid.eu/CGMES/3.0#fc908c16-468f-4a64-ba74-6f57175e0005> cim:AnalogLimit.value "99" .
}
<urn:uri:63528ef9-48ff-469b-a58e-ba274f2a10bb> { # forwardDifferences
<http://fullgrid.eu/CGMES/3.0#87478acb-cd1f-40a6-b4a7-59ec99f8b063> cim:IdentifiedObject.description "SET_PNT_1 test" .
<http://fullgrid.eu/CGMES/3.0#fc908c16-468f-4a64-ba74-6f57175e0005> cim:AnalogLimit.value "100" .
}
I like the definitions in https://vocab.org/changeset/schema.html:
However, it should in principle expand DCAT rather than defining their own term for CreatedDate
etc
I think that solid: is a very important project and I would like to follow this. However, I am not happy with the terms. It is too syntax based rather than Semantic.
I would really like to agree on the CIM JSON-LD serialisation of difference model before we look into trix or N3. We worked with this in this issue: https://github.com/3lbits/CIM4NoUtility/issues/279
Here I listed the following references:
If we start with the set theory: In set theory, the symmetric difference of two sets $A$ and $B$ includes elements that are in either $A$ or $B$, but not in both. This operation is denoted as $A \triangle B$ and is defined formally as:
$$ A \triangle B = (A \setminus B) \cup (B \setminus A) $$
Given sets $A = {1, 2, 3, 4}$ and $B = {3, 4, 5, 6}$, the symmetric difference is:
$$ A \triangle B = {1, 2, 5, 6} $$
This result contains elements that are in either $A$ or $B$ but not in both.
In set theory, the difference (or relative complement) between two sets $A$ and $B$ is the set of elements that are in $A$ but not in $B$. This operation is commonly denoted as $A \setminus B$ and is defined as:
$$ A \setminus B = { x \in A \mid x \notin B } $$
Given sets $A = {1, 2, 3, 4}$ and $B = {3, 4, 5}$, the difference set is:
$$ A \setminus B = {1, 2} $$
This result contains only the elements in $A$ that are not in $B$.
reverseDifference is
$$ A \setminus B = {1, 2} $$
forwardDifference is
$$ B \setminus A = {5} $$
@VladimirAlexiev you wrote:
I'm not sure PowerSystemProject should be subclassed from Model (Dataset). Yes, it comes with a set of statements that describe what is being built. But say you want to change the commissioned date or some other aspect: how will you do that?
https://github.com/3lbits/CIM4NoUtility/discussions/321#discussioncomment-10741215 shows that without named graphs, you get a tragedy of tangled statements
I agree with both statement - even it my hand is on it... I think PowerSystemProject must be a separate class as a specialisation of cim:Project. The problem was that we wanted to use DifferenceModel with defining the content. We need to resolve this first.
You wrote:
dm:DifferenceModel.preconditions # TODO; # don't know what this is, gotta read the spec
Precondition statements, comprising statements found in both B1 and B2 and considered to be dependencies of the difference model in an application defined sense.
dm:preconditions is a property of the difference model whose value is the collection of precondition statements.
7.2.4.5 Preconditions and concurrency The precondition statements are a subset of both B1 and B2 and carry no difference information. In simple, sequential model revision scenarios they can be omitted. For a large shared model, sequential revision is not always feasible. Revisions are likely to be constructed concurrently by different participants, without reference to each other. Concurrency issues must be handled, but the conventional database-oriented approach of using locks to detect incompatible concurrent transactions is not feasible on a web-scale. The precondition statements are an alternative to locks. Informally, they represent the information that would have been read-locked in an equivalent database transaction. Software agents that process difference models can check that the preconditions hold and, if not, warn of incompatible model revisions. The choice of statements to include as preconditions is application-specific (as is the choice of which information to lock in a database transaction). Preconditions should include statements that would affect decisions of the agent that produced the model revision.
7.2.4.6 Difference model template
<rdf:RDF xmlns:rdf=”http://www.w3.org/1999/02/22-rdf-syntax-ns#”
xmlns:cim=”cim-namespace-uri”
xmlns:md=”cim-model-description_uri”
xmlns:dm=”difference-model-namespace-uri”
xml:base="urn:uuid:">
<dm:DifferenceModel rdf:about=model-uri>
<!--Content:(literal-property|resource-property|compound-property)*
–->
<dm:preconditions parseType=”Statements”>
<!-– Content: (definition|description)* –->
</dm:preconditions>
<dm:forwardDifferences parseType=”Statements”>
<!-– Content: (definition|description)* –->
</dm:forwardDifferences>
<dm:reverseDifferences parseType=”Statements”>
<!-– Content: (definition|description)* –->
</dm:reverseDifferences>
</dm:DifferenceModel>
</rdf:RDF>
We have to change the md and dm namespace since they are now referring to -552:
@prefix dm: <http://iec.ch/TC57/61970-552/DifferenceModel/1#>
@prefix md: <http://iec.ch/TC57/61970-552/ModelDescription/1#> .
I do not understand the need for two namespaces. I think one alternative is to extent DCAT. However, in addition we would need to use rdfg:Graph
in the serialisation so that we can support the uniqueness of identifier in the same graph.
To be compliant with Set theory DifferenceSet should be called SymmetricDifferenceSet - however that we can include in the description. The easiest way to handle the syntax would be to exchange the ForwardDifferenceSet and the ReverseDifferenceSet as separate distribution (instance set) .
I think the dm:preconditions
is in general referring to the dataset version. This needs to be know to ably the changes. I do not see any point that ForwardDifferenceSet and ReverseDifferenceSet should have any dataset (header) information as they should be the same as DifferenceSet. I also think that this could be the same as the target DataSet. The following are relevant dataset information:
<urn:uuid:6b7387f2-c7f9-11ed-afa1-0242ac120002> a dct-cim:DifferenceSet ;
dcterms:accessRights <https://energy.referencedata.eu/Confidentiality/OPDEConfidential> ;
dcterms:conformsTo <https://ap.cim4.eu/RemedialAction/2.3> ;
dcterms:description "Statnett Remedial Action dataset"@en ;
dcterms:identifier "6b7387f2-c7f9-11ed-afa1-0242ac120002" ;
dcterms:issued "2024-04-10T10:00:00Z"^^xsd:dateTime ;
dcat:isVersionOf <https://energy.referencedata.eu/Model/Statnett-RA> ;
dcat:keyword "RA" ;
dcterms:license <https://creativecommons.org/licenses/by/4.0/> ;
dcterms:publisher <https://energy.referencedata.eu/EIC/10X1001A1001A38Y> ;
dcterms:references <urn:uuid:99ae9f41-0a91-4d21-a483-7398c160da96> ;
dcterms:spatial <https://energy.referencedata.eu/Frame/NO-Power-Transmission-System> ;
dcat:startDate "2024-04-10T05:00:00Z"^^xsd:dateTime ;
dcterms:title "20240410_NO-RA" ;
dcat:version "2.0.0" ;
adms:versionNotes "This is the second version."@en ;
prov:wasGeneratedBy <https://energy.referencedata.eu/Action/CGM-RA> ;
prov:wasRevisionOf <urn:uuid:34f39767-fe2d-45f8-b5f8-aa60375071f1>;
dct-cim:forwardDifferenceSet <urn:uuid:a4025d93-bf8c-4d38-baeb-ddc2219e7570>;
dct-cim:reversalDifferenceSet <urn:uuid:e5a0a4ad-f115-4f79-af3f-45a5317398c3>;
dcterms:conformsTo
describe what the target dataset should be validated against
dcterms:references
: cannot be used to define the set that should be updated since the new full model will also have references.
prov:wasRevisionOf
- This refer to the dataset that is modified through the DifferenceSet
Example - without the header.
{
"@context": {
"rdf": "http://www.w3.org/1999/02/22-rdf-syntax-ns#",
"cim": "https://cim.ucaiug.io/ns#",
"dcat": "http://www.w3.org/ns/dcat#",
"dcterms": "http://purl.org/dc/terms/#",
"dcat-cim": " https://cim4.eu/nc/dcat-cim#"
},
"@graph": [
{
"@id": "urn:uuid:a4025d93-bf8c-4d38-baeb-ddc2219e7570",
"@type": "dcat:Dataset",
"dcterms:title": "Forward Difference Set",
"dcterms:identifier": "a4025d93-bf8c-4d38-baeb-ddc2219e7570",
"dcat-cim:forwardDifferenceSet": [
{
"@id": "http://fullgrid.eu/CGMES/3.0#87478acb-cd1f-40a6-b4a7-59ec99f8b063",
"cim:IdentifiedObject.description": "SET_PNT_1 test"
},
{
"@id": "http://fullgrid.eu/CGMES/3.0#fc908c16-468f-4a64-ba74-6f57175e0005",
"cim:AnalogLimit.value": "100"
}
]
},
{
"@id": "urn:uuid:e5a0a4ad-f115-4f79-af3f-45a5317398c3",
"@type": "dcat:Dataset",
"dcterms:title": "Reverse Difference Set",
"dcterms:identifier": "a4025d93-bf8c-4d38-baeb-ddc2219e7570",
"dcat-cim:reverseDifferenceSet": [
{
"@id": "http://fullgrid.eu/CGMES/3.0#87478acb-cd1f-40a6-b4a7-59ec99f8b063",
"cim:IdentifiedObject.description": "SET_PNT_1"
},
{
"@id": "http://fullgrid.eu/CGMES/3.0#fc908c16-468f-4a64-ba74-6f57175e0005",
"cim:AnalogLimit.value": "99"
}
]
}
]
}
@Sveino convert the last example to trig and you'll see that forward, reverse
have 2 values each, which are nodes not triple sets.
JSONLD nesting structure does not create nested subgraphs. You must use graph URIs explicitly.
An example is in rdf-improved/README
Updated Example based on the discussion on 2024-10-31. The example should support it to be resolvable. The namespace and transformation will be address in other issues. I just want to established the goal for the JSON-LD and TRIG description of a difference model.
{
"@context": {
"rdf": "http://www.w3.org/1999/02/22-rdf-syntax-ns#",
"cim": "https://cim.ucaiug.io/ns#",
"dcat": "http://www.w3.org/ns/dcat#",
"dcterms": "http://purl.org/dc/terms/#",
"dcat-cim": "https://cim4.eu/ns/dcat-cim#",
"xsd": "http://www.w3.org/2001/XMLSchema#",
"prov": "http://www.w3.org/ns/prov#",
"base": "https://test.model4powersystem.eu/FullGrid/"
},
"@graph": [
{
"@graph": [
{
"@id": "6b7387f2-c7f9-11ed-afa1-0242ac120002",
"@type": "dcat-cim:DifferenceSet",
"dcterms:identifier": "6b7387f2-c7f9-11ed-afa1-0242ac120002",
"dcterms:issued": {
"@value": "2021-11-19T23:16:27Z",
"@type": "xsd:dateTime"
},
"dcterms:description": {
"@value": "CGMES Conformity Assessment Test Configuration. ..."
},
"dcat:isVersionOf": {
"@id": "https://energy.referencedata.eu/Model/FullGrid/EQOP"
},
"dcterms:references": {
"@id": "99ae9f41-0a91-4d21-a483-7398c160da96"
},
"prov:wasRevisionOf": {
"@id": "34f39767-fe2d-45f8-b5f8-aa60375071f1"
},
"dcat-cim:forwardDifferenceSet": {
"@id": "a4025d93-bf8c-4d38-baeb-ddc2219e7570"
},
"dcat-cim:reversalDifferenceSet": {
"@id": "e5a0a4ad-f115-4f79-af3f-45a5317398c3"
}
}
]
},
{
"@graph": [
{
"@id": "a4025d93-bf8c-4d38-baeb-ddc2219e7570",
"@type": "dcat:Dataset",
"dcterms:title": "Forward Difference Set",
"dcterms:identifier": "a4025d93-bf8c-4d38-baeb-ddc2219e7570"
},
{
"@id": "87478acb-cd1f-40a6-b4a7-59ec99f8b063",
"cim:IdentifiedObject.description": "SET_PNT_1 test"
},
{
"@id": "fc908c16-468f-4a64-ba74-6f57175e0005",
"cim:AnalogLimit.value": "100"
}
]
},
{
"@graph": [
{
"@id": "e5a0a4ad-f115-4f79-af3f-45a5317398c3",
"@type": "dcat:Dataset",
"dcterms:identifier": "e5a0a4ad-f115-4f79-af3f-45a5317398c3",
"dcterms:title": "Reverse Difference Set"
},
{
"@id": "87478acb-cd1f-40a6-b4a7-59ec99f8b063",
"cim:IdentifiedObject.description": "SET_PNT_1"
},
{
"@id": "fc908c16-468f-4a64-ba74-6f57175e0005",
"cim:AnalogLimit.value": "99"
}
]
}
]
}
TRIG
@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
@prefix cim: <https://cim.ucaiug.io/ns#> .
@prefix dcat: <http://www.w3.org/ns/dcat#> .
@prefix dcterms: <http://purl.org/dc/terms/#> .
@prefix dcat-cim: <https://cim4.eu/ns/dcat-cim#> .
@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .
@prefix prov: <http://www.w3.org/ns/prov#> .
@prefix base: <https://test.model4powersystem.eu/FullGrid/> .
# Graph New full graph
<https://test.model4powersystem.eu/FullGrid/6b7387f2-c7f9-11ed-afa1-0242ac120002> {
<6b7387f2-c7f9-11ed-afa1-0242ac120002> a dcat-cim:DifferenceSet ;
dcterms:identifier "6b7387f2-c7f9-11ed-afa1-0242ac120002" ;
dcterms:issued "2021-11-19T23:16:27Z"^^xsd:dateTime ;
dcterms:description "CGMES Conformity Assessment Test Configuration. ..." ;
dcat:isVersionOf <https://energy.referencedata.eu/Model/FullGrid/EQOP> ;
dcterms:references <99ae9f41-0a91-4d21-a483-7398c160da96> ;
prov:wasRevisionOf <34f39767-fe2d-45f8-b5f8-aa60375071f1> ;
dcat-cim:forwardDifferenceSet <a4025d93-bf8c-4d38-baeb-ddc2219e7570> ;
dcat-cim:reversalDifferenceSet <e5a0a4ad-f115-4f79-af3f-45a5317398c3> .
}
# Forward Graph
<https://test.model4powersystem.eu/FullGrid/a4025d93-bf8c-4d38-baeb-ddc2219e7570> {
<a4025d93-bf8c-4d38-baeb-ddc2219e7570> a dcat:Dataset ;
dcterms:title "Forward Difference Set" ;
dcterms:identifier "a4025d93-bf8c-4d38-baeb-ddc2219e7570" .
<87478acb-cd1f-40a6-b4a7-59ec99f8b063> cim:IdentifiedObject.description "SET_PNT_1 test" .
<fc908c16-468f-4a64-ba74-6f57175e0005> cim:AnalogLimit.value "100" .
}
# Reverse Graph
<https://test.model4powersystem.eu/FullGrid/graph3> {
<e5a0a4ad-f115-4f79-af3f-45a5317398c3> a dcat:Dataset ;
dcterms:identifier "e5a0a4ad-f115-4f79-af3f-45a5317398c3" ;
dcterms:title "Reverse Difference Set" .
<87478acb-cd1f-40a6-b4a7-59ec99f8b063> cim:IdentifiedObject.description "SET_PNT_1" .
<fc908c16-468f-4a64-ba74-6f57175e0005> cim:AnalogLimit.value "99" .
}
In the meeting 2024-11-21 we agree to the following UML for the DifferenceSet:
For the record, the above code blocks are copies of
There are various little things to fix in these examples:
@Sveino (and @griddigit-ci) in https://github.com/Sveino/Inst4CIM-KG/issues/15#issuecomment-2350981370: