search

SDM-TIB / SDM-RDFizer

An Efficient RML-Compliant Engine for Knowledge Graph Construction
https://doi.org/10.5281/zenodo.3872103
Apache License 2.0
107 stars 25 forks source link

Are JSON keys expected if present in the RML mapping? #97

Closed nicolastoira closed 1 year ago

nicolastoira commented 1 year ago

I do have a mapping which iterates over a specific key. Nevertheless, that key is not present in the JSON data because it is not a mandatory property. My expectation is that the mapper should pick up and convert the fields from the JSON data to RDF data only when a match is found according to the defined RML mapping. It seems that here this is not valid, and that on the other hand if the key is not found in the JSON data, then the execution of the rdfizer fails. Is it expected?

Command: python3 -m rdfizer -c rdfizer_config.txt

RML mapping:

rml:logicalSource [ rml:iterator "$.exampleKey[*]" ;
            rml:referenceFormulation ql:JSONPath ;
            rml:source "patient-ok.json" ]

Output:

Semantifying OutputRDFkg-D1...
***
Traceback (most recent call last):
  File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/nicola/workspace/venv/lib/python3.8/site-packages/rdfizer/__main__.py", line 49, in <module>
    main()
  File "/home/nicola/workspace/venv/lib/python3.8/site-packages/rdfizer/__main__.py", line 45, in main
    semantify(config_path)
  File "/home/nicola/workspace/venv/lib/python3.8/site-packages/rdfizer/__init__.py", line 4474, in semantify
    number_triple += executor.submit(semantify_json, sorted_sources[source_type][source][triples_map], triples_map_list, ",",output_file_descriptor, data, sorted_sources[source_type][source][triples_map].iterator).result()
  File "/usr/lib/python3.8/concurrent/futures/_base.py", line 437, in result
    return self.__get_result()
  File "/usr/lib/python3.8/concurrent/futures/_base.py", line 389, in __get_result
    raise self._exception
  File "/usr/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/home/nicola/workspace/venv/lib/python3.8/site-packages/rdfizer/__init__.py", line 1465, in semantify_json
    row = row[tp.split("[*]")[0]]
KeyError: 'exampleKey'

Example JSON data:

{
"key1" : {
    "id": "ID"
}
}
nicolastoira commented 1 year ago

Failing there:

if "$" != tp and "" != tp:
                if "[*][*]" in tp:
                    row = row[tp.split("[*][*]")[0]]
                elif "[*]" in tp:
                    row = row[tp.split("[*]")[0]]
                elif "*" == tp:
                    pass
                else:
                    row = row[tp]
eiglesias34 commented 1 year ago

Hello again,

I found the issue and fixed it. Now, if the key is not present, the SDM-RDFizer won't crash.

Thank you again, Enrique Iglesias

nicolastoira commented 1 year ago

Thank you! Though it is still fixing in the case we have another JSON Path as iterator without the [*].

For example:

rml:iterator "$..exampleKey

nicolastoira commented 1 year ago

Maybe the case when we have .. in the path should be added to the list of conditions.

eiglesias34 commented 1 year ago

Just to be clear, the case with the .. is incorrect, right?

nicolastoira commented 1 year ago

We do have some mapping that look for matches by looking at all the property names matching:

:hasQuantityMapping a rr:TriplesMap ;
    rml:logicalSource [ rml:iterator "$..hasQuantity" ;
            rml:referenceFormulation ql:JSONPath ;
            rml:source "patient-data-input.json" ] ;

where that mapping is called by a join condition:

[ rr:objectMap [ rr:joinCondition [ rr:child "hasQuantity.id" ;
                            rr:parent "id" ] ;
                    rr:parentTriplesMap :hasQuantityMapping ] ;

and it is used when you have data like this:

{
 "upperKey": {
    "hasQuantity": {
         "id": "ID"
}
}
}
eiglesias34 commented 1 year ago

Hello again,

Just one more question, and then I'll be able to update the SDM-RDFizer. When we have .. in the iterator there should only be one key at that level of the JSON file, correct?

SIncerely, Enrique Iglesias

nicolastoira commented 1 year ago

Yeah, usually there is only one key in the sub-object. But we need to do it like this in order to match the join condition.

eiglesias34 commented 1 year ago

Hello @nicolastoira,

I made the changes we discussed yesterday. Please test it out. I hope this solves your problem.

SIncerely, Enrique Igleisas

nicolastoira commented 1 year ago

It is still failing for this property:

:AllergyMapping a rr:TriplesMap ;
    rml:logicalSource [ rml:iterator "$.content.pre:Allergy[*]" ;
            rml:referenceFormulation ql:JSONPath ;
            rml:source "patient-ok.json" ] ;
    rr:predicateObjectMap [ rr:objectMap [ rr:joinCondition [ rr:child "pre:hasReactionTypeCode.termid" ;
                            rr:parent "termid" ] ;
                    rr:parentTriplesMap :hasReactionTypeCodeMapping ] ;
            rr:predicate pre:hasReactionTypeCode ] ;
    rr:subjectMap [ rr:class pre:Allergy ;
            rr:template "https://example/Allergy-{id}" ] .

with invoked mapping:

:hasReactionTypeCodeMapping a rr:TriplesMap ;
    rml:logicalSource [ rml:iterator "$..pre:hasReactionTypeCode" ;
            rml:referenceFormulation ql:JSONPath ;
            rml:source "patient-ok.json" ] ;
    rr:predicateObjectMap [ rr:objectMap [ rml:reference "iri" ;
                    rr:termType rr:IRI ] ;
            rr:predicate rdf:type ] ;
    rr:subjectMap [ rr:template "https://example/test-{termid}" ]

where the JSON data is the following:

{

    "content": {
            "pre:Allergy": [
                {
                "id" : "ID",
                "pre:hasReactionTypeCode": {"termid": "TERMID", "iri" : "http://example/1121331"}
            }
        ]
    }

}
eiglesias34 commented 1 year ago

One quick question. Does .. mean that it should go down to the level denoted in the iterator, or is it only one skipped level?

nicolastoira commented 1 year ago

Using .. was the only solution we found to connect the nested parts via the join condition. In this case, "$..pre:hasReactionTypeCode" extracts all the field's values with key pre:hasReactionTypeCode at any level of nesting. Then it matches this value with the value found in the upper mapping and defines a resource in RDF. In principle, we'll only have one match, but if we have another key named pre:hasReactionTypeCode, then we will get it twice in the RDF data but that's fine because we can remove it be deduplicating afterwards.

To summarize, .. goes down to all nested levels and looks for that key in the maps.

eiglesias34 commented 1 year ago

Hello again,

I finally made it work with the example you sent me. Hopefully, there will be no more issues regarding this problem.

Thank you again for using the SDM-RDFizer.

Sincerely, Enrique Iglesias

nicolastoira commented 1 year ago

Hi, I think it improved. Nevertheless, it is still not working if some required properties are missing. I think this depends also on the special use case. In my use case, the expected behavior is that if a required property is missing and it's marked as required in the JSON schema, then it's fine it just won't be mapped to the RDF data. But maybe for other cases, the logic should fail if that property is missing. Therefore maybe we can close it. Even though it shouldn't be related to the JSON schema but only on the JSON data and on the RML mappings.

For me is still failing because it doesn't find a property in the JSON data in the second object of an array of objects:

Traceback (most recent call last):
  File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/nicola/workspace/venv/lib/python3.8/site-packages/rdfizer/__main__.py", line 49, in <module>
    main()
  File "/home/nicola/workspace/venv/lib/python3.8/site-packages/rdfizer/__main__.py", line 45, in main
    semantify(config_path)
  File "/home/nicola/workspace/venv/lib/python3.8/site-packages/rdfizer/__init__.py", line 4517, in semantify
    number_triple += executor.submit(semantify_json, sorted_sources[source_type][source][triples_map], triples_map_list, ",",output_file_descriptor, data, sorted_sources[source_type][source][triples_map].iterator).result()
  File "/usr/lib/python3.8/concurrent/futures/_base.py", line 444, in result
    return self.__get_result()
  File "/usr/lib/python3.8/concurrent/futures/_base.py", line 389, in __get_result
    raise self._exception
  File "/usr/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/home/nicola/workspace/venv/lib/python3.8/site-packages/rdfizer/__init__.py", line 1524, in semantify_json
    i += semantify_json(triples_map, triples_map_list, delimiter, output_file_descriptor, sub_row, iterator.replace(new_iterator[:-1],""))
  File "/home/nicola/workspace/venv/lib/python3.8/site-packages/rdfizer/__init__.py", line 1874, in semantify_json
    join_iterator(parent_data, triples_map_element.iterator, triples_map_element, predicate_object_map.object_map)
  File "/home/nicola/workspace/venv/lib/python3.8/site-packages/rdfizer/__init__.py", line 160, in join_iterator
    row = row[tp]
KeyError: 'pre:hasSeverityCode'
eiglesias34 commented 1 year ago

Hello again,

I updated it again. This time I got a little more creative with the example you sent me. Here is the data source I used:

{

    "content": {
            "pre:Allergy": [
                {
                "id" : "ID",
                "pre:hasReactionTypeCode": {"termid": "TERMID", "iri" : "http://example/1121331"}
            },
            {
                "id" : "ID2"
            },
            {
                "id" : "ID3",
                "pre:hasReactionTypeCode": {"iri" : "http://example/1121331"}
            }
        ]
    }

}

Hopefully, I was able to cover more possible cases, and it won't explode anymore.

Thank you again.

SIncerely, Enrique Iglesias

nicolastoira commented 1 year ago

Hi Enrique, thank you for the support. Unfortunately, it is still failing for a more complex example. I list below the configuration and data used. You can give a look and decide if this should be fixed in the rdfizer or if it's too specific for my use case. In the meantime I'll close the issue. Thanks.

Configuration:

[default]
main_directory: /home/nicola/workspace/lib

[datasets]
number_of_datasets: 1
output_folder: ${default:main_directory}/graph
all_in_one_file: yes
remove_duplicate: yes
enrichment: no
ordered: no
name: OutputRDFkg1

[dataset1]
name: OutputRDFkg-D1
mapping: ${default:main_directory}/rml_generated_mapping.ttl

RML mapping:

@prefix owl: <http://www.w3.org/2002/07/owl#> .
@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
@prefix resource: <https://example.com/rdf/example-resource/> .
@prefix : <https://example.com/rdf/example-rml/example#> .
@prefix ql: <http://semweb.mmlab.be/ns/ql#> .
@prefix rml: <http://semweb.mmlab.be/ns/rml#> .
@prefix rr: <http://www.w3.org/ns/r2rml#> .
@prefix example: <https://example.com/rdf/example-ontology/example#> .
@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .

:AllergyMapping a rr:TriplesMap ;
    rml:logicalSource [ rml:iterator "$.content.example:Allergy[*]" ;
            rml:referenceFormulation ql:JSONPath ;
            rml:source "patient-ok.json" ] ;
    rr:predicateObjectMap [ rr:objectMap [ rml:reference "example:hasSubstanceCategory.iri" ;
                    rr:termType rr:IRI ] ;
            rr:predicate example:hasSubstanceCategory ],
        [ rr:objectMap [ rr:joinCondition [ rr:child "example:hasSeverityCode.termid" ;
                            rr:parent "termid" ] ;
                    rr:parentTriplesMap :hasSeverityCodeMapping ] ;
            rr:predicate example:hasSeverityCode ],
        [ rr:objectMap [ rr:joinCondition [ rr:child "example:hasSubstance.id" ;
                            rr:parent "id" ] ;
                    rr:parentTriplesMap :hasSubstanceMapping ] ;
            rr:predicate example:hasSubstance ],
        [ rr:objectMap [ rr:joinCondition [ rr:child "example:hasReactionTypeCode.termid" ;
                            rr:parent "termid" ] ;
                    rr:parentTriplesMap :hasReactionTypeCodeMapping ] ;
            rr:predicate example:hasReactionTypeCode ],
        [ rr:objectMap [ rml:reference "example:hasFirstRecordDateTime" ;
                    rr:datatype xsd:dateTime ] ;
            rr:predicate example:hasFirstRecordDateTime ],
        [ rr:objectMap [ rr:parentTriplesMap :DataProviderInstituteMapping ] ;
            rr:predicate example:hasDataProviderInstitute ],
        [ rr:objectMap [ rml:reference "example:hasLastReactionDateTime" ;
                    rr:datatype xsd:dateTime ] ;
            rr:predicate example:hasLastReactionDateTime ],
        [ rr:objectMap [ rr:joinCondition [ rr:child "example:hasVerificationStatusCode.termid" ;
                            rr:parent "termid" ] ;
                    rr:parentTriplesMap :hasVerificationStatusCodeMapping ] ;
            rr:predicate example:hasVerificationStatusCode ],
        [ rr:objectMap [ rr:parentTriplesMap :SubjectPseudoIdentifierMapping ] ;
            rr:predicate example:hasSubjectPseudoIdentifier ] ;
    rr:subjectMap [ rr:class example:Allergy ;
            rr:template "https://example.com/rdf/example-resource/DATA-PROVIDER-ID-Allergy-{id}" ] .

:hasVerificationStatusCodeMapping a rr:TriplesMap ;
    rml:logicalSource [ rml:iterator "$..example:hasVerificationStatusCode" ;
            rml:referenceFormulation ql:JSONPath ;
            rml:source "patient-ok.json" ] ;
    rr:predicateObjectMap [ rr:objectMap [ rml:reference "iri" ;
                    rr:termType rr:IRI ] ;
            rr:predicate rdf:type ] ;
    rr:subjectMap [ rr:template "https://example.com/rdf/example-resource/Code-{termid}" ] .

:hasReactionTypeCodeMapping a rr:TriplesMap ;
    rml:logicalSource [ rml:iterator "$..example:hasReactionTypeCode" ;
            rml:referenceFormulation ql:JSONPath ;
            rml:source "patient-ok.json" ] ;
    rr:predicateObjectMap [ rr:objectMap [ rml:reference "iri" ;
                    rr:termType rr:IRI ] ;
            rr:predicate rdf:type ] ;
    rr:subjectMap [ rr:template "https://example.com/rdf/example-resource/Code-{termid}" ] .

:hasSeverityCodeMapping a rr:TriplesMap ;
    rml:logicalSource [ rml:iterator "$..example:hasSeverityCode" ;
            rml:referenceFormulation ql:JSONPath ;
            rml:source "patient-ok.json" ] ;
    rr:predicateObjectMap [ rr:objectMap [ rml:reference "iri" ;
                    rr:termType rr:IRI ] ;
            rr:predicate rdf:type ] ;
    rr:subjectMap [ rr:template "https://example.com/rdf/example-resource/Code-{termid}" ] .

:hasSeverityCodeMappingCode a rr:TriplesMap ;
    rml:logicalSource [ rml:iterator "$..example:hasSeverityCode" ;
            rml:referenceFormulation ql:JSONPath ;
            rml:source "patient-ok.json" ] ;
    rr:predicateObjectMap [ rr:objectMap [ rml:reference "example:hasCodingSystemAndVersion" ;
                    rr:datatype xsd:string ] ;
            rr:predicate example:hasCodingSystemAndVersion ],
        [ rr:objectMap [ rml:reference "example:hasName" ;
                    rr:datatype xsd:string ] ;
            rr:predicate example:hasName ],
        [ rr:objectMap [ rml:reference "example:hasIdentifier" ;
                    rr:datatype xsd:string ] ;
            rr:predicate example:hasIdentifier ] ;
    rr:subjectMap [ rr:class example:Code ;
            rr:template "https://example.com/rdf/example-resource/DATA-PROVIDER-ID-Code-{id}" ] .

:hasSeverityCodeMappingTerminology a rr:TriplesMap ;
    rml:logicalSource [ rml:iterator "$..example:hasSeverityCode" ;
            rml:referenceFormulation ql:JSONPath ;
            rml:source "patient-ok.json" ] ;
    rr:predicateObjectMap [ rr:objectMap [ rml:reference "iri" ;
                    rr:termType rr:IRI ] ;
            rr:predicate rdf:type ] ;
    rr:subjectMap [ rr:template "https://example.com/rdf/example-resource/Code-{termid}" ] .

:hasSubstanceMapping a rr:TriplesMap ;
    rml:logicalSource [ rml:iterator "$..example:hasSubstance" ;
            rml:referenceFormulation ql:JSONPath ;
            rml:source "patient-ok.json" ] ;
    rr:predicateObjectMap [ rr:objectMap [ rr:joinCondition [ rr:child "example:hasCode.id" ;
                            rr:parent "id" ] ;
                    rr:parentTriplesMap :hasCodeMappingCode ] ;
            rr:predicate example:hasCode ],
        [ rr:objectMap [ rml:reference "example:hasGenericName" ;
                    rr:datatype xsd:string ] ;
            rr:predicate example:hasGenericName ],
        [ rr:objectMap [ rr:joinCondition [ rr:child "example:hasQuantity.id" ;
                            rr:parent "id" ] ;
                    rr:parentTriplesMap :hasQuantityMapping ] ;
            rr:predicate example:hasQuantity ],
        [ rr:objectMap [ rr:joinCondition [ rr:child "example:hasCode.termid" ;
                            rr:parent "termid" ] ;
                    rr:parentTriplesMap :hasCodeMappingTerminology ] ;
            rr:predicate example:hasCode ] ;
    rr:subjectMap [ rr:class example:Substance ;
            rr:template "https://example.com/rdf/example-resource/DATA-PROVIDER-ID-Substance-{id}" ] .

:hasCodeMappingCode a rr:TriplesMap ;
    rml:logicalSource [ rml:iterator "$..example:hasCode" ;
            rml:referenceFormulation ql:JSONPath ;
            rml:source "patient-ok.json" ] ;
    rr:predicateObjectMap [ rr:objectMap [ rml:reference "example:hasName" ;
                    rr:datatype xsd:string ] ;
            rr:predicate example:hasName ],
        [ rr:objectMap [ rml:reference "example:hasCodingSystemAndVersion" ;
                    rr:datatype xsd:string ] ;
            rr:predicate example:hasCodingSystemAndVersion ],
        [ rr:objectMap [ rml:reference "example:hasIdentifier" ;
                    rr:datatype xsd:string ] ;
            rr:predicate example:hasIdentifier ] ;
    rr:subjectMap [ rr:class example:Code ;
            rr:template "https://example.com/rdf/example-resource/DATA-PROVIDER-ID-Code-{id}" ] .

:hasCodeMappingTerminology a rr:TriplesMap ;
    rml:logicalSource [ rml:iterator "$..example:hasCode" ;
            rml:referenceFormulation ql:JSONPath ;
            rml:source "patient-ok.json" ] ;
    rr:predicateObjectMap [ rr:objectMap [ rml:reference "iri" ;
                    rr:termType rr:IRI ] ;
            rr:predicate rdf:type ] ;
    rr:subjectMap [ rr:template "https://example.com/rdf/example-resource/Code-{termid}" ] .

:hasQuantityMapping a rr:TriplesMap ;
    rml:logicalSource [ rml:iterator "$..example:hasQuantity" ;
            rml:referenceFormulation ql:JSONPath ;
            rml:source "patient-ok.json" ] ;
    rr:predicateObjectMap [ rr:objectMap [ rml:reference "example:hasValue" ;
                    rr:datatype xsd:double ] ;
            rr:predicate example:hasValue ],
        [ rr:objectMap [ rml:reference "example:hasComparator.iri" ;
                    rr:termType rr:IRI ] ;
            rr:predicate example:hasComparator ],
        [ rr:objectMap [ rr:joinCondition [ rr:child "example:hasUnit.id" ;
                            rr:parent "id" ] ;
                    rr:parentTriplesMap :hasUnitMapping ] ;
            rr:predicate example:hasUnit ] ;
    rr:subjectMap [ rr:class example:Quantity ;
            rr:template "https://example.com/rdf/example-resource/DATA-PROVIDER-ID-Quantity-{id}" ] .

:hasUnitMapping a rr:TriplesMap ;
    rml:logicalSource [ rml:iterator "$..example:hasUnit" ;
            rml:referenceFormulation ql:JSONPath ;
            rml:source "patient-ok.json" ] ;
    rr:predicateObjectMap [ rr:objectMap [ rml:reference "example:hasCode.iri" ;
                    rr:termType rr:IRI ] ;
            rr:predicate example:hasCode ] ;
    rr:subjectMap [ rr:class example:Unit ;
            rr:template "https://example.com/rdf/example-resource/DATA-PROVIDER-ID-Unit-{id}" ] .

:hasCodeMapping a rr:TriplesMap ;
    rml:logicalSource [ rml:iterator "$..example:hasCode" ;
            rml:referenceFormulation ql:JSONPath ;
            rml:source "patient-ok.json" ] ;
    rr:predicateObjectMap [ rr:objectMap [ rml:reference "iri" ;
                    rr:termType rr:IRI ] ;
            rr:predicate rdf:type ] ;
    rr:subjectMap [ rr:template "https://example.com/rdf/example-resource/Code-{termid}" ] .

:DataProviderInstituteMapping a rr:TriplesMap ;
    rml:logicalSource [ rml:iterator "$.example:DataProviderInstitute" ;
            rml:referenceFormulation ql:JSONPath ;
            rml:source "patient-ok.json" ] ;
    rr:predicateObjectMap [ rr:objectMap [ rr:joinCondition [ rr:child "example:hasCode.id" ;
                            rr:parent "id" ] ;
                    rr:parentTriplesMap :hasCodeMappingCode ] ;
            rr:predicate example:hasCode ],
        [ rr:objectMap [ rr:joinCondition [ rr:child "example:hasCode.termid" ;
                            rr:parent "termid" ] ;
                    rr:parentTriplesMap :hasCodeMappingTerminology ] ;
            rr:predicate example:hasCode ] ;
    rr:subjectMap [ rr:class example:DataProviderInstitute ;
            rr:template "https://example.com/rdf/example-resource/DATA-PROVIDER-ID-DataProviderInstitute-{id}" ] .

:SubjectPseudoIdentifierMapping a rr:TriplesMap ;
    rml:logicalSource [ rml:iterator "$.example:SubjectPseudoIdentifier" ;
            rml:referenceFormulation ql:JSONPath ;
            rml:source "patient-ok.json" ] ;
    rr:predicateObjectMap [ rr:objectMap [ rr:parentTriplesMap :DataProviderInstituteMapping ] ;
            rr:predicate example:hasDataProviderInstitute ],
        [ rr:objectMap [ rml:reference "example:hasIdentifier" ;
                    rr:datatype xsd:string ] ;
            rr:predicate example:hasIdentifier ] ;
    rr:subjectMap [ rr:class example:SubjectPseudoIdentifier ;
            rr:template "https://example.com/rdf/example-resource/DATA-PROVIDER-ID-SubjectPseudoIdentifier-{id}" ] .

JSON data:

{
    "id": "https://example.com/rdf/example-ontology/example/2022-1.json",
    "schema": "https://example.com/rdf/example-ontology/example/schema",
    "title": "PatientRecord",
    "targetType": "RDF",
    "license": "",
    "example:DataProviderInstitute": {
        "id" : "fdg",
        "example:hasCode" : {
            "id" : "HudsfhG",
            "example:hasCodingSystemAndVersion" : "dfh",
            "example:hasIdentifier" : "dfhdfd"
            }}
            ,
    "example:SubjectPseudoIdentifier": {
        "id" : "dfhdfhn",
        "example:hasIdentifier" : "sdfhsfbsf"
    },
    "content": {
            "example:Allergy": [
                {
                "id" : "ysdgsdg",
                "example:hasFirstRecordDateTime": "2022-04-24T11:55:43.304Z",
                "example:hasLastReactionDateTime": "2022-03-31T11:55:43.304Z",
                "example:hasVerificationStatusCode" : {"termid": "example_AB-410605003", "iri" : "http://example.info/id/410605003"},
                "example:hasSubstance": {
                    "id": "sbsrgh5agb",
                    "example:hasGenericName": "ffhhbthb",
                    "example:hasCode":{
                        "id": "rbgaerg435b",
                        "example:hasCodingSystemAndVersion": "abc",
                        "example:hasIdentifier": "dsgdsg",
                        "example:hasName": "sdgsd"
                    },
                    "example:hasQuantity": {
                        "id": "GTE-0.8880891468892481-mmsblHgsbr",
                        "example:hasValue": 0.8880891468892481,
                        "example:hasComparator": {"iri": "https://example.com/rdf/example-ontology/example#GreaterThanOrEqual"},
                        "example:hasUnit":{
                            "id": "mmsblHgsbr",
                            "example:hasCode":  {
                                "iri": "https://example.com/rdf/example-resource/ucum/mmsblHgsbr"
                            }
                        }
                    }
                },
                "example:hasSeverityCode": {"termid": "example_AB-723505004", "iri" : "http://example.info/id/723505004"},
                "example:hasReactionTypeCode": {"termid": "example_AB-419199007", "iri" : "http://example.info/id/419199007"},
                "example:hasSubstanceCategory": {"iri": "https://example.com/rdf/example-ontology/example#Food"}
            },
            {
                "id": "allergy2"
            }
        ]
    }

}

Error message:

Semantifying OutputRDFkg-D1...
TM: https://example.com/rdf/example-rml/example#AllergyMapping
TM: https://example.com/rdf/example-rml/example#AllergyMapping
Traceback (most recent call last):
  File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/nicola/workspace/venv/lib/python3.8/site-packages/rdfizer/__main__.py", line 49, in <module>
    main()
  File "/home/nicola/workspace/venv/lib/python3.8/site-packages/rdfizer/__main__.py", line 45, in main
    semantify(config_path)
  File "/home/nicola/workspace/venv/lib/python3.8/site-packages/rdfizer/__init__.py", line 4536, in semantify
    number_triple += executor.submit(semantify_json, sorted_sources[source_type][source][triples_map], triples_map_list, ",",output_file_descriptor, data, sorted_sources[source_type][source][triples_map].iterator).result()
  File "/usr/lib/python3.8/concurrent/futures/_base.py", line 444, in result
    return self.__get_result()
  File "/usr/lib/python3.8/concurrent/futures/_base.py", line 389, in __get_result
    raise self._exception
  File "/usr/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/home/nicola/workspace/venv/lib/python3.8/site-packages/rdfizer/__init__.py", line 1539, in semantify_json
    i += semantify_json(triples_map, triples_map_list, delimiter, output_file_descriptor, sub_row, iterator.replace(new_iterator[:-1],""))
  File "/home/nicola/workspace/venv/lib/python3.8/site-packages/rdfizer/__init__.py", line 1901, in semantify_json
    if temp_data in join_table[triples_map_element.triples_map_id + "_" + predicate_object_map.object_map.child[0]] and temp_data != "":
KeyError: 'https://example.com/rdf/example-rml/example#hasSeverityCodeMapping_example:hasSeverityCode.termid'
eiglesias34 commented 1 year ago

Hi again,

I was able to make your example work. But there is one thing that is wrong with your mapping. When having parent triples maps in JSON, the triples map referred to as the parent must have the same iterator, or the child iterator must be a sub-string of the parent's iterator. The main reason for this is after reaching a certain level in the JSON, it becomes difficult to go back to the previous one. Just in case, I wrote to one of the authors of RML to make sure what I just said about the iterators is correct. The rest of the mapping works now.

I hope this help in your work, Sincerely, Enrique Iglesias

eiglesias34 commented 1 year ago

Hello again,

What I said regarding the iterators last time is not completely true. You can do parent triples maps between two triples maps with different iterators, but they need to be treated like a join between two different data sources at the level of the engine. That mistake is on me. Now, I must figure out how to do this in the engine.

Sincerely, Enrique Iglesias