search

RMLio / rmlmapper-java

The RMLMapper executes RML rules to generate high quality Linked Data from multiple originally (semi-)structured data sources
http://rml.io
MIT License
146 stars 61 forks source link

NormalizeDate: idlab-fn:normalizeDateTime doesn't work with `X` pattern, grel:toDate doesn't work with `yyyy-MM-dd HH:mm:ss'+00:00'`, #184

Open bblfish opened 2 years ago

bblfish commented 2 years ago

I have the following csv data where the second column is close to an xsd:dateTimeStamp format.

rowId,timestamp,data_source,index,modality,count,locationrange,speed,measurement_type,refRoadSegment
2880,2021-09-30 22:00:00+00:00,cropland,schoolstraat,,2865.9627,"POLYGON ((4.472119613057031 51.0207367817    8503, 4.479745501704901 51.02015981281664, 4.481188984378538 51.01915526700179, 4.481395603399528 51.01491    440180675, 4.482127375233162 51.0126494593307, 4.478332541440828 51.00987874451822, 4.471306586318185 51.0    0931972134209, 4.469785136941451 51.01107192511326, 4.461581573935097 51.01126112376583, 4.460354540525603     51.01217411043578, 4.46032051949892 51.01700860830363, 4.468571374013287 51.01807235138044, 4.47211961305    7031 51.02073678178503))",,,
2881,2021-09-30 22:15:00+00:00,cropland,schoolstraat,,3788.4589,"POLYGON ((4.472119613057031 51.0207367817    8503, 4.479745501704901 51.02015981281664, 4.481188984378538 51.01915526700179, 4.481395603399528 51.01491    440180675, 4.482127375233162 51.0126494593307, 4.478332541440828 51.00987874451822, 4.471306586318185 51.0    0931972134209, 4.469785136941451 51.01107192511326, 4.461581573935097 51.01126112376583, 4.460354540525603     51.01217411043578, 4.46032051949892 51.01700860830363, 4.468571374013287 51.01807235138044, 4.47211961305    7031 51.02073678178503))",,,

To map the second column I have written the following mapping rule

@base <http://example.org/> .  ## see issue https://github.com/RMLio/rmlmapper-java/issues/178

@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .
@prefix skos: <http://www.w3.org/2004/02/skos/core#> .
@prefix rml: <http://semweb.mmlab.be/ns/rml#> .
@prefix rr: <http://www.w3.org/ns/r2rml#> .
@prefix sosa: <http://www.w3.org/ns/sosa/> .
@prefix fnml: <http://semweb.mmlab.be/ns/fnml#> .
@prefix grel: <http://users.ugent.be/~bjdmeest/function/grel.ttl#> .
@prefix fno: <https://w3id.org/function/ontology#> .

<#allDataMap> rr:predicateObjectMap [
    rr:predicate sosa:resultTime ;
    rr:objectMap [
        rr:datatype xsd:dateTimeStamp;
        fnml:functionValue [
                rr:predicateObjectMap [
                    rr:predicate fno:executes ;
                    rr:objectMap [ rr:constant grel:date_toDate ]
                ];
                rr:predicateObjectMap [
                    rr:predicate  grel:valueParameter;
                    rr:objectMap [ rml:reference "timestamp"]
                ];
                rr:predicateObjectMap [
                    rr:predicate grel:p_string_pattern;
                    rr:objectMap [ rr:constant "yyyy-MM-dd HH:mm:ssX"];
                ];
            ];
       ];
] .

Given the other turtle files, which I will append to this report I get the following output. It seems to be able to find the right function and the apply the right methods, but it does not seem to understand how to map a Date object to return an xsd:dateTimeStamp (or any other value) and so the result is empty.

$ rmlmapper   -m awv.sources.ttl -m column1.ttl -m column2.v2.ttl -v |& tail
10:05:55.403 [main] DEBUG b.u.i.k.f.a.f.Instantiator          .getMethod(66) - Getting instantiation for http://users.ugent.be/~bjdmeest/function/grel.ttl#date_toDate
10:05:55.403 [main] DEBUG b.u.i.k.f.a.f.Instantiator          .getClass(110) - Trying to find a Class for io.fno.grel.DateFunctions
10:05:55.403 [main] DEBUG b.u.i.k.f.a.f.Instantiator          .getMethod(79) - Found class class io.fno.grel.DateFunctions
10:05:55.403 [main] DEBUG b.u.i.k.f.a.f.Instantiator          .getMethod(165) - Trying to find method with name toDate
10:05:55.403 [main] DEBUG b.u.i.k.f.a.f.Instantiator          .getMethod(172) - Found method with matching name toDate and matching parameter count (2)
10:05:55.403 [main] DEBUG b.u.i.k.f.a.f.Instantiator          .getMethod(222) - Found method by name and expected arguments. Checking return type...
10:05:55.403 [main] INFO  b.u.i.k.f.a.d.DefaultDataTypeConverter.isSuperTypeOf(27) - Omitting supertype check in DefaultTypeConverter
10:05:55.403 [main] DEBUG b.u.i.k.f.a.f.Instantiator          .getMethod(225) - Found method!
10:05:55.403 [main] DEBUG b.u.i.k.f.a.f.Instantiator          .getMethod(87) - Found method toDate
10:05:55.404 [main] ERROR be.ugent.rml.cli.Main               .main(404) - null

This seems somewhat similar to issue https://github.com/RMLio/rmlmapper-java/issues/142

all data and code: example.tar.gz

bjdmeest commented 2 years ago

Hmmm, for some reason grel:toDate doesn't work. However, we have another function that should do smth similar, however it doesn't seem to handle the X pattern too well, the RML below seems to work at my end:

@base <http://example.org/> .  ## see issue https://github.com/RMLio/rmlmapper-java/issues/178

@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .
@prefix skos: <http://www.w3.org/2004/02/skos/core#> .
@prefix rml: <http://semweb.mmlab.be/ns/rml#> .
@prefix rr: <http://www.w3.org/ns/r2rml#> .
@prefix sosa: <http://www.w3.org/ns/sosa/> .
@prefix fnml: <http://semweb.mmlab.be/ns/fnml#> .
@prefix grel: <http://users.ugent.be/~bjdmeest/function/grel.ttl#> .
@prefix fno: <https://w3id.org/function/ontology#> .
@prefix idlab-fn: <http://example.com/idlab/function/> .

<#allDataMap> rr:predicateObjectMap [
    rr:predicate sosa:resultTime ;
    rr:objectMap [
        rr:datatype xsd:dateTimeStamp;
        fnml:functionValue [
                rr:predicateObjectMap [
                    rr:predicate fno:executes ;
                    rr:objectMap [ rr:constant idlab-fn:normalizeDateTime ]
                ];
                rr:predicateObjectMap [
                    rr:predicate  idlab-fn:strDate;
                    rr:objectMap [ rml:reference "timestamp"]
                ];
                rr:predicateObjectMap [
                    rr:predicate idlab-fn:pattern;
                    rr:objectMap [ rr:constant "yyyy-MM-dd HH:mm:ss'+00:00'"];
                ];
            ];
       ];
] .
bblfish commented 2 years ago

Thanks for the help. I saw test cases using idlab-fn , but as the ontology is on the domain example.com I was not sure if that should be used. Perhaps it would be worth publishing those ontologies with descriptions of how they work at a place you own?

DylanVanAssche commented 1 year ago

The idlab-fn are functions used for internal purposes and research projects, they are not standardized in any case and may change at any time, that's why they are not published.

namedgraph commented 1 year ago

@bjdmeest fn:normalizeDate does not work either:

ERROR be.ugent.knows.idlabFunctions.IDLabFunctions  - Text '2016-07' could not be parsed: Unable to obtain LocalDate from TemporalAccessor: {MonthOfYear=7, Year=2016},ISO of type java.time.format.Parsed; format pattern: "yyyy-MM", input: "2016-07", language: "en"