search

Informatievlaanderen / VSDS-Linked-Data-Interactions

https://informatievlaanderen.github.io/VSDS-Linked-Data-Interactions/
European Union Public License 1.2
4 stars 7 forks source link

Bug new rdf-writer in be.vlaanderen.informatievlaanderen.ldes.ldio.LdioHttpOut #380

Closed kselin closed 1 year ago

kselin commented 1 year ago

Describe the bug Output is defined like below but still get following error:

ldio-workbench  | Exception in thread "pool-4-thread-20" java.lang.NullPointerException: Cannot invoke "String.length()" because "string" is null
ldio-workbench  |       at org.apache.jena.atlas.json.io.JSWriter.outputQuotedString(JSWriter.java:193)
ldio-workbench  |       at org.apache.jena.atlas.json.io.JSWriter.outputQuotedString(JSWriter.java:183)
ldio-workbench  |       at org.apache.jena.atlas.json.io.JsonWriter.visit(JsonWriter.java:221)
ldio-workbench  |       at org.apache.jena.atlas.json.JsonString.visit(JsonString.java:36)
ldio-workbench  |       at org.apache.jena.atlas.json.io.JsonWriter.writeObjectLong(JsonWriter.java:118)
ldio-workbench  |       at org.apache.jena.atlas.json.io.JsonWriter.visit(JsonWriter.java:71)
ldio-workbench  |       at org.apache.jena.atlas.json.JsonObject.visit(JsonObject.java:39)
ldio-workbench  |       at org.apache.jena.atlas.json.JsonValue.output(JsonValue.java:67)
ldio-workbench  |       at org.apache.jena.atlas.json.JsonValue.toString(JsonValue.java:58)
ldio-workbench  |       at be.vlaanderen.informatievlaanderen.ldes.ldi.rdf.formatter.JsonLdWriter.getFramedContext(JsonLdWriter.java:39)
ldio-workbench  |       at be.vlaanderen.informatievlaanderen.ldes.ldi.rdf.formatter.JsonLdWriter.write(JsonLdWriter.java:25)
ldio-workbench  |       at be.vlaanderen.informatievlaanderen.ldes.ldio.LdioHttpOut.accept(LdioHttpOut.java:36)
ldio-workbench  |       at be.vlaanderen.informatievlaanderen.ldes.ldio.LdioHttpOut.accept(LdioHttpOut.java:18)
ldio-workbench  |       at be.vlaanderen.informatievlaanderen.ldes.ldio.services.LdiSender.lambda$accept$0(LdiSender.java:34)
ldio-workbench  |       at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
ldio-workbench  |       at java.base/java.util.AbstractList$RandomAccessSpliterator.forEachRemaining(AbstractList.java:720)
ldio-workbench  |       at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
ldio-workbench  |       at java.base/java.util.stream.ForEachOps$ForEachTask.compute(ForEachOps.java:290)
ldio-workbench  |       at java.base/java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:754)
ldio-workbench  |       at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373)
ldio-workbench  |       at java.base/java.util.concurrent.ForkJoinTask.invoke(ForkJoinTask.java:686)
ldio-workbench  |       at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateParallel(ForEachOps.java:159)
ldio-workbench  |       at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateParallel(ForEachOps.java:173)
ldio-workbench  |       at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:233)
ldio-workbench  |       at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596)
ldio-workbench  |       at java.base/java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:765)
ldio-workbench  |       at be.vlaanderen.informatievlaanderen.ldes.ldio.services.LdiSender.accept(LdiSender.java:34)
ldio-workbench  |       at be.vlaanderen.informatievlaanderen.ldes.ldio.services.LdiSender.accept(LdiSender.java:18)
ldio-workbench  |       at java.base/java.lang.Iterable.forEach(Iterable.java:75)
ldio-workbench  |       at be.vlaanderen.informatievlaanderen.ldes.ldio.services.ComponentExecutorImpl.lambda$transformLinkedData$1(ComponentExecutorImpl.java:32)
ldio-workbench  |       at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
ldio-workbench  |       at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
ldio-workbench  |       at java.base/java.lang.Thread.run(Thread.java:833)

outputs:
        - name: be.vlaanderen.informatievlaanderen.ldes.ldio.LdioHttpOut
          config:
            endpoint: https://webhook.site/2b3799e0-95a8-4731-af0b-8751598991a1
            # content-type: application/ld+json
            rdf-writer:
              content-type: application/ld+json
              frame-type: http://def.isotc211.org/iso19156/2011/Observation#OM_Observation

Expected behavior Expect to see framed json-ld output.

Full config

orchestrator:
  pipelines:
    - name: observations
      description: "This pipeline will take data from crowdscan ldes and send to kafka"
      input:
        name: be.vlaanderen.informatievlaanderen.ldes.ldi.client.LdioLdesClient
        config:
          url: https://test.crowdscan.be/ldes-beacon/observations/by-page-1w?pageNumber=1/2
          sourceFormat: text/turtle
          retries:
            enabled: true
      transformers:
        - name: be.vlaanderen.informatievlaanderen.ldes.ldi.SparqlConstructTransformer
          config:
            query: |
              PREFIX terms: <http://purl.org/dc/terms/>
              PREFIX ngsi-ld: <https://uri.etsi.org/ngsi-ld/>
              PREFIX OM: <http://def.isotc211.org/iso19156/2011/Observation#>
              PREFIX time: <http://www.w3.org/2006/time#>
              PREFIX sosa: <http://www.w3.org/ns/sosa/> 
              PREFIX seb: <https://data.vlaanderen.be/ns/sensoren-en-bemonstering#>
              PREFIX schema: <http://schema.org/>
              PREFIX prov: <http://www.w3.org/ns/prov#>
              CONSTRUCT {
                ?obs a OM:OM_Observation ;
                OM:OM_Observation.featureOfInterest [
                  a ngsi-ld:Relationship;
                  ngsi-ld:hasObject ?featureOfInterest
                ] ;
                OM:OM_Observation.observedProperty  [
                  a ngsi-ld:Property;
                  ngsi-ld:hasValue ?observedProperty
                ] ;
                OM:OM_Observation.phenomenonTime  [
                  a ngsi-ld:Property ;
                  ngsi-ld:hasValue [
                    a time:Interval ;
                    time:hasEnd ?endTime ;
                  time:hasXSDDuration ?duration
                  ]
                ] ;
                OM:OM_Observation.procedure  [
                  a ngsi-ld:Property;
                  ngsi-ld:hasValue [
                    a sosa:Procedure ;
                    seb:Observatieprocedure.specificatie ?procedure
                  ]
                ] ;
                OM:OM_Observation.value  [
                    a ngsi-ld:Property;
                    ngsi-ld:hasValue [
                      a schema:QuantitativeValue ;
                      schema:value ?value
                  ]
                ] ;
                OM:OM_Observation.resultTime  [
                  a ngsi-ld:Property;
                  ngsi-ld:hasValue ?resultTime
                ] ;
                prov:generatedAtTime [
                  a ngsi-ld:Property;
                  ngsi-ld:hasValue ?generatedAtTime
                ] 
              }
              WHERE {
                ?versieObs a OM:OM_Observation ;
                terms:isVersionOf ?obs ; 
                OM:OM_Observation.featureOfInterest ?featureOfInterest ;
                OM:OM_Observation.observedProperty ?observedProperty ;
                OM:OM_Observation.phenomenonTime/time:hasEnd/time:inXSDDateTimeStamp ?endTime ;
                OM:OM_Observation.phenomenonTime/time:hasXSDDuration ?duration ;
                OM:OM_Observation.procedure/seb:Observatieprocedure.specificatie ?procedure ;
                OM:OM_Observation.result/schema:value ?value ;
                OM:OM_Observation.resultTime ?resultTime ;
                prov:generatedAtTime ?generatedAtTime .
              }
      outputs:
        - name: be.vlaanderen.informatievlaanderen.ldes.ldio.LdioHttpOut
          config:
            endpoint: https://webhook.site/2b3799e0-95a8-4731-af0b-8751598991a1
            # content-type: application/ld+json
            rdf-writer:
              content-type: application/ld+json
              frame-type: http://def.isotc211.org/iso19156/2011/Observation#OM_Observation

Docker-compose file

services:
  ldio-workbench:
    container_name: ldio-workbench
    image: ldio-framing (image from latest version develop branch)
    ports:
      - 8083:8083
    volumes:
      - ./ldio-config.yml:/ldio/application.yml:ro
Yalz commented 1 year ago

This issue is now fixed as part of the latest LDIO image