search

linkedpipes / etl

LinkedPipes ETL is an RDF based, lightweight ETL tool
https://etl.linkedpipes.com
Other
144 stars 30 forks source link

Debug from broken #621

Closed jakubklimek closed 5 years ago

jakubklimek commented 5 years ago

See https://etl.dev.nkod.opendata.cz/#/executions

2018-10-16 06:12:06,771 [web-server-1810] INFO  c.l.e.e.e.ExecutionObserver - onExecutionBegin
2018-10-16 06:12:06,838 [asynchExecutor-1] DEBUG o.e.r.s.n.NativeStore - Initializing NativeStore...
2018-10-16 06:12:06,840 [asynchExecutor-1] DEBUG o.e.r.s.n.NativeStore - Data dir is /data/lp/etl/working/data/1539663126620-13-d13a8ce7-b6f3-4e17-b35a-21648e076063/working/pipeline_repository-1
2018-10-16 06:12:06,905 [asynchExecutor-1] DEBUG o.e.r.s.n.TripleStore - No indexes specified, using default indexes: spoc,posc
2018-10-16 06:12:06,906 [asynchExecutor-1] DEBUG o.e.r.s.n.NativeStore - NativeStore initialized
2018-10-16 06:12:07,564 [asynchExecutor-1] INFO  c.l.e.e.e.ExecutionObserver - onPipelineLoaded
2018-10-16 06:12:07,756 [asynchExecutor-1] INFO  c.l.e.d.c.r.RepositoryManager - Repository policy: http://linkedpipes.com/ontology/repository/PerInputRepository
2018-10-16 06:12:07,756 [asynchExecutor-1] INFO  c.l.e.d.c.r.RepositoryManager - Repository type: http://linkedpipes.com/ontology/repository/NativeStore
2018-10-16 06:12:07,756 [asynchExecutor-1] INFO  c.l.e.d.c.r.RepositoryManager - Directory: /data/lp/etl/working/data/1539663126620-13-d13a8ce7-b6f3-4e17-b35a-21648e076063/working/working-sesame-369
2018-10-16 06:12:07,910 [asynchExecutor-1] INFO  c.l.e.e.e.ExecutionObserver - onMapComponentBegin : https://etl.dev.nkod.opendata.cz/resources/pipelines/1539609073926/105
2018-10-16 06:12:07,915 [asynchExecutor-1] DEBUG c.l.e.e.d.DataUnitContainer - map by reference: https://etl.dev.nkod.opendata.cz/resources/pipelines/1539609073926/105/port/Configuration from /data/lp/etl/working/data/1539609336814-11-7c477d52-554f-4b2e-9b3d-cba216d082d8/working/dataunit-000-2
2018-10-16 06:12:07,917 [asynchExecutor-1] DEBUG c.l.e.e.d.DataUnitContainer - map by reference: https://etl.dev.nkod.opendata.cz/resources/pipelines/1539609073926/105/port/FilesOutput from /data/lp/etl/working/data/1539609336814-11-7c477d52-554f-4b2e-9b3d-cba216d082d8/working/dataunit-001-3
2018-10-16 06:12:07,918 [asynchExecutor-1] INFO  c.l.e.e.e.ExecutionObserver - onMapComponentSuccessful : https://etl.dev.nkod.opendata.cz/resources/pipelines/1539609073926/105
2018-10-16 06:12:07,923 [asynchExecutor-1] INFO  c.l.e.e.e.ExecutionObserver - onMapComponentBegin : https://etl.dev.nkod.opendata.cz/resources/pipelines/1539609073926/111
2018-10-16 06:12:07,929 [asynchExecutor-1] DEBUG c.l.e.e.d.DataUnitContainer - map by reference: https://etl.dev.nkod.opendata.cz/resources/pipelines/1539609073926/111/port/Configuration from /data/lp/etl/working/data/1539609336814-11-7c477d52-554f-4b2e-9b3d-cba216d082d8/working/dataunit-002-4
2018-10-16 06:12:07,930 [asynchExecutor-1] DEBUG c.l.e.e.d.DataUnitContainer - load from file: https://etl.dev.nkod.opendata.cz/resources/pipelines/1539609073926/111/port/Metadata
2018-10-16 06:12:07,931 [asynchExecutor-1] ERROR c.l.e.e.e.ExecutionObserver - onMapComponentFailed : https://etl.dev.nkod.opendata.cz/resources/pipelines/1539609073926/111
com.linkedpipes.etl.executor.ExecutorException: Can't bindToPipeline data unit: https://etl.dev.nkod.opendata.cz/resources/pipelines/1539609073926/111/port/Metadata
    at com.linkedpipes.etl.executor.dataunit.DataUnitContainer.initialize(DataUnitContainer.java:99)
    at com.linkedpipes.etl.executor.dataunit.DataUnitManager.onComponentMapByReference(DataUnitManager.java:122)
    at com.linkedpipes.etl.executor.component.MapComponent.execute(MapComponent.java:29)
    at com.linkedpipes.etl.executor.pipeline.PipelineExecutor.executeComponent(PipelineExecutor.java:274)
    at com.linkedpipes.etl.executor.pipeline.PipelineExecutor.executeComponents(PipelineExecutor.java:248)
    at com.linkedpipes.etl.executor.pipeline.PipelineExecutor.execute(PipelineExecutor.java:84)
    at com.linkedpipes.etl.executor.web.servlet.ExecutionServlet.lambda$execute$0(ExecutionServlet.java:66)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1135)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
    at java.base/java.lang.Thread.run(Thread.java:844)
Caused by: com.linkedpipes.etl.executor.api.v1.LpException: Can't execute repository procedure.
    at com.linkedpipes.etl.dataunit.core.rdf.ActionExecutor.execute(ActionExecutor.java:24)
    at com.linkedpipes.etl.dataunit.core.rdf.BaseRdf4jDataUnit.execute(BaseRdf4jDataUnit.java:35)
    at com.linkedpipes.etl.dataunit.core.rdf.DefaultSingleGraphDataUnit.loadContentFromTurtle(DefaultSingleGraphDataUnit.java:200)
    at com.linkedpipes.etl.dataunit.core.rdf.DefaultSingleGraphDataUnit.initialize(DefaultSingleGraphDataUnit.java:73)
    at com.linkedpipes.etl.executor.dataunit.DataUnitContainer.initialize(DataUnitContainer.java:96)
    ... 9 common frames omitted
Caused by: java.lang.NullPointerException: null
    at com.linkedpipes.etl.dataunit.core.rdf.ActionExecutor.execute(ActionExecutor.java:18)
    ... 13 common frames omitted
2018-10-16 06:12:07,936 [asynchExecutor-1] INFO  c.l.e.d.c.r.RepositoryManager - Closing all remaining repositories.
2018-10-16 06:12:07,936 [asynchExecutor-1] INFO  c.l.e.d.c.r.RepositoryManager - Deleting content ...
2018-10-16 06:12:07,936 [asynchExecutor-1] INFO  c.l.e.d.c.r.RepositoryManager - Deleting content ... done
2018-10-16 06:12:07,965 [asynchExecutor-1] ERROR c.l.e.d.c.r.RepositoryManager - Manager was asked to close unmanaged repository.
2018-10-16 06:12:07,965 [asynchExecutor-1] ERROR c.l.e.d.c.r.RepositoryManager - Manager was asked to close unmanaged repository.
2018-10-16 06:12:07,965 [asynchExecutor-1] DEBUG o.e.r.s.n.NativeStore - Shutting down NativeStore...
2018-10-16 06:12:07,965 [asynchExecutor-1] DEBUG o.e.r.s.n.NativeStore - NativeStore shut down
2018-10-16 06:12:07,966 [asynchExecutor-1] DEBUG o.e.r.s.n.NativeStore - NativeStore shut down
2018-10-16 06:12:07,966 [asynchExecutor-1] INFO  c.l.e.e.e.ExecutionObserver - onExecutionEnd
skodapetr commented 5 years ago

What exactly should be visible at the instance?

jakubklimek commented 5 years ago

You can see the failed executions there. However, you can try any instance yourself. Just try debug from, it fails.