search

linkedpipes / etl

LinkedPipes ETL is an RDF based, lightweight ETL tool
https://etl.linkedpipes.com
Other
147 stars 30 forks source link

e-httpGetList: Detailed log is not represented in Reports #508

Closed jakubklimek closed 6 years ago

jakubklimek commented 7 years ago

When the Detailed log is enabled, HTTP headers are logged in the log. This is, however, not machine processable. The detailed information should also be reported in Reports.

jakubklimek commented 6 years ago

The reports now generate multiple entries that get mixed up:

<http://www.volby.cz/opendata/se2016/SE2016_XML.pdf/report> a <https://vocabulary.etl.linkedpipes.com/report/Report> ;
    <https://vocabulary.etl.linkedpipes.com/report/duration> "3208"^^<http://www.w3.org/2001/XMLSchema#long> ;
    <https://vocabulary.etl.linkedpipes.com/report/end> "2017-11-23T11:59:52.587"^^<http://www.w3.org/2001/XMLSchema#datetime> ;
    <https://vocabulary.etl.linkedpipes.com/report/start> "2017-11-23T11:59:49.379"^^<http://www.w3.org/2001/XMLSchema#datetime> ;
    <https://vocabulary.etl.linkedpipes.com/report/status> <https://vocabulary.etl.linkedpipes.com/report/Success> ;
    <https://vocabulary.etl.linkedpipes.com/report/task> <http://www.volby.cz/opendata/se2016/SE2016_XML.pdf> .

<http://www.volby.cz/opendata/se2016/SE2016_XML.pdf/report> <http://plugins.linkedpipes.com/ontology/e-httpGetFiles#errorMessage> "200"^^<http://www.w3.org/2001/XMLSchema#int> .

<http://www.volby.cz/opendata/se2016/SE2016_XML.pdf/report> <http://plugins.linkedpipes.com/ontology/e-httpGetFiles#errorMessage> "302"^^<http://www.w3.org/2001/XMLSchema#int> ;
    <http://plugins.linkedpipes.com/ontology/e-httpGetFiles#response> <http://www.volby.cz/opendata/se2016/SE2016_XML.pdf/response> .

<http://www.volby.cz/opendata/se2016/SE2016_XML.pdf/report/header/1> <http://plugins.linkedpipes.com/ontology/e-httpGetFiles#fileName> "Cache-Control" ;
    <http://plugins.linkedpipes.com/ontology/e-httpGetFiles#value> "no-cache" .

<http://www.volby.cz/opendata/se2016/SE2016_XML.pdf/report/header/2> <http://plugins.linkedpipes.com/ontology/e-httpGetFiles#fileName> "Connection" ;
    <http://plugins.linkedpipes.com/ontology/e-httpGetFiles#value> "close" .

<http://www.volby.cz/opendata/se2016/SE2016_XML.pdf/report/header/3> <http://plugins.linkedpipes.com/ontology/e-httpGetFiles#fileName> "Pragma" ;
    <http://plugins.linkedpipes.com/ontology/e-httpGetFiles#value> "no-cache" .

<http://www.volby.cz/opendata/se2016/SE2016_XML.pdf/report/header/4> <http://plugins.linkedpipes.com/ontology/e-httpGetFiles#fileName> "Location" ;
    <http://plugins.linkedpipes.com/ontology/e-httpGetFiles#value> "https://www.volby.cz/opendata/se2016/SE2016_XML.pdf" .

<http://www.volby.cz/opendata/se2016/SE2016_XML.pdf/response> <http://plugins.linkedpipes.com/ontology/e-httpGetFiles#hasHeaderObject> <http://www.volby.cz/opendata/se2016/SE2016_XML.pdf/report/header/1> , <http://www.volby.cz/opendata/se2016/SE2016_XML.pdf/report/header/2> , <http://www.volby.cz/opendata/se2016/SE2016_XML.pdf/report/header/3> , <http://www.volby.cz/opendata/se2016/SE2016_XML.pdf/report/header/4> ;
    <http://plugins.linkedpipes.com/ontology/e-httpGetFiles#responseLine> "HTTP/1.1 302 Found : Moved Temporarily" .

Notice the two errorMessages.

And why does the header name use e-httpGetFiles#fileName predicate?