[Pipelines] Pipeline and intype=url

jnehring commented 8 years ago

This pipeline does not work:

[{
        "method": "POST",
        "endpoint": "http://api.freme-project.eu/current/toolbox/nif-converter/",
        "parameters": {
            "intype": "url",
            "input" : "https://www.theguardian.com/world/2009/oct/27/berlin-wall-short-history"
        },
        "headers": {
            "content-type": "text/html",
            "accept": "text/turtle"
        }
    }, 
    {
        "method": "POST",
        "endpoint": "http://api.freme-project.eu/current/e-entity/freme-ner/documents/",
        "parameters": {
            "language": "en",
            "dataset" : "dbpedia"
        },
        "headers": {
            "content-type": "text/turtle",
            "accept": "text/html"
        }
    }

]

It causes an Internal Server Error.

ArneBinder commented 8 years ago

The reason is, that roundtripping in the pipeline service requires any content to convert to turtle, but the body of the first request is empty.

jnehring commented 8 years ago

Lets leave this bug open, currently we have more important things to do.

ArneBinder commented 7 years ago

You can get the urlfilter functionality (also in combination with roundtripping) when you store a pipeline which handles just turtle and call it via pipelining/chain/{id} with the request parameters intype=url and input={YOUR_URL}. Furthermore, you can set the accept header to text/html to get roundtripping funcitonaility. The same holds for #89.

Try adding this pipeline:

[
{
  "method": "POST",
  "endpoint": "https://api-dev.freme-project.eu/current/e-entity/freme-ner/documents",
  "parameters": {
     "dataset": "dbpedia", 
    "language": "en"
  }
},
{
  "method": "POST",
  "endpoint": "https://api-dev.freme-project.eu/current/e-link/documents/",
  "parameters": {
    "templateid": "1"
  }
}
]

and call it with (do not forget to set the pipeline id):

curl -X POST -H "Accept: text/html" -H "Content-Type: text/html" -d '' "http://api-dev.freme-project.eu/current/pipelining/chain/{THE_NEW_PIPELINE_ID}?intype=url&input=http://beautifulberlin.blogspot.de/2009/07/short-history-of-berlin.html"

Perhaps, this should be mentioned in the section Using a Pipeline of the FREME pipelines article.

freme-project / basic-services

[Pipelines] Pipeline and intype=url #88