Documentation Pipelines

ghsnd commented 9 years ago

Hi,

Here's a first attempt to document the Pipelines service, which resides in the broker in eu.freme.broker.eservices.Pipelines. At the moment there is only one method.

There are no parameters because the input type is always application/json and the output type is determined by the last request in the chain. This means that the POST request must have a content-type: application/json header, and the body represents the chained requests to other services.

Example requests:

These are examples of the first 3 pipelines that should work. In theory a combination of every service is possible, as long as the next service can handle the output of the previous one.

e-Entity, e-Link

POST http://api.freme-project.eu/current/pipelining/chain
content-type: application/json

[
  {
    "type": "POST",
    "endpoint": "http://api.freme-project.eu/current/e-entity/dbpedia-spotlight/documents",
    "parameters": {
      "language": "en",
      "prefix": "http://freme-project.eu/"
    },
    "headers": {
      "content-type": "text/plain",
      "accept": "text/turtle"
    },
    "body": "This summer there is the Zomerbar in Antwerp, one of the most beautiful cities in Belgium."
  },
  {
    "type": "POST",
    "endpoint": "http://api.freme-project.eu/current/e-link/documents/",
    "parameters": {
      "templateid": "3",
      "prefix": "http://freme-project.eu/"
    },
    "headers": {
      "content-type": "text/turtle",
      "accept": "text/turtle"
    }
  }
]

e-Entity, e-Link, e-Translate

POST http://api.freme-project.eu/current/pipelining/chain
content-type: application/json

[
  {
    "type": "POST",
    "endpoint": "http://api.freme-project.eu/current/e-entity/dbpedia-spotlight/documents",
    "parameters": {
      "language": "en",
      "prefix": "http://freme-project.eu/"
    },
    "headers": {
      "content-type": "text/plain",
      "accept": "text/turtle"
    },
    "body": "The Belfry in Ghent is one of the oldest buildings in Belgium."
  },
  {
    "type": "POST",
    "endpoint": "http://api.freme-project.eu/current/e-link/documents/",
    "parameters": {
      "templateid": "3",
      "prefix": "http://freme-project.eu/"
    },
    "headers": {
      "content-type": "text/turtle",
      "accept": "text/turtle"
    }
  },
  {
    "type": "POST",
    "endpoint": "http://api.freme-project.eu/current/e-translation/tilde",
    "parameters": {
      "source-lang": "en",
      "target-lang": "fr",
      "prefix": "http://freme-project.eu/"
    },
    "headers": {
      "content-type": "text/turtle",
      "accept": "text/turtle"
    }
  }
]

e-Entity, e-Link, e-Terminology

POST http://api.freme-project.eu/current/pipelining/chain
content-type: application/json

[
  {
    "type": "POST",
    "endpoint": "http://api.freme-project.eu/current/e-entity/freme-ner/documents",
    "parameters": {
      "dataset": "dbpedia",
      "language": "en",
      "prefix": "http://freme-project.eu/"
    },
    "headers": {
      "content-type": "text/plain",
      "accept": "text/turtle"
    },
    "body": "With just 200,000 residents, Reykjavík ranks as one of Europe’s smallest capital cities. But when Iceland’s total population only hovers around 300,000, it makes sense that the capital is known as the “big city” and offers all the cultural perks of a much larger place.\n\n“From live music almost every night to cosy cafes, colourful houses and friendly cats roaming the street, Reykjavík has all the charms of a small town in a fun capital city,” said Kaelene Spence, "
  },
  {
    "type": "POST",
    "endpoint": "http://api.freme-project.eu/current/e-link/documents/",
    "parameters": {
      "templateid": "3",
      "prefix": "http://freme-project.eu/"
    },
    "headers": {
      "content-type": "text/turtle",
      "accept": "text/turtle"
    }
  },
  {
    "type": "POST",
    "endpoint": "http://api.freme-project.eu/current/e-terminology/tilde",
    "parameters": {
      "source-lang": "en",
      "target-lang": "nl",
      "prefix": "http://freme-project.eu/"
    },
    "headers": {
      "content-type": "text/turtle",
      "accept": "text/turtle"
    }
  }
]

Explanation of the body

The body is a JSON string. It represents a list of request objects, which are handled in that order. In the code, such a request object is a SerializedRequst.

The members of the object:

type: the type of the request. Possible values are GET or POST.
endpoint: the endpoint of the service.
parameters: a map of request parameters. Possible values depend on the service. Typical parameters are informat, outformat, input, etc.
headers: a map of request headers. Possible values depend on the service. Typical headers are content-type and accept.
body: The contents to send to the service. This also depends on the service of course.

Remark: Only the first request can have content, i.e. a body or a parameter input. This is because the output of the first request serves as input for the second, and so on. If there is content in a subsequent request, it will be ignored.

Note: There exist some classes in the package eu.freme.eservices.pipelines.requests to generate the body. Take a look at the RequestFactory and the RequestBuilder; the unit tests are also a good place to start.

Response

The response will be the response of the service of the last request in the chain, or an error response when something goes wrong.

ghsnd commented 9 years ago

Hi, Milan noticed that the parameter "prefix" is not used in e-Link (it is ignored). Since there is a default defined for "prefix" anyway, it might be better to remove all the "prefix": "http://freme-project.eu/" lines in the examples...

ghsnd commented 9 years ago

And I also changed "type" to "method", e.g. "method": "POST". Thus the examples need to be changed accordingly...

josauder commented 9 years ago

Closing for now, Documented as described above. Reopen if any issues appear

freme-project / freme-project.github.io