Add a webservice for getting the statistics

[jschaeff]

First task for this is to build an API in the openAPI3 standard, for instance using the swagger online tools.

In order to imagine a suitable API, you can look at the matrix document. First 2 rows define the questions and the granularity level.

The code attached to this project needs a better documentation, I'm on it (see issue #11)

The datamodel is specified in the code : https://github.com/EIDA/eida-statistics/tree/main/backend_database

You can use this project to bring up your own empty database if needed.

You can create a directory for the webservice specification and implementation at the root of this project.

[jschaeff]

Query method

• more human oriented
• enables aggregation on all possible parameters. aggregate_on=all is a shortcut.
• CAVEEAT : nbclients can not be summed up across the responses
• output format in JSON with all the metadata

• output format in CSV also (maybe metadata as comment), default

  GET statistic/1/query?start=2021-01,end=2021-12,net=NL,sta=STA*,country=GR,FR&aggregate_on=all&format=json

   {
     "request_metadata": "",
     "matching": "net=NL,sta=STA*,country=GR,FR,start=2021-01,end=2021-12",
     "aggregated_on": ["month", "datacenter", "country", "net", ...],
     "values": [
      {
          "month": "*"
          "net": "NL",
          "sta": "STA*",
          "loc": "*",
          "cha": "*",
          "data_centre": "*",
          "country": "*",
          "bytes": 456684,
          "requests": 43,
          "successfull_requests": 30,
          "nbclients": 345 # This is an evaluation of the internal representation in the database
      }
    ]
  }

the aggregate_on parameter needs a clear explanation for the user.

GET statistic/1/query?start=2021-01,end=2021-12,net=NL,sta=STA*,country=GR,FR&aggregate_on=month,datacenter

{
   "request_metadata": "",
   "matching": "net=NL,sta=STA*,country=GR,FR,start=2021-01,end=2021-12",
   "aggregated_on": ["month", "datacenter", "country", "net", ...],
   "values": [
   {
    "month": "2021-01",
    "net": "NL",
    "sta": "STA*",
    "loc": "*",
    "cha": "*",
    "data_centre": "NOA",
    "country": "*",
    "bytes": 456684,
    "requests": 43,
    "successfull_requests": 30,
    "nbclients": 345 # This is an evaluation of the internal representation in the database
    },
    {
    "month": "2021-02",
    "net": "NL",
    "sta": "STA*",
    "loc": "*",
    "cha": "*",
    "data_centre": "RESIF",
    "country": "*",
    "bytes": 456684,
    "requests": 43,
    "successfull_requests": 30,
    "nbclients": 345 # This is an evaluation of the internal representation in the database
    }
   ]
}

Stats method

more robot oriented

GET statistic/1/dataselect_stats?start=2021-01,end=2021-12,net=NL,sta=STA*,country=GR,FR

{
  "version": "",
  "results": [
    {
    "month": "2021-01",
    "net": "NL",
    "sta": "STA1",
    "loc": "00",
    "cha": "HHZ",
    "datacentre": "NOA",
    "country": "GR",
    "bytes": 456684,
    "requests": 43,
    "successfull_requests": 30,
    "clients": "\x128b7fffffffff8ef137c60000000002832c9b" # hll object
    },
    ...
    ]
}

[jschaeff]

Today's review:

• add a version number in the API path
• add a /1/health method replying 200 if the service is up and running, database available
• rename dataselect_stats to dataselect (statistics/1/dataselect)

[jschaeff]

Webservice released in production. Thanks to the work of @vpet98