Create Status Endpoint

[jag3773]

Story

As a publisher I want to know if there are problems with resources that have been pushed into the pipeline so that I know if they are actually in the catalog or not.

Notes

The intention is to have a /status endpoint which returns build status information in JSON format. We should get a status indication for each catalog: 3, uw.2, ts.2. Further, if there are errors in the errors table those should also be provided in the response.

First task is to fill out the specification below on what we can easily make happen.

Desired JSON Output Specification

[da1nerd]

@jag3773 how is this for the endpoint output?

{
    "status": {
        "3": "complete",
        "uw.2": "incomplete",
        "ts.2": "in-progress"
    },
    "timestamp": "2017-10-20T17:27:53.578223+00:00",
    "errors": {
        "webhook": [],
        "ts_v2_catalog": [{
            "message": "some error",
            "timestamp": "2017-10-20T17:27:53.578223+00:00",
            "retries": 2
        }],
        "uw_v2_catalog": [],
        "catalog": [],
        "signing": []
    }
}

[jag3773]

@neutrinog Would it be easy enough to pivot the data so each catalog is at the top level?

For example:

{
  "catalogs": [
    {
      "name": "3",
      "status": "complete",
      "timestamp": "2017-10-20T17:27:53.578223+00:00",
      "errors": "none"
    },
    {
      "name": "uw.2",
      "status": "incomplete",
      "timestamp": "2017-10-20T17:27:53.578223+00:00",
      "errors": "none"
    },
    {
      "name": "ts.2",
      "status": "in-progress",
      "timestamp": "2017-10-20T17:27:53.578223+00:00",
      "errors": [
        {
          "message": "some error",
          "timestamp": "2017-10-20T17:27:53.578223+00:00",
          "retries": 2
        }
      ]
    }
  ]
}

[da1nerd]

we could. that's how I was originally going to do it. however we have two lambdas that don't fit that structure: signing, webhook.

We could follow your structure and then add a global errors section that contains errors for those two lambdas.

[jag3773]

What status information do we have for those extra 2 lambdas? If we renamed catalogs to functions would that suffice?

[da1nerd]

We don't have any status information for those two just errors. We collect status for the three APIs and report errors for five lambdas.

technically we could place webhook errors under the v3 catalog errors since everything first goes through the v3 api. However signing is used by all three and we don't currently have a way to determine which api it's running for.

[da1nerd]

we could mock the status for all of the lambdas. So the webhook and signing lambda would just have a status of either ok or error depending on the presence of errors. Then we can name catalogs to functions like you suggested. And the status for the three apis could also be simplified to ok or error. (complete or in-progress => ok)

[da1nerd]

@jag3773 what about this? I removed the status field since we can infer the status from the existence of errors. This does not indicate if a catalog is currently being built. Do we need that information?

{
  "functions": [
    {
      "name": "3",
      "lambda": "catalog",
      "timestamp": "2017-10-20T17:27:53.578223+00:00",
      "errors": []
    },
    {
      "name": "uw.2",
      "lambda": "uw_v2_catalog",
      "timestamp": "2017-10-20T17:27:53.578223+00:00",
      "errors": []
    },
    {
      "name": "ts.2",
      "lambda": "ts_v2_catalog",
      "timestamp": "2017-10-20T17:27:53.578223+00:00",
      "errors": [
        {
          "message": "some error",
          "timestamp": "2017-10-20T17:27:53.578223+00:00",
          "retries": 2
        }
      ]
    },
    {
      "name": "signing",
      "lambda": "signing",
      "timestamp": "2017-10-20T17:27:53.578223+00:00",
      "errors": []
    },
    {
      "name": "webhook",
      "lambda": "webhook",
      "timestamp": "2017-10-20T17:27:53.578223+00:00",
      "errors": []
    }
  ]
}

[jag3773]

@neutrinog Yes, I would like a status field as that makes creating a monitoring check a lot easier.

[jag3773]

I'd vote to mock the status field as "complete". That way I can create a simple check that errors if any of the function status fields != "complete".

[da1nerd]

can you provide a json sample?

[jag3773]

{
  "functions": [
    {
      "name": "3",
      "status": "complete",
      "lambda": "catalog",
      "timestamp": "2017-10-20T17:27:53.578223+00:00",
      "errors": []
    },
    {
      "name": "uw.2",
      "status": "complete",
      "lambda": "uw_v2_catalog",
      "timestamp": "2017-10-20T17:27:53.578223+00:00",
      "errors": []
    },
    {
      "name": "ts.2",
      "status": "in-progress",
      "lambda": "ts_v2_catalog",
      "timestamp": "2017-10-20T17:27:53.578223+00:00",
      "errors": [
        {
          "message": "some error",
          "timestamp": "2017-10-20T17:27:53.578223+00:00",
          "retries": 2
        }
      ]
    },
    {
      "name": "signing",
      "status": "complete",
      "lambda": "signing",
      "timestamp": "2017-10-20T17:27:53.578223+00:00",
      "errors": []
    },
    {
      "name": "webhook",
      "status": "complete",
      "lambda": "webhook",
      "timestamp": "2017-10-20T17:27:53.578223+00:00",
      "errors": []
    }
  ]
}

[da1nerd]

ah ok. like we had originally but the signing and webhook functions will always be "complete".

[da1nerd]

@jag3773 what do you think? https://dev-api.door43.org/v3/lambda/status

[jag3773]

Perfect!

[da1nerd]

It's been deployed to production.

[jag3773]

Great, looks good.