Icinga / icinga-notifications-web

Icinga Notifications Web β€” Manage incidents and who gets notified about them how and when
GNU General Public License v2.0
11 stars 0 forks source link

Add an HTTP API to configure contacts and contactgroups #176

Open nilmerg opened 5 months ago

nilmerg commented 5 months ago

Current State

At the moment contacts can only be configured by using the UI. Contactgroups cannot be configured at all. (see #174)

Problem

We can safely assume that consumers already have their users and usergroups defined somewhere. Maintaining them again for Icinga Notifications shouldn't be necessary.

Solution

We should provide a way to automate their creation. The easiest way is to provide a basic HTTP API to do so:

Authorization

The API must require the notifications/api/v1 permission.

Endpoints

Users: notifications/api/v1/contacts[/<identifier> | ?<filter>] Groups: notifications/api/v1/contactgroups[/<identifier> | ?<filter>]

Parameters

identifier This is a UUID. For resources created in the UI, this is a UUIDv4.

filter A usual filter query string

Request Body Validity

Resource Structure (Request and Response)

Contact

{
    id: identifier,
    full_name: string,
    username?: string,
    default_channel: string,
    groups?: identifier[],
    addresses?: {}
}

Contactgroup

{
    id: identifier,
    name: string,
    users?: identifier[]
}

Methods

GET

POST

PUT

DELETE

Schema Changes

Implementation Requirements

julianbrost commented 5 months ago

User

{
    full_name: string,
    username: identifier,
    default_channel: string,
    groups?: identifier[]
}

Group

{
    name: identifier,
    users?: identifier[]
}

So there will be two places where group memberships can be updated? Will the behavior be that on an update, memberships that aren't given, will be removed, i.e. to add a group to a user, you have to query the user first? Does omitting the groups/users attribute mean, that memberships remain unchanged?

  • The username column of the contact table must not be nullable

That column is used to store an optional reference to an Icinga Web user. That would imply that each contact is linked to one then.

  • The name column of the contactgroup table must be unique

That column currently stores a display name. Sure, that can be encoded as part of an URL, but is this desired here?

But in general, sounds like you want/need a user-chosen primary key for those tables. I wouldn't rule out just doing this instead.

nilmerg commented 5 months ago

So there will be two places where group memberships can be updated? Will the behavior be that on an update, memberships that aren't given, will be removed, i.e. to add a group to a user, you have to query the user first? Does omitting the groups/users attribute mean, that memberships remain unchanged?

Yes. Yes. Yes. My expectation is that whoever uses this API, just imports data from another source, so there's no need to fetch anything first, as all information is already available.

But in general, sounds like you want/need a user-chosen primary key for those tables. I wouldn't rule out just doing this instead.

I had a discussion with Eric about the use of UUIDs, which I'd chosen first. Though, they'd still need to reference an identifier (UUIDv5) that is known to both sides, i.e. mandatory in any case. Otherwise (UUIDv4) we'd had to prevent changes in the UI to resources created through the API and vice versa. The primary key isn't an option, hence the username. The group name is indeed more of a label right now.

My goal was, not to limit edits in any way. A resource created through the API should be changeable in the UI. This means, by creating one in the UI, the identifier must be provided, just the same as when creating it through the API.

julianbrost commented 5 months ago

I had a discussion with Eric about the use of UUIDs, which I'd chosen first. Though, they'd still need to reference an identifier (UUIDv5) that is known to both sides, i.e. mandatory in any case. Otherwise (UUIDv4) we'd had to prevent changes in the UI to resources created through the API and vice versa. The primary key isn't an option, hence the username. The group name is indeed more of a label right now.

My goal was, not to limit edits in any way. A resource created through the API should be changeable in the UI. This means, by creating one in the UI, the identifier must be provided, just the same as when creating it through the API.

Sounds like you think something is a problem where I'd say it's perfectly fine. UUIDs are an obvious choice, so let's stick to that. I'd say it's perfectly fine to make the primary key a UUID without any restrictions on the version. If a contact ist created using Web, it just gets a random UUID (v4) assigned. If a contact is created using the API, the client chooses the UUID however it desires. If it syncs from somewhere else that already uses an UUID to identify the source object, use that, otherwise, use a hash-based UUID (v5) or even use a random UUID and keep some state, doesn't matter for us, that would be a decision for the API client author.

Creating a contact in Web and then updating it using the API should be possible, yes, but if your sync source is the primary data source anyways, why wouldn't you create all contacts using the API? If your use-case is to just update individual attributes like an e-mail address for existing contacts, the API client could still query the contact by username using the filter mechanism and then update it by the returned ID. Or it could just query all contacts and then update those, where an update is necessary. So that would still be possible without a predictable ID.

nilmerg commented 5 months ago

Fine. Let's use UUIDs as identifier. contact.username and contactgroup.name are left untouched then. The UUID of a resource will be part of the structure as id key. But I wouldn't make the UUID the new primary key. We'd have to change every reference in the schema to contacts and groups then.

Oh, btw, I forgot to include addresses. -.-

julianbrost commented 5 months ago

But I wouldn't make the UUID the new primary key. We'd have to change every reference in the schema to contacts and groups then.

Yes, but it sounds like the way cleaner option, doesn't it? Would this result in an unreasonable amount of work in Notifications Web?

nilmerg commented 5 months ago

Yes, but it sounds like the way cleaner option, doesn't it?

It's not required. I'd rather add a new column for this, as a start. We still don't have versioning, so the schema isn't stable anyway..

julianbrost commented 5 months ago

However, if we just replace the primary key type of these two columns, it's a bit of an arbitrary mix.

Another idea for how two different columns could make sense in my opinion: keep the current numeric ID as-is and add a second column, something like external_id or external_key that is nullable and unique. For objects created using the web interface, this value is just not set, but if objects are created via the API, it's an optional field the client can use to store auxiliary information to later identify the same record. I.e. the default identifier stays the numeric ID, but if desired, the API client could access the objects also by this external ID (or username for that matter), all of these would be fast due to the existence of a corresponding index. The type could either be UUID/16 bytes or even just some string type so that the client could store whatever they want, like LDAP DN, ID reference to whatever other database, etc. without requiring any hashing, thus allowing lookups in the other direction as well.

nilmerg commented 5 months ago

the default identifier stays the numeric ID

Nope. That goes against everything I read. If we keep our numeric ID, it's won't be exposed in the API.

The type could either be...

A single type. I really don't want to support multiple ways if UUID is one of them. It should be the only one.

thus allowing lookups in the other direction as well.

This is something we already solved and agreed on:

Creating a contact in Web and then updating it using the API should be possible, yes, but if your sync source is the primary data source anyways, why wouldn't you create all contacts using the API? If your use-case is to just update individual attributes like an e-mail address for existing contacts, the API client could still query the contact by username using the filter mechanism and then update it by the returned ID.

However, if we just replace the primary key type of these two columns, it's a bit of an arbitrary mix.

Then let's introduce a new column of type UUID and make it required.

julianbrost commented 5 months ago

the default identifier stays the numeric ID

Nope. That goes against everything I read. If we keep our numeric ID, it's won't be exposed in the API.

Why not? Also, I'm not really sure what you read.

The type could either be...

A single type. I really don't want to support multiple ways if UUID is one of them. It should be the only one.

Of course we should pick one. Just wanted to say that the exact choice won't matter for the rest I wrote.

thus allowing lookups in the other direction as well.

This is something we already solved and agreed on:

That's not what I wanted to say with that. If you have a field large enough to store an unhashed reference, an API client could retrieve a list of all contacts, look them up by say a stored LDAP DN, and check if anything needs to be updated. Not sure how commonly someone would want to build something like this, I just wanted to say that's something that would additionally be possible if it was a "store whatever you want" type.

nilmerg commented 5 months ago

Why not? Also, I'm not really sure what you read.

To prevent enumeration attacks.

If you have a field large enough to store an unhashed reference, an API client could retrieve a list of all contacts, look them up by say a stored LDAP DN, and check if anything needs to be updated.

If the identifier is a UUID, chosen by the client, no checks are necessary. The client just PUTs the data it has and nothing else. And all with a "store whatever you want" type.

julianbrost commented 5 months ago

Why not? Also, I'm not really sure what you read.

To prevent enumeration attacks.

Yes, in general, unpredictable IDs add an additional layer of defense. But isn't the API supposed to allow listing all objects anyways?

If the identifier is a UUID, chosen by the client, no checks are necessary. The client just PUTs the data it has and nothing else. And all with a "store whatever you want" type.

Syncing object deletion would be annoying that way though as you don't really have a way of telling how that UUID was generated. If you have an contact that says external_id = 'uid=poorguy,ou=something,dc=example,dc=com', a sync client could simply check if that still exists in LDAP and if it doesn't, issue a DELETE request.

nilmerg commented 5 months ago

We concluded that two columns (pk + external_uuid), both required, are sufficient for now. Making the pk the UUID can be done at a later point, together with a custom fact for the client to identify its own resources. (To allow safe removals)

ncosta-ic commented 3 months ago

Requests

[!NOTE] Captions: ${\texttt{\color{#CFBAF0}GET \color{#B9FBC0}POST \color{#FDE4CF}PUT \color{#FFCFD2}DELETE}}$ indicate the request method Captions: 🟒 πŸ”΄ declare whether a request is expected to return a success or a failure

${\texttt{\color{#B9FBC0}POST\,\,\,\:\:}}$ πŸ”΄  1. New contact group (invalid body format)
#### Description Create a new contact group with a `YAML` payload, while declaring the body type as `application/json`.
#### Request **URL**: `http://localhost/icingaweb2/notifications/api/v1/contactgroups` **Headers**: `Accept: application/json, Content-Type: application/json` **Body**: ```json --- payload: invalid ```
#### Response **Headers**: `500 Internal Server Error` **Body**: ```json { "status": "error", "message": "Syntax error: '---\npayload: invalid\n'" } ```
${\texttt{\color{#B9FBC0}POST\,\,\,\:\:}}$ πŸ”΄  2. New contact group (invalid body type)
#### Description Create a new contact group with a `YAML` payload.
#### Request **URL**: `http://localhost/icingaweb2/notifications/api/v1/contactgroups` **Headers**: `Accept: application/json, Content-Type: text/yaml` **Body**: ```yaml --- payload: invalid ```
#### Response **Headers**: `400 Bad Request` **Body**: ```json { "status": "error", "message": "No JSON content" } ```
${\texttt{\color{#B9FBC0}POST\,\,\,\:\:}}$ πŸ”΄  3. New contact group (incomplete body)
#### Description Create a new contact group with a valid `JSON` payload, that is missing the mandatory `name` field.
#### Request **URL**: `http://localhost/icingaweb2/notifications/api/v1/contactgroups` **Headers**: `Accept: application/json, Content-Type: application/json` **Body**: ```json { "id": "0817d973-398e-41d7-9ef2-61cdb7ef41a2", "users": [] } ```
#### Response **Headers**: `400 Bad Request` **Body**: ```json { "status": "error", "message": "Invalid request body: the fields id and name must be present and of type string" } ```
${\texttt{\color{#B9FBC0}POST\,\,\,\:\:}}$ πŸ”΄  4. New contact group (with filter)
#### Description Create a new contact group with a valid `JSON` payload, while providing a filter.
#### Request **URL**: `http://localhost/icingaweb2/notifications/api/v1/contactgroups?id=0817d973-398e-41d7-9ef2-61cdb7ef41a2` **Headers**: `Accept: application/json, Content-Type: application/json` **Body**: ```json { "id": "0817d973-398e-41d7-9ef2-61cdb7ef41a2", "name": "Test group", "users": [] } ```
#### Response **Headers**: `400 Bad Request` **Body**: ```json { "status": "error", "message": "Filter is only allowed for GET requests" } ```
${\texttt{\color{#B9FBC0}POST\,\,\,\:\:}}$ πŸ”΄  5. New contact group (with identifier)
#### Description Create a new contact group with a valid `JSON` payload, while providing an identifier.
#### Request **URL**: `http://localhost/icingaweb2/notifications/api/v1/contactgroups/0817d973-398e-41d7-9ef2-61cdb7ef41a2` **Headers**: `Accept: application/json, Content-Type: application/json` **Body**: ```json { "id": "0817d973-398e-41d7-9ef2-61cdb7ef41a2", "name": "Test group", "users": [] } ```
#### Response **Headers**: `404 Not Found` **Body**: ```json { "status": "error", "message": "Contactgroup not found" } ```
${\texttt{\color{#B9FBC0}POST\,\,\,\:\:}}$ 🟒  6. New contact group (create)
#### Description Create a new contact group with a valid `JSON` payload.
#### Request **URL**: `http://localhost/icingaweb2/notifications/api/v1/contactgroups` **Headers**: `Accept: application/json, Content-Type: application/json` **Body**: ```json { "id": "0817d973-398e-41d7-9ef2-61cdb7ef41a2", "name": "Test group", "users": [] } ```
#### Response **Headers**: `201 Created, Location: notifications/api/v1/contactgroups/0817d973-398e-41d7-9ef2-61cdb7ef41a2` **Body**: ```json { "status": "success", "data": null } ```
${\texttt{\color{#B9FBC0}POST\,\,\,\:\:}}$ πŸ”΄  7. New contact group (replace, equal identifier)
#### Description Replace a contact group while providing the same identifier in both the `JSON` payload and Request-URI.
#### Request **URL**: `http://localhost/icingaweb2/notifications/api/v1/contactgroups/0817d973-398e-41d7-9ef2-61cdb7ef41a2` **Headers**: `Accept: application/json, Content-Type: application/json` **Body**: ```json { "id": "0817d973-398e-41d7-9ef2-61cdb7ef41a2", "name": "Test group", "users": [] } ```
#### Response **Headers**: `422 Unprocessable Entity` **Body**: ```json { "status": "error", "message": "Contactgroup already exists" } ```
${\texttt{\color{#B9FBC0}POST\,\,\,\:\:}}$ 🟒  8. New contact group (replace, new identifier)
#### Description Replace a contact group while providing a new identifier in the `JSON` payload.
#### Request **URL**: `http://localhost/icingaweb2/notifications/api/v1/contactgroups/0817d973-398e-41d7-9ef2-61cdb7ef41a2` **Headers**: `Accept: application/json, Content-Type: application/json` **Body**: ```json { "id": "0817d973-398e-41d7-9ef2-61cdb7ef41a3", "name": "Test group (replaced)", "users": [] } ```
#### Response **Headers**: `201 Created, Location: notifications/api/v1/contactgroups/0817d973-398e-41d7-9ef2-61cdb7ef41a3` **Body**: ```json { "status": "success", "data": null } ```
${\texttt{\color{#FDE4CF}PUT\,\,\,\,\,\,\,\,\,}}$ πŸ”΄  9. Update contact group (invalid body format)
#### Description Update a contact group with a `YAML` payload, while declaring the body type as `application/json`.
#### Request **URL**: `http://localhost/icingaweb2/notifications/api/v1/contactgroups/0817d973-398e-41d7-9ef2-61cdb7ef41a3` **Headers**: `Accept: application/json, Content-Type: application/json` **Body**: ```json --- payload: invalid ```
#### Response **Headers**: `500 Internal Server Error` **Body**: ```json { "status": "error", "message": "Syntax error: '---\npayload: invalid\n'" } ```
${\texttt{\color{#FDE4CF}PUT\,\,\,\,\,\,\,\,\,}}$ πŸ”΄ 10. Update contact group (without identifier)
#### Description Update a contact group with a `YAML` payload, while not providing an identifier in the Request-URI.
#### Request **URL**: `http://localhost/icingaweb2/notifications/api/v1/contactgroups` **Headers**: `Accept: application/json, Content-Type: application/json` **Body**: ```json { "id": "0817d973-398e-41d7-9ef2-61cdb7ef41a3", "name": "Test group (replaced)", "users": [] } ```
#### Response **Headers**: `400 Bad Request` **Body**: ```json { "status": "error", "message": "Identifier is required" } ```
${\texttt{\color{#FDE4CF}PUT\,\,\,\,\,\,\,\,\,}}$ πŸ”΄ 11. Update contact group (identifier mismatch)
#### Description Update a contact group with a `YAML` payload, while providing different identifiers in the request body and Request-URI.
#### Request **URL**: `http://localhost/icingaweb2/notifications/api/v1/contactgroups/0817d973-398e-41d7-9ef2-61cdb7ef41a3` **Headers**: `Accept: application/json, Content-Type: application/json` **Body**: ```json { "id": "0817d973-398e-41d7-9ef2-61cdb7ef41a4", "name": "Test group (replaced)", "users": [] } ```
#### Response **Headers**: `400 Bad Request` **Body**: ```json { "status": "error", "message": "Identifier mismatch" } ```
${\texttt{\color{#FDE4CF}PUT\,\,\,\,\,\,\,\,\,}}$ 🟒 12. Create contact group (with identifier)
#### Description Create a _new_ contact group with a `YAML` payload, while providing its identifier in the Request-URI.
#### Request **URL**: `http://localhost/icingaweb2/notifications/api/v1/contactgroups/0817d973-398e-41d7-9ef2-61cdb7ef41a4` **Headers**: `Accept: application/json, Content-Type: application/json` **Body**: ```json { "id": "0817d973-398e-41d7-9ef2-61cdb7ef41a4", "name": "Test group 2", "users": [] } ```
#### Response **Headers**: `201 Created` **Body**: ```json { "status": "success", "data": null } ```
${\texttt{\color{#FDE4CF}PUT\,\,\,\,\,\,\,\,\,}}$ 🟒 13. Update contact group (with identifier)
#### Description Update an existing contact group with a `YAML` payload, while providing its identifier in the Request-URI.
#### Request **URL**: `http://localhost/icingaweb2/notifications/api/v1/contactgroups/0817d973-398e-41d7-9ef2-61cdb7ef41a4` **Headers**: `Accept: application/json, Content-Type: application/json` **Body**: ```json { "id": "0817d973-398e-41d7-9ef2-61cdb7ef41a4", "name": "Test group 2 (updated)", "users": [] } ```
#### Response **Headers**: `204 No Content` **Body**: `none`
${\texttt{\color{#CFBAF0}GET\,\,\,\,\,\,\,\,\,}}$ 🟒 14. Get contact groups
#### Description Gets all contact groups currently stored at the endpoint.
#### Request **URL**: `http://localhost/icingaweb2/notifications/api/v1/contactgroups` **Headers**: `Accept: application/json` **Body**: `none`
#### Response **Headers**: `200 OK` **Body**: ```json [ { "id": "0817d973-398e-41d7-9ef2-61cdb7ef41a3", "name": "Test group (replaced)", "users": [] }, { "id": "0817d973-398e-41d7-9ef2-61cdb7ef41a4", "name": "Test group 2 (updated)", "users": [] } ] ```
${\texttt{\color{#CFBAF0}GET\,\,\,\,\,\,\,\,\,}}$ 🟒 15. Get specific group (with identifier)
#### Description Gets a specific contact group by providing its identifier in the Request-URI.
#### Request **URL**: `http://localhost/icingaweb2/notifications/api/v1/contactgroups/0817d973-398e-41d7-9ef2-61cdb7ef41a3` **Headers**: `Accept: application/json` **Body**: `none`
#### Response **Headers**: `200 OK` **Body**: ```json { "status": "success", "data": [ { "id": "0817d973-398e-41d7-9ef2-61cdb7ef41a3", "name": "Test group (replaced)", "users": [] } ] } ```
${\texttt{\color{#CFBAF0}GET\,\,\,\,\,\,\,\,\,}}$ 🟒 16. Get specific group (with filter)
#### Description Gets a specific contact group by providing a filter for a matching identifier in the Request-URI.
#### Request **URL**: `http://localhost/icingaweb2/notifications/api/v1/contactgroups?id=0817d973-398e-41d7-9ef2-61cdb7ef41a3` **Headers**: `Accept: application/json` **Body**: `none`
#### Response **Headers**: `200 OK` **Body**: ```json [ { "id": "0817d973-398e-41d7-9ef2-61cdb7ef41a3", "name": "Test group (replaced)", "users": [] } ] ```
${\texttt{\color{#CFBAF0}GET\,\,\,\,\,\,\,\,\,}}$ 🟒 17. Get specific groups (no matching name)
#### Description Gets specific contact groups, while providing a non-matching `name` filter.
#### Request **URL**: `http://localhost/icingaweb2/notifications/api/v1/contactgroups?name=Non-Existent%20Group` **Headers**: `Accept: application/json` **Body**: `none`
#### Response **Headers**: `200 OK` **Body**: ```json [] ```
${\texttt{\color{#CFBAF0}GET\,\,\,\,\,\,\,\,\,}}$ πŸ”΄ 18. Get specific group (non-existent identifier)
#### Description Gets a specific contact group by providing a non-existent identifier in the Request-URI.
#### Request **URL**: `http://localhost/icingaweb2/notifications/api/v1/contactgroups/0817d973-398e-41d7-9ef2-61cdb7ef41a5` **Headers**: `Accept: application/json` **Body**: `none`
#### Response **Headers**: `404 Not Found` **Body**: ```json { "status": "error", "message": "Contactgroup not found" } ```
${\texttt{\color{#FFCFD2}DELETE}}$ πŸ”΄ 19. Delete contact group (no identifier)
#### Description Deletes a contact group, while not providing an identifier in the Request-URI.
#### Request **URL**: `http://localhost/icingaweb2/notifications/api/v1/contactgroups` **Headers**: `Accept: application/json` **Body**: `none`
#### Response **Headers**: `400 Bad Request` **Body**: ```json { "status": "error", "message": "Identifier is required" } ```
${\texttt{\color{#FFCFD2}DELETE}}$ πŸ”΄ 20. Delete contact group (non-existing identifier)
#### Description Deletes a contact group, while providing an identifier which doesn't exist.
#### Request **URL**: `http://localhost/icingaweb2/notifications/api/v1/contactgroups/0817d973-398e-41d7-9ef2-61cdb7ef41a5` **Headers**: `Accept: application/json` **Body**: `none`
#### Response **Headers**: `404 Not Found` **Body**: ```json { "status": "error", "message": "Contactgroup not found" } ```
${\texttt{\color{#FFCFD2}DELETE}}$ 🟒 21. Delete contact group (existing identifier)
#### Description Deletes a contact group, while providing an existing identifier.
#### Request **URL**: `http://localhost/icingaweb2/notifications/api/v1/contactgroups/0817d973-398e-41d7-9ef2-61cdb7ef41a3` **Headers**: `Accept: application/json` **Body**: `none`
#### Response **Headers**: `204 No Content` **Body**: `none`
${\texttt{\color{#FFCFD2}DELETE}}$ 🟒 21. Delete contact group (same as request 20)
#### Description Deletes a contact group, while providing an existing identifier.
#### Request **URL**: `http://localhost/icingaweb2/notifications/api/v1/contactgroups/0817d973-398e-41d7-9ef2-61cdb7ef41a4` **Headers**: `Accept: application/json` **Body**: `none`
#### Response **Headers**: `204 No Content` **Body**: `none`

Broken requests

sukhwinder33445 commented 3 months ago

Thanks for the tests. Please write directly in the PR next time. Requests 1 and 9 now throw 400 Bad Request. Request 5 is implemented as the requirements above. If the ID is specified, an attempt is made to replace the entry.