UTRS Appeal transfer - Githubissues

stwalkerster commented 1 year ago

Re-hashing #26...

Thoughts have again resurfaced about transferring UTRS requests over to ACC. UTRS could send us email, provided IP, webserver IP, UA, block info, and the generic appeal reason. The initial thoughts we've had revolve around a HTTP API between ACC and UTRS, where UTRS will transfer the relevant appeal data to ACC, ACC returns a token to UTRS, then UTRS can redirect the user back to ACC with the token for ACC to take over handling of the request.

Below is a quick sequence diagram for what I initially expect the process to look like:

sequenceDiagram

actor BlockedUser
actor Admin
participant UTRS
participant ACC

note over BlockedUser,UTRS: Standard UTRS process happens first

Admin->>UTRS: Approval of transfer
UTRS->>ACC:Send transferral request (with all necessary appeal data + appeal id)
ACC-->>UTRS:Token issued on receipt of data
UTRS-->>Admin: Notification of successful transfer

UTRS->>BlockedUser: Notification (email) containing link to ACC with token
UTRS->>UTRS: Close request as transferred
BlockedUser->>ACC:User clicks link to ACC with token
ACC-->>BlockedUser:ACC presents form for remaining data
BlockedUser->>ACC:Submission of missing data

ACC->>UTRS: Notification of ACC request ID (using token)
UTRS-->>ACC: Acknowledgement

ACC->>ACC:Invalidate token for new requests

note over ACC:ACC team handles request

ACC->>BlockedUser: Notification of account creation

ACC->>UTRS: Notification of ACC request status (using token)
UTRS-->>ACC: Acknowledgement

The HTTP API will be secured via a HMAC-based authentication system. We'll agree some way of organising the fields and hashing them with a shared-secret key. The result will be transmitted with the request, and the recipient will perform the same calculation. If the results match, then the sender must have the same key as the recipient.

This is still a work-in-progress to figure out exactly what and how we're going to attempt this.

API details

Two APIs need to be written, one ACC's side and one on UTRS' side.

These APIs are essentially both the same design and use the same format.

ACC "data transfer" API

ACC will present an API (eg: /internal.php/api/utrsinbound) which will accept a JSON document. This API is to be used to receive appeal data from UTRS as the first part of the transfer process.

Data Transfer API definition

The JSON document received by ACC should consist of a single object containing the following keys: Field | Type | Description ------|------|--------- `nonce` | string | [Single-use token](https://en.wikipedia.org/wiki/Cryptographic_nonce) to prevent replay attacks. This must be different on each API call, even if all the other data is identical. ACC should validate that is is unique `utrsId` | integer | ID of the UTRS appeal `email` | string | Email address of the requester `emailConfirmed` | bool | Whether the email address is already validated by UTRS. `networkIdentity` | object[] | Network information stored for the appeal (see below) `blockInfo` | object[] | Information about the block. TODO: figure out what info is available and from that what is useful `queueId` | `int?` (nullable int) | ACC queue to defer request into `domain` | string | ACC domain short name (eg `enwiki`) `comments` | object[] | Comments to automatically create on the ACC request `comment` is an array of objects. each object is: Field | Type | Description ------|------|--------- `comment` | string | Content of comment `origin` | string | Who created the comment "utrs" or "admin" or "user" `username` | `string?` | If origin==admin, the username of the admin, null otherwise `blockInfo` is an array of objects. each object is: Field | Type | Description ------|------|--------- `comment` | string | Block reason `admin` | string | Who set the block `networkIdentity` should be an array of objects, with each object having the following information: Field | Type | Description ------|------|--------- `ipv4` | string? | IPv4 address `ipv6` | string? | IPv6 address `ua` | string? | User agent + client hints `trusted` | bool | Is this data trusted (captured/validated by UTRS, or user-provided?)

This must be delivered as a HTTP POST request, with the following HTTP headers set:

Content-Type: application/json
Cache-Control: no-store
Authorization: Bearer 123456abcdef
X-ACC-API-Version: 1

The Authorization header value must be the word Bearer followed by a SHA-384 HMAC of the entire body content as a string. This can be generated in PHP with the hash_hmac('sha384', $data, $key) function. We will agree on a shared key.

If the data format of the request or response body needs to change at a later date, we will bump the X-ACC-API-Version header as required.

All other headers should be sent with the exact values shown above.

Example

```json { "nonce": "EsczJwh9cfY8hXrL5tDdlh6onOhYGpFBjKbhipdbhpM" "utrsId":2345, "email":"bob@example.com", "emailConfirmed":true, "networkIdentity":[ { "ip":"127.0.0.15", "ch":null, "trusted":true, // checkuser data }, { "ip":"1.2.3.4", "ua":null, "trusted":false, // appeal for } ], "blockInfo":[ { "comment":"for the lolz", "admin":"AmandaNP" }, { "comment":"no u", "admin":"Stwalkerster" } ], "domain": "enwiki" "queueId": null, "comments": [ { "comment": "I like json", "origin": "admin" }, { "comment": "other info", "origin": "utrs" } ] } ```

ACC will then respond with a HTTP response. For example, the request above may produce the below response.

Headers:

Content-Type: application/json
Cache-Control: no-store

Body:

{
    "status":"OK",
    "error":null,
    "utrsId":2345,
    "token":"123456abcdef",
    "url":"https://accounts.wmflabs.org/index.php/utrs?t=123456abcdef"
}

Data Transfer API response definition

Field | Type | Description ------|------|--------- `status` | string | Either "OK" or some other value. Only "OK" indicates success. `error` | string or null | If `status` is "OK", then this field will be null. Otherwise, this field will contain an error message. `utrsId` | integer | ID of the UTRS appeal `token` | string | The token representing the received data at ACC. This token must be provided to ACC to create an account request linked to this UTRS appeal. `url` | string | The URL to direct the user to in order to complete their appeal. This field should be an opaque URL to UTRS and provided as-is to the appellant. This URL will include the token value above

UTRS should check the status field is "OK", and then pass the token and/or URL back to the user. If the status field is not "OK", a relevant error message will be placed in the "error" field of the response.

UTRS request notification

UTRS will present an API which will accept a JSON document. The JSON document should consist of a single object containing the following keys:

Request status request definition

Field | Type | Description ------|------|--------- `nonce` | string | [Single-use token](https://en.wikipedia.org/wiki/Cryptographic_nonce) to prevent replay attacks. This must be different on each API call, even if all the other data is identical. UTRS should validate that it is unique. `utrsId` | integer | ID of the UTRS appeal `token` | string | The token representing the received data at ACC. `accId` | integer | ID of the ACC request `status` | string | Current status of the request. Will be either `open` or `closed` `closureType` | string? | Further information about the request closure reason

This must be delivered as a HTTP POST request, with the following HTTP headers set:

Content-Type: application/json
Cache-Control: no-store
Authorization: Bearer 123456abcdef
X-UTRS-API-Version: 1

See notes in the ACC section about the meaning of the headers.

Example:

{
    "nonce": "onOSnXh81TNvhNHf7m_IRhPAQcORtoig5Qf1pLx2bK0"
    "utrsId": 2345,
    "token": "123456abcdef",
    "accId": 345678,
    "status": "open"
}

UTRS will then respond with a HTTP response. For example, the request above may produce the below response.

Headers:

Content-Type: application/json
Cache-Control: no-store

Body:

{
    "status":"OK",
    "error":null
}

Request status response definition

Field | Type | Description ------|------|--------- `status` | string | Either "OK" or some other value. Only "OK" indicates success. `error` | string or null | If `status` is "OK", then this field will be null. Otherwise, this field will contain an error message.

dqwiki commented 1 year ago

@stwalkerster After reviewing and rethinking today, additionally, I would actually prefer a report back with the result of the appeal transfer (created, not created, potentially templated reason - not the actual text). That said, this can very much be a down-the-road request and does not have to be implemented on release.

Motivation

stwalkerster commented 1 year ago

I would actually prefer a report back with the result of the appeal transfer (created, not created, potentially templated reason - not the actual text).

Understood, I'll try to factor that into the design. Bear in mind that it's possible you may get multiple notifications from ACC as it's possible for a request to be closed, reopened, and closed again. What I'll probably do is send UTRS every status update, and leave it to UTRS to decide what to do with that data. (Remember that status update does not include queue changes, it'll just be open/close with reason)

What I might actually do is use the UTRS-side API as a general status reporting system so it's used for both telling you about the request ID, and updating you on the request's status. I need to think a little more about this though.

I'm also going to make some more tweaks to the API definition. I want to split out ip in networkIdentity into ipv4 and ipv6, though if this causes problems for you I'm not too attached to it.

stwalkerster commented 1 year ago

OK, the other change I've made is to remove the validation step for the HTTP response of requests. I've also done some layout tweaks to the description above, putting some stuff in collapsible sections, and moving the diagram from an image to GitHub's native diagramming

dqwiki commented 7 months ago

@stwalkerster A response can contain up to 4 items:	Field name	Meaning
status	----	As requested
code?	HTTP error code	Will be zero if none
api_error_code	An internal "track this error down" code for UTRS	0 is no error, otherwise errors start at 1000
message	Error message	String of the error

[ ] I will need clarity on how we want the single use token generated, stored and checked.
I am creating a master token-generating interface at its included partially in native features - so you can regenerate it as you see fit
The only real thing remaining is setting up the security - the response is already as expected otherwise

stwalkerster commented 7 months ago

A response can contain up to 4 items:

Is this for the request status response?

I will need clarity on how we want the single use token generated, stored and checked.

How does something like this sound for the token?

time() . "." . base64_encode(random_bytes(24));

When it's received, split it on the . and store the two parts in a database table as two separate columns. If the first part compared to the current time is more than say 10 minutes, treat it as invalid. If the second part is in the database table already, treat it as invalid. Then you can have a regular cleanup job on the database table for old entries as things outside the time window will always be invalid.

enwikipedia-acc / waca

UTRS Appeal transfer #779

API details

ACC "data transfer" API

UTRS request notification