Usage accounting

[rowleya]

It could be potentially useful for the spalloc service to support user usage accounting, in particular for EBRAINS users. This would make it easier to keep track of usage on the Jupyter service for example, where it is otherwise necessary to track usage more manually.

Notes:

• This would probably need to be selective based on "user type" e.g. local users wouldn't want to be subject to constraints, but remote users might have quotas that shouldn't be exceeded.
  • A local user might be recognised by IP address or some other mechanism.
  • An EBRAINS user might be recognised by using an EBRAINS token.
• There is an NMPI quota service that this could interact with.
  • This could be updated at each "keep-alive", if this is still implemented, by then subtracting resource from the quota equivalent to size of machine * keep alive duration.
  • This would mean that infinite keep-alive would be disallowed for EBRAINS users.
  • If the quota service reports that the user has run out of quota, the machine would be released and the job killed (with an out-of-quota message).

[rowleya]

An additional thought - the current NMPI quota is based on collabs. The collab ID would then need to be specified in the job request along with the EBRAINS authentication token. The process would then be to first check (via another service) that the user can access the collab, then the quota is removed from the collab assignment as described above.

[dkfellows]

Identifying users is one of the critical parts of this. See #338

[Christian-B]

One option here is two have two stage quota.

Stage one all users have a reasonable individual quota that we track locally.

Stage two if specific users need more quota they have to provide their collab ID and then we extend in large blocks individual quota by reducing their collab assignment.

This has the advantage of working for most users without the extra collab ID overhead, reduces the calls to the collab and means that other users in a collab would have some usage even if the collab runs out.

[rowleya]

We could also have an interface here that interacts with the NMPI quota at a different level if we can agree it with them. This would be that they send us any quota updates, and their API queries ours when a request for how much quota is available / used is made.

[dkfellows]

I've added a local quota system (using board-seconds as the fundamental unit) but I don't understand yet how to merge this with a remote quota system.

[rowleya]

Interaction is as yet undefined; we can ask Andrew Davidson about quotas etc. and see if we can make something that will work at some point. Having a local system at least means that we can control things, which is the first step!

[dkfellows]

Still need some way of editing the quota. Currently it's only settable by direct database access, which is obviously unacceptable.

[dkfellows]

Quotas also supported with OIDC login.

Default quota isn't right, but that's a config parameter.

[dkfellows]

Can now add or remove quota via administration interface, in units of 1, 10, 100, and 1000 board hours.

Just need a default. And a decision about whether local users (those with non-HBP logins) get different default quotas.

[dkfellows]

See also #413

[dkfellows]

See also #414