CLI via Hub/proxy and/or UIS

[dwsutherland]

Long awaited, and relates to:

2123

3005

3528

4742

The hub started UIS negotiates an API authentication token that can be used via the hub proxy or directly with the UIS, and in the absence of the hub (cylc gui) the UIS generates it's own token.

The scheduler currently has two comms methods tcp and ssh, and now that the client has been abstracted somewhat (#4742), we could add an http option/client that picks up the API token. All the CLI GraphQL queries/mutations should work as is.

Tasks:

• [ ] - cylc-uiserver (https://github.com/cylc/cylc-uiserver/pull/396)
  • [x] Make the API info (token, URL ...etc) available to the user (read-only).
  • [x] Create new client to send commands via the Hub/UIS using this API info.
• [ ] - cylc-flow (https://github.com/cylc/cylc-flow/pull/5267)
  • [x] Add http comms method and global configuration.
  • [x] Import UI Server client (require UIS to be installed).

Sample (from the UIS task)

#!/usr/bin/env python3

import json
import requests

from cylc.uiserver.app import API_INFO_FILE

f = open(API_INFO_FILE, "r")
api_info = json.loads(f.read())
f.close()

query = '''
query {
  workflows {
    id
    stateTotals
  }
}
'''

r = requests.post(api_info["url"] + 'cylc/graphql',
    headers={
        'Authorization': f'token {api_info["token"]}',
    },
    json={'query': query}
)

r.raise_for_status()
data = r.json()

print(json.dumps(data, indent=4))

$ ./uis_api.py
{
    "data": {
        "workflows": [
            {
                "id": "~sutherlander/linear/run1",
                "stateTotals": {
                    "waiting": 2,
                    "expired": 0,
                    "preparing": 0,
                    "submit-failed": 1,
                    "submitted": 0,
                    "running": 0,
                    "failed": 0,
                    "succeeded": 0
                }
            }
        ]
    }
}

[dwsutherland]

going via the UIS:

(uiserver) sutherlander@graphic-vbox:~$ cylc ping --debug linear
2022-12-07T21:18:59+13:00 DEBUG - Loading site/user config
    files
2022-12-07T21:18:59+13:00 DEBUG - Reading file
    /home/sutherlander/.cylc/flow/8.1.0.dev/global.cylc
2022-12-07T21:18:59+13:00 DEBUG - http:send {'command':
    'graphql', 'args': {'request_string': '\nquery ($wFlows: [ID]) {\n
    workflows(ids: $wFlows) {\n    id\n    name\n    port\n    pubPort\n
    }\n}\n', 'variables': {'wFlows': ['linear/run1']}}, 'meta': {'prog': 'ping',
    'host': 'graphic-vbox', 'comms_method': 'http'}}
2022-12-07T21:18:59+13:00 DEBUG - http:recv {'data':
    {'workflows': [{'id': '~sutherlander/linear/run1', 'name': 'linear/run1',
    'port': 43072, 'pubPort': 43070}]}}
linear/run1 running on graphic-vbox:43072 43070

We man need a general communication method global config setting, all I could find is the platform one, which as far as I can tell is only for jobs: https://cylc.github.io/cylc-doc/nightly/html/reference/config/global.html#global.cylc\[platforms\]\[%3Cplatform%20name%3E\]communication%20method

Had to $ export CYLC_TASK_COMMS_METHOD='http' to get it using HTTP UI Server client.

[oliver-sanders]

Yep, but of an oversight there, we could do with the ability to configure the comms method for non-job comms anyway (e.g. if the scheduler host's Cylc ports are not open)

https://github.com/cylc/cylc-flow/issues/5265