Add a QS Web Server

[prjemian]

We'll need to add a QS Web Server, such as the bluesky-httpserver project, to be able to build the QS Web Client. See this revision to the above figure:

Originally posted by @prjemian in https://github.com/BCDA-APS/qs-web2023/issues/4#issuecomment-1574588435

[prjemian]

And, since the bluesky-httpserver expects a kafka server, we should add this to our system as well.

[prjemian]

Note that while trying out the bluesky-webclient, we got a Python exception that suggested there is a version problem in the instructions when calling one of the other bluesky packages.

(bluesky_2023_2) prjemian@zap:~/.../Bluesky/bluesky-webclient$ uvicorn bluesky_queueserver.server.server:app --host localhost --port 60610
Traceback (most recent call last):
  File "/home/prjemian/.conda/envs/bluesky_2023_2/bin/uvicorn", line 11, in <module>
    sys.exit(main())
  File "/home/prjemian/.conda/envs/bluesky_2023_2/lib/python3.10/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/home/prjemian/.conda/envs/bluesky_2023_2/lib/python3.10/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/home/prjemian/.conda/envs/bluesky_2023_2/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/prjemian/.conda/envs/bluesky_2023_2/lib/python3.10/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/home/prjemian/.conda/envs/bluesky_2023_2/lib/python3.10/site-packages/uvicorn/main.py", line 410, in main
    run(
  File "/home/prjemian/.conda/envs/bluesky_2023_2/lib/python3.10/site-packages/uvicorn/main.py", line 578, in run
    server.run()
  File "/home/prjemian/.conda/envs/bluesky_2023_2/lib/python3.10/site-packages/uvicorn/server.py", line 61, in run
    return asyncio.run(self.serve(sockets=sockets))
  File "/home/prjemian/.conda/envs/bluesky_2023_2/lib/python3.10/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/home/prjemian/.conda/envs/bluesky_2023_2/lib/python3.10/asyncio/base_events.py", line 646, in run_until_complete
    return future.result()
  File "/home/prjemian/.conda/envs/bluesky_2023_2/lib/python3.10/site-packages/uvicorn/server.py", line 68, in serve
    config.load()
  File "/home/prjemian/.conda/envs/bluesky_2023_2/lib/python3.10/site-packages/uvicorn/config.py", line 473, in load
    self.loaded_app = import_from_string(self.app)
  File "/home/prjemian/.conda/envs/bluesky_2023_2/lib/python3.10/site-packages/uvicorn/importer.py", line 24, in import_from_string
    raise exc from None
  File "/home/prjemian/.conda/envs/bluesky_2023_2/lib/python3.10/site-packages/uvicorn/importer.py", line 21, in import_from_string
    module = importlib.import_module(module_str)
  File "/home/prjemian/.conda/envs/bluesky_2023_2/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 992, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1004, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'bluesky_queueserver.server'

[prjemian]

We just might be able to do all this from our user account (without needing IT to install more software). Just maybe.

[prjemian]

Note: Not bluesky-httpserver. Use bluesky_queueserver.server.server:appinstead.

[prjemian]

Nope, back to the httpserver project, which has lots of advice for starting the server in various ways. We'll need to learn some new things.

[prjemian]

Block diagram is more like this:

Or maybe a flask http server instead?

[prjemian]

Aha! We found some help here. This command started a (fully unconfigured) server: QSERVER_HTTP_SERVER_SINGLE_USER_API_KEY=mykey uvicorn --host localhost --port 60610 bluesky_httpserver.server:app

Next, we will learn how to configure it.

[prjemian]

Reading further on that page, tried another command as suggested:

http GET http://localhost:60610/api/status 'Authorization: ApiKey mykey'

which gave this response:

(bluesky_2023_2) prjemian@zap:/tmp$ http GET http://localhost:60610/api/status 'Authorization: ApiKey mykey'
HTTP/1.1 200 OK
content-length: 876
content-type: application/json
date: Tue, 06 Jun 2023 14:42:58 GMT
server: uvicorn
server-timing: app;dur=4.3
set-cookie: bluesky_httpserver_csrf=NO3DgxmJXOJaL-qyGTHzh-mwpC8FtOO6OdLr7Xth8L0; HttpOnly; Path=/; SameSite=lax

{
    "devices_allowed_uid": "4b127449-e7e3-42af-b5c9-d9b9bddc938a",
    "devices_existing_uid": "3690c19d-4958-4b1b-b76f-58c669b7ce63",
    "items_in_history": 2,
    "items_in_queue": 0,
    "lock": {
        "environment": false,
        "queue": false
    },
    "lock_info_uid": "4d2c68c2-9c3a-4e4f-a287-38cee1ad6945",
    "manager_state": "idle",
    "msg": "RE Manager v0.0.18",
    "pause_pending": false,
    "plan_history_uid": "a9ffeac2-f050-4c11-abf8-f6c0a1726f12",
    "plan_queue_mode": {
        "loop": false
    },
    "plan_queue_uid": "d41b4e82-6586-496e-bf10-63ba91d3ef9e",
    "plans_allowed_uid": "0d11bd94-bfc4-455d-9c51-feef8b73c781",
    "plans_existing_uid": "6975a43f-b069-429c-b438-2db1b1b1e7fd",
    "queue_stop_pending": false,
    "re_state": "idle",
    "run_list_uid": "b627de39-55cf-42c4-8aba-602c0da5ba12",
    "running_item_uid": null,
    "task_results_uid": "4ef5dfeb-8806-4caa-bff8-578f0b0bbf93",
    "worker_background_tasks": 0,
    "worker_environment_exists": true,
    "worker_environment_state": "idle"
}

---

Note: The http command line application was provided by installing the httpie package.

[prjemian]

With the growing list of additional packages to run this software, it would help to create a custom conda environment file to provide the full list of packages.

[prjemian]

The configuration of the bluesky-httpserver is described here.

[prjemian]

For development (only) it can be useful to skip the usual API key requirement as described here.

[prjemian]

Looks like this is complete. Can this issue be closed?