cli: demo experience

[tgeoghegan]

We want it to be easy for prospective users of Divvi Up to give it a whirl and do some simple aggregations. To enable that, we want to make it easy to use the included compose.yaml to spin up a control plane, web app and a pair of aggregators, and then use divviup dap-client to upload and collect.

Missing pieces are:

• [x] Distribute compose.yaml in divviup-api releases so that users don't have to check out the project (#1098)
• [x] Pairing aggregators with control plane in compose setup (#1099)
• [x] rewrite cli/README.md to discuss docker compose (#1123)
• [x] test in CI that docker compose up works (#1097)
• [x] test demo script in CI (#1126)
• [ ] Automatically update container images in compose.yaml
• [x] Do something about divviup_api_vite so you can run compose.yaml without checking out sources (#1124)
• [x] fix startup race between postgres and migrator jobs (#1125)
• [ ] make collector credential handling less awkward (#1033, #1073)
• [ ] Publish the demo script (cli/README.md) somewhere like docs.divviup.org

[inahga]

We may want to clarify the stability policy of divviup-api, now that we would effectively be shipping it to end-users.

I think we still want to be empowered to take incompatible CLI and configuration changes to divviup-api (the server) and the docker-compose file without having to care about compatibility. Maybe the policy ought to be "always download the docker-compose.yml file corresponding to the Janus release"?

[inahga]

It makes me somewhat uncomfortable how we rely on the integration-testing feature for local dev and demo to work.

Its purpose is to remove the authentication requirement and allow http URLs for aggregators, so that Janus integration tests that don't want to have to provision users or certificates can work.

Not sure if there's any action to take here, really. If anything we can break integration-testing into two features or runtime flags no-auth and allow-http, for clarity.

Long term I was hoping to replace the "no authentication required" part of integration-testing with a locally-running authentication provider (i.e. Keycloak) programmed with some fixture users, so we still can exercise authentication code paths in testing and dev. That of course won't work for the demo--I don't think we want to bother users with signing in with fake accounts.

[tgeoghegan]

In #1098, David notes:

This looks good, but the compose.yaml file still has one dependency on the surrounding repo, so it won't work standalone just yet. The divviup_api_vite service has a build stanza, and mounts the webapp source code. We'll need to either use the existing divviup_api image's bundled assets, or bake a new image with just a static web server, the webapp assets, and a way to mount an appropriate api_url file in as well. (the former may be trickier to set up since it uses host-based routing, but maybe we can make it work with setting the right environment variables to localhost with specific ports)

and also:

One other consideration: how will we phase updating the image version numbers in the compose file, tagging commits, and uploading release artifacts? For now, the compose file attached to a release will trail behind the version of the release itself. If we updated tags in the compose file before creating the release, then the compose file itself would have to go untested to some degree, since container images are generated after the release is created.

[tgeoghegan]

There's a race between the various DB migrate jobs (well, services, docker compose doesn't have jobs): the depends on condition is "service started", but the DB might not be up yet. I suspect we could rig up a simple healthcheck for the DB services that blocks until the DB is ready, and then we can change the depends ons in the migrator jobs.

[philips]

You may know about this but I am getting dependency failed to start: container divviup-api-docker2-divviup_api-1 is unhealthy on the compose.yaml that is checked in. Full log: https://gist.github.com/philips/c4d57b280a8e6a9fc80dd5372576120b

Also, I get an empty account list when I try to go through the README instructions.

$ ./divviup-aarch64-apple-darwin  account list
[]

[divergentdave]

I installed podman version 4.9.3 on Linux, and it's using version 2.27.1 of the docker-compose plugin as an external compose provider. With this setup, podman compose up just works for me, and all services report as healthy. The divviup-api health check just runs SELECT 1 on the database, so whatever's going on, it's probably an issue with the health check command itself, or more weird database permissions problems. It might be worth trying a different version of podman, as they've made a long series of Docker compatibility fixes over recent years.

[philips]

@divergentdave Yeah, I installed the latest, 5.1.1, and still encountering the same thing. I installed Docker Desktop and it is fine. I will just proceed from there.