search

GSA-TTS / jemison

An exploration of the space of search
Other
0 stars 0 forks source link

:vertical_traffic_light: Add `admin` component/CLI #8

Closed jadudm closed 1 day ago

jadudm commented 5 days ago

At a glance

In order to manage the living application as a system owner I want knobs, buttons, and pictures with boxes and arrows on the back of each one.

Acceptance Criteria

We use DRY behavior-driven development wherever possible.

### then... 
- [ ] Remove end-user API interactions from existing services
- [ ] Add `admin` service
- [ ] Create `admin` CLI (to simplify API interactions, [tview](https://pkg.go.dev/github.com/rivo/tview))

Shepherd

Background

Currently, one or two services have an integrated API. This is undesirable: it creates a larger security surface that we do not want to manage. We want all components to only talk to two things: the queue (Postgres) and the object store (S3). The only components that should have outside interactions are:

  1. the fetch service (which retrieves and stores web pages),
  2. the serve service (which receives and processes queries via API), and
  3. the admin service (which receives and processes administrative requests via API)

The admin service then interacts with other components entirely via queue. If we ever want to enable direct service-to-service communication, we will do so 1) via API, and 2) with shared secrets, randomly generated at deploy time, that only the services know about.

The admin service should leverage an SQLite database for its configuration/storage. That DB can be fetched from S3 at startup, and stashed back to S3 periodically/after changes. (Or, we can use Postgres... but, I would rather PG remains for the queues only unless we have a compelling reason to do otherwise.) This eliminates issues of migrations, etc., as the DB is always directly syncd to this service. (It is also easily observable, locally and in production.)

Security Considerations

Required per CM-4.

We will use API keys for access, and there will be two headers:

We consider the key id to be public information, and the key itself to be a secret. In this way, we will encode the approved key ids directly into the repository, and the keys will remain secret/with individuals/deploy secrets. As a result, access to the admin service is via PR to the repository, and a deploy must take place for access to be granted.

This will also leave the door open for integration with api.data.gov.

Process checklist - [ ] Has a clear story statement - [ ] Can reasonably be done in a few days (otherwise, split this up!) - [ ] Shepherds have been identified - [ ] UX youexes all the things - [ ] Design designs all the things - [ ] Engineering engineers all the things - [ ] Meets acceptance criteria - [ ] Meets [QASP conditions](https://derisking-guide.18f.gov/qasp/) - [ ] Presented in a review - [ ] Includes screenshots or references to artifacts - [ ] Tagged with the sprint where it was finished - [ ] Archived ### If there's UI... - [ ] Screen reader - Listen to the experience with a screen reader extension, ensure the information presented in order - [ ] Keyboard navigation - Run through acceptance criteria with keyboard tabs, ensure it works. - [ ] Text scaling - Adjust viewport to 1280 pixels wide and zoom to 200%, ensure everything renders as expected. Document 400% zoom issues with USWDS if appropriate.
jadudm commented 1 day ago

I'm going to move this to... blocked? Actually, I'm going to close it as not planned.

Rationale

Anywhere I have user interaction, I need to deal with authentication and an attack surface.

So. Lets remove it.

The reason for this is to provide search/scrape/fetch/other configuration data. This data does not change often. Therefore, we could do all configuration through static files, commit them to a repository (using our access controls there, and review processes to gate changes), and then push/pull from that repository for system configuration changes.

This eliminates a huge number of moving pieces. It is easier to verify/validate, and saves a ton of work.

So.

It'll have to go