systeminit / si

The System Initiative software
https://systeminit.com
Apache License 2.0
955 stars 67 forks source link

Add forklift service #4546

Closed nickgerace closed 1 week ago

nickgerace commented 1 week ago

Description

This PR adds the forklift service, which "forklifts" data into a data warehouse. Right now, it only "forklifts" one set of data: billing events. That functionality is "opt-in" and the layered load config for the forklift instance must be used to enable data warehouse stream client use.

In addition to these changes, we are publishing fully-formed billing events over NATS Jetstream in four scenarios: the HEAD change set's pointer has changed, the HEAD change set's status has changed, a resource has been created, and a resource has been deleted.

Additional Changes

nickgerace commented 1 week ago

This PR is taking on a slightly different form and introducing the new "forklift" service. Still hasn't come out of a drafts yet though.

nickgerace commented 1 week ago

Adding @sprutton1 and @johnrwatson to review since this a new service.

nickgerace commented 1 week ago

These two minor/cosmetic changes I want to make before merging, as a heads up, but this is still good to go.