osxhacker / demo-microsite

An exploration of microsite/microservice concepts.
BSD 3-Clause "New" or "Revised" License
2 stars 0 forks source link

Autonomous Production Support #16

Open osxhacker opened 8 months ago

osxhacker commented 8 months ago

Add autonomous production support capability to the system. Specifically, introduce a new operations service which monitors system capabilities, such as the microservices, repositories, and API gateway.

Due to the nature of this feature, a reactive system approach is applicable. Encoding decision support rules normally required of an SRE will be done with Drools.

Percepts related to this effort include, but are not limited to:

When deemed applicable based on observed facts and support rules, actionable recommendations must be emitted. A departure from the "observation-then-recommendation" pattern is when Kafka is observed to be offline. In this case, the service must operate in "emergency mode" and act to shut down the system.

osxhacker commented 8 months ago

~Due to the intricacy of this enhancement, work is planned to be introduced to master in multiple atomic feature enhancement branches. Each will be documented with a comment added to this issue.~

Once the new service is deemed feature complete, this ticket can be closed.

osxhacker commented 8 months ago

This feature presents an opportunity to explore multi-repository system concerns as they pertain to CircleCI/GitHub functionality, such as GitHub Actions and GitHub Packages. As such, the Autonomous Production Support detail design and implementation artifacts shall reside in a separate git repository.

There will inevitably be demo-microsite artifact impacts required to support this approach. Individual work items be pursued as normal, including issues created to track same as applicable.

osxhacker commented 7 months ago

Refactored to introduce support for multi-repository system development with PR #18