search

Comcast / sirius

A distributed system library for managing application reference data
http://comcast.github.io/sirius/
Apache License 2.0
298 stars 49 forks source link

Add BrainlessRequestHandler case object to skip Sirius bootstrap #137

Closed HaloFour closed 5 years ago

HaloFour commented 5 years ago

We have a number of XtvAPI nodes that don't populate an in-memory brain but instead exist to facilitate Sirius. This includes the ingest cluster which handle PAXOS consensus as well as nodes that are used strictly for Sirius catchup. We pass no-op implementations of RequestHandler but Sirius still goes through a full replay of the Uberstore which can add 20-30 minutes to the time it takes to bootstrap the service. As far as I could tell that replay only serves to populate the brain and other housekeeping such as index repair happen beforehand, so it should be safe to skip the log replay.

I propose a special case object implementation of RequestHandler that Sirius would recognize and if that is the instance of the RequestHandler passed to SiriusFactory then log replay will be skipped.

clinedome-work commented 5 years ago

Oof, both PRs have changes to the StateSup. In the other, it was changing the foldLeft to a foreach, and this one completely subsumes that but is in conflict. You'll need to resolve it, I can make the easy fix but can't push it to your branch. :|

HaloFour commented 5 years ago

I expected this, I'll merge from master