Before the Document Layer binds to its listening address, it reads data from its backing FoundationDB cluster. This may present problems with health checking. Depending on how the organization's is using health checks, they may want to have health checks that succeed even if downstream dependencies are failing. For instance, if a FoundationDB database goes unavailable, the operators may not want the Document Layer instances to report as unhealthy, because that could cause a cascading series of failures. It may be preferable to have the Document Layer instances continue to receive traffic, so they can report more helpful error messages to their upstream clients.
Could we restructure this flow so that the Document Layer instances open their listening connection before reading from their FoundationDB cluster?
Before the Document Layer binds to its listening address, it reads data from its backing FoundationDB cluster. This may present problems with health checking. Depending on how the organization's is using health checks, they may want to have health checks that succeed even if downstream dependencies are failing. For instance, if a FoundationDB database goes unavailable, the operators may not want the Document Layer instances to report as unhealthy, because that could cause a cascading series of failures. It may be preferable to have the Document Layer instances continue to receive traffic, so they can report more helpful error messages to their upstream clients.
Could we restructure this flow so that the Document Layer instances open their listening connection before reading from their FoundationDB cluster?