Improve xDS health check

[jrhee17]

Motivation:

It is recommended to review https://github.com/line/armeria/pull/5802 prior to this PR

This changeset attempts to solve several problems:

Custom filter logic

xDS considers all endpoints when computing whether a PrioritySet is in panic state. For instance, if the percentage of unhealthy endpoints exceeds a preconfigured panic threshold, the endpoint selection includes all endpoints regardless of the degraded status. While armeria supports an HealthCheckedEndpointGroup out of the box, it filters out healthy endpoints automatically.

• https://github.com/line/armeria/blob/72ebfe12abf7804723a490fe7acfe3a45d46089f/xds/src/main/java/com/linecorp/armeria/xds/client/endpoint/DefaultLoadBalancer.java#L99-L105
• ref: https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/upstream/load_balancing/panic_threshold

In order to resolve this, I propose that a AbstractHealthCheckedEndpointGroupBuilder#healthCheckedEndpointPredicate API is added

Per-cluster health check configuration

Per-cluster member health check is difficult with the current API since a single parameter set is statically defined for an entire health checked endpoint group.

• ref: https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/upstream/health_checking#per-cluster-member-health-check-config

We already have an abstraction AbstractHealthCheckedEndpointGroupBuilder#newCheckerFactory. I propose that this API be used for the purpose of xDS. In order to support the parameters xDS allows configuring, I propose that parameters are passed to HttpHealthChecker via the constructor instead of the HealthCheckerContext. In order to support this change, HttpHealthChecker has been moved to an internal package so the xds module can also access it.

Modifications:

• Added APIs AbstractHealthCheckedEndpointGroupBuilder#healthCheckedEndpointPredicate and modified HealthCheckedEndpointGroup to filter endpoints based on the predicate.
  • Health checked Endpoints now have attributes HEALTHY and DEGRADED set.
  • HealthCheckedEndpointGroup#setEndpoints is now called on every invocation of updateHealth, not just when a health status changes.
• xDS sometimes needs to override the health checked Endpoint, so modified to receive health checked parameters from the HttpHealthChecker constructor instead of HealthCheckerContext.
  • In the process, HttpHealthChecker has been moved to an internal package
  • HealthCheckerContext#originalEndpoint has been added to retrieve the original endpoint. The other APIs haven't been deprecated since it is potentially useful information that other HealthCheckers may be using.
• Introduced a XdsHealthCheckedEndpointGroupBuilder which implements its own checkerFactory.
• EndpointUtil#coarseHealth has been updated to consider HEALTHY, DEGRADED attributes when determining health.

Result:

• The behavior of XdsEndpointGroup is more aligned with envoy in terms of health checking
• Preparation for zone-aware load balancing

[github-actions[bot]]

🔍 Build Scan® (commit: 0ae3789f34d1ba9a3a1d2b2088a003ea7b39eff2)

Job name	Status	Build Scan®
build-self-hosted-unsafe-jdk-8	✅	https://ge.armeria.dev/s/vtgtvqk2ryxde
build-self-hosted-unsafe-jdk-21-snapshot-blockhound	✅	https://ge.armeria.dev/s/lsxzwaxry5gek
build-self-hosted-unsafe-jdk-17-min-java-17-coverage	✅	https://ge.armeria.dev/s/6y6j2wenyqrum
build-self-hosted-unsafe-jdk-17-min-java-11	✅	https://ge.armeria.dev/s/eecrpuk5naj4c
build-self-hosted-unsafe-jdk-17-leak	✅	https://ge.armeria.dev/s/hye3e2lgci4qq
build-self-hosted-unsafe-jdk-11	✅	https://ge.armeria.dev/s/dns2eeungr3xy
build-macos-12-jdk-21	✅	https://ge.armeria.dev/s/qlrls3iw3xhyu

[jrhee17]

This PR is now back to reliably passing the CI