search

strangelove-ventures / cosmos-operator

Cosmos Operator is a kubernetes operator for managing cosmos nodes
Apache License 2.0
80 stars 19 forks source link

CosmosFullNode sentry creates privval services. #351

Open DavidNix opened 1 year ago

DavidNix commented 1 year ago

Like p2p services, 1 per pod.

This is so remote signers can connect to a stable address.

We probably need to deprecate the readiness probe strategy. Readiness has to be turned off for sentries to be added to the service.

It's a chicken or egg problem. Comet tries to connect immediately and blocks thus failing readiness probes. The remote signer must connect to the sentry pod but can't if the sentry is not ready. (We have other safeguards for rollouts which now check the state of Comet.) So removing the probes should have little impact.

It's worth investigating how StatefulSets give each pod a stable network id.

The service is likely still necessary given the user may need to configure it. E.g. We used internal lb services on GKE.

DavidNix commented 1 year ago

We really want to explore using the pod's stable DNS name given https://github.com/strangelove-ventures/cosmos-operator/pull/360 instead of creating a service for each pod. But advantage to a service is you can configure it more, such as internal LB IPs.