Closed Anmol1696 closed 8 months ago
For local setup: https://mjpitz.com/blog/2020/10/21/local-ingress-domains-kind/
It might make more sense to divide the ingress problem into local and on cloud.
Port-forwarding can also be standardized with shell scripts to make the process smooth for users.
In https://github.com/osmosis-labs/mesh-security-infra/pull/8, got the ingress working with following setting:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: nginx-ingress
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /$1
nginx.ingress.kubernetes.io/use-regex: "true"
cert-manager.io/issuer: "cert-issuer"
spec:
ingressClassName: nginx
tls:
- hosts:
- explorer.thestarship.io
secretName: explorer.nginx-ingress-tls
- hosts:
- registry.thestarship.io
secretName: registry.nginx-ingress-tls
- hosts:
- rest.mesh-1-genesis.thestarship.io
secretName: rest.mesh-1-genesis.nginx-ingress-tls
- hosts:
- rpc.mesh-1-genesis.thestarship.io
secretName: rpc.mesh-1-genesis.nginx-ingress-tls
rules:
- host: "explorer.thestarship.io"
http:
paths:
- pathType: Prefix
path: "/"
backend:
service:
name: explorer
port:
name: http
- host: "registry.thestarship.io"
http:
paths:
- pathType: ImplementationSpecific
path: "/(.*)"
backend:
service:
name: registry
port:
name: http
- host: "rest.mesh-1-genesis.thestarship.io"
http:
paths:
- pathType: ImplementationSpecific
path: "/(.*)"
backend:
service:
name: mesh-1-genesis
port:
name: rest
- pathType: ImplementationSpecific
path: "/faucet/(.*)"
backend:
service:
name: mesh-1-genesis
port:
name: faucet
- pathType: ImplementationSpecific
path: "/exposer/(.*)"
backend:
service:
name: mesh-1-genesis
port:
name: exposer
- host: "rpc.mesh-1-genesis.thestarship.io"
http:
paths:
- pathType: ImplementationSpecific
path: "/(.*)"
backend:
service:
name: mesh-1-genesis
port:
name: rpc
Couple of open questions now?
Should ingress be made part of the starship config file, or left to the overlays
with generated yaml files.
Can look in config like:
ingress:
enabled: true
ingressClassName: nginx
domain: "thestarship.io" ## specifies the domain that is configured
## maybe
cert-manager:
issuer: "cert-issuer"
## TODO: better name
genesis-only: true ## if false, will create rules for all the validator nodes, exposing each one individually
From this following domains need to be predefined: explorer.<domain>
, registry.<domain>
, <chain-name>.<domain>
...
Ingress.yaml file will keep getting longer, it would be really benificial to be able to generate ingress yaml file.
Why this should not be done?
How will ingress effect max number of validator nodes that we can setup down the line? How many ingress rules are allowed What is the additional overhead?
ingress-controller
and cert-manager
part of deploymentMaybe not since this can be cluster wide, we can not assume people just have the whole cluster for Starship only. This is what makes ingress and certs even more tricky, we wont be able to enforce something from Starship.
Cert-manager also configured with: https://cert-manager.io/docs/tutorials/acme/nginx-ingress/
Fk. Ingress seems to be frozen. Newer paradigm of Gateway API seems to be hot thing. https://kubernetes.io/docs/concepts/services-networking/gateway/
We should move away from ingress as well then, before we start.
Overview
Currently we are using
kubectl port-forward
to get access to the RPC ports and expose the chains. We relay on this for both local as well as remote cluster on a k8s digital ocean.Proposal
Inorder to unify the experince of everyone down the line, we dont expect people to use
kubectl
at all. The proposal here is to use ingress to expose the RPC ports for various chains. We can use the following ingressThere are a coupld of considerations to be made here:
rpc.<node>.<chain>.shuttle.one
?For local setup, need to map the hosts to the IPs and for remote need to create cloudflare rules with tls ending secrets.