⚠️⚠️ WARNING! ⚠️⚠️ The Materialize K8s-eip-operator will soon be archived and is no longer be under active development.
Manage external connections to Kubernetes pods or nodes using AWS Elastic IPs (EIPs).
This operator manages the following:
eip.materialize.cloud/eip_uid
) for later identification.external-dns.alpha.kubernetes.io/target
on the pod, so that the external-dns
operator knows to use the IP of the EIP instead of the external IP of the primary ENI on the host.You must be using the AWS VPC CNI plugin. This is the default for AWS EKS, so you probably don't need to change anything here.
You must set AWS_VPC_K8S_CNI_EXTERNALSNAT=true
on the aws-node daemonset.
kubectl set env daemonset -n kube-system aws-node AWS_VPC_K8S_CNI_EXTERNALSNAT=true
See the AWS external-snat docs for more details.
For external-dns
support, you must be using a version with headless ClusterIp support, either by waiting until this PR is merged or by using a fork with it already included.
{
"Version": "2012-10-17",
"Statement": [
{
"Action": [
"ec2:AllocateAddress",
"ec2:ReleaseAddress",
"ec2:DescribeAddresses",
"ec2:AssociateAddress",
"ec2:DisassociateAddress",
"ec2:DescribeNetworkInterfaces",
"ec2:CreateTags",
"ec2:DeleteTags",
"ec2:DescribeInstances",
"ec2:ModifyNetworkInterfaceAttribute",
"servicequotas:GetServiceQuota"
],
"Effect": "Allow",
"Resource": "*"
}
]
}
apiVersion: v1
kind: ServiceAccount
metadata:
name: eip-operator
annotations:
eks.amazonaws.com/role-arn: arn:aws:iam::ACCOUNT-ID:role/IAM-SERVICE-ROLE-NAME
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: eip-operator
rules:
- apiGroups: [""]
resources: ["pods", "pods/status"]
verbs: ["get", "watch", "list", "update", "patch"]
- apiGroups: [""]
resources: ["nodes", "nodes/status"]
verbs: ["get"]
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: eip-operator-viewer
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: eip-operator
subjects:
- kind: ServiceAccount
name: eip-operator
namespace: default
CLUSTER_NAME
environment variable. NAMESPACE
and DEFAULT_TAGS
are optional.
If you do not set the NAMESPACE
environment variable, the eip-operator will operate on all namespaces.
Even in this global mode, the eip and pod must be in the same namespace.
apiVersion: apps/v1
kind: Deployment
metadata:
name: eip-operator
spec:
strategy:
type: Recreate
selector:
matchLabels:
app: eip-operator
template:
metadata:
labels:
app: eip-operator
spec:
containers:
- name: eip-operator
image: materialize/k8s-eip-operator:latest
env:
- name: NAMESPACE
value: default
- name: CLUSTER_NAME
value: my-example-cluster
- name: DEFAULT_TAGS
value: '{"tag1": "value1", "tag2": "value2"}'
Instantiate an Eip Kubernetes object, specifying the podName
in the spec.
apiVersion: "materialize.cloud/v2"
kind: Eip
metadata:
name: my-new-eip
spec:
selector:
pod:
podName: my-pod
Add the eip.materialize.cloud/manage=true
label to the pod with name matching the podName
specified above.
No need to manually create the Eip in this case, the operator can do it for you.
Add both the eip.materialize.cloud/manage=true
and eip.materialize.cloud/autocreate_eip=true
labels to your pod.
Do NOT manually create the Eip Kubernetes object if setting the eip.materialize.cloud/autocreate_eip=true
label, or the two objects will fight over your pod.
apiVersion: "materialize.cloud/v2"
kind: Eip
metadata:
name: my-new-eip
spec:
selector:
node:
selector:
some-label: some-value
some-other-label: some-other-value
The node selector should contain a set of labels which should match a single node - if multiple nodes are matched, the EIP will be attached to one of them arbitrarily.
Add the eip.materialize.cloud/manage=true
label to the node whose labels match the labels in the selector.
If using Cilium in ENI mode, you can still use this operator, but you will need to disable masquerade for pods with EIPs assigned. Cilium (as of 1.12.0) does not seem to support configuring masquerade on a per-pod basis, so you will need to do one of the following:
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: cilium-eip-no-masquerade-agent
spec:
selector:
matchLabels:
app: cilium-eip-no-masquerade-agent
template:
metadata:
labels:
app: cilium-eip-no-masquerade-agent
spec:
containers:
- command:
- ./cilium-eip-no-masquerade-agent
env:
- name: RUST_LOG
value: INFO
image: materialize/k8s-eip-operator
name: eip-operator
securityContext:
privileged: true
dnsPolicy: ClusterFirstWithHostNet
hostNetwork: true
restartPolicy: Always
serviceAccount: eip-operator
serviceAccountName: eip-operator
./bin/run-tests
We now have support for sending traces using the OpenTelemetry OTLP format. This is configured through environment variables:
OPENTELEMETRY_ENDPOINT
is the endpoint to send the logs to.
OPENTELEMETRY_HEADERS
is a json formatted map of key/value pairs to be included in the GRPC request headers.
OPENTELEMETRY_TOPLEVEL_FIELDS
is a json formatted map of key/value pairs to be included in all trace spans emitted from the service.
OPENTELEMETRY_LEVEL_TARGETS
is a tracing
crate level filter. Defaults to "DEBUG"
.
OPENTELEMETRY_SAMPLE_RATE
is a float value controlling the trace sample rate. Default is 0.05.