AlexanderYastrebov
commented
1 year ago What is the skipper version you run? Please use one of the recent versions from https://github.com/zalando/skipper/releases
Open vcozzolino opened 1 year ago
AlexanderYastrebov
commented
1 year ago What is the skipper version you run? Please use one of the recent versions from https://github.com/zalando/skipper/releases
vcozzolino
commented
1 year ago I just noticed that the daemonset was pointing at an older version of Skipper. I replaced the image tag with the one you mentioned and it works fine now. Maybe the YAML could be updated to point at the latest tag :).
What about the PodSecurityPolicy issue? Is it OK to ignore it?
Thanks for the help!
szuecs
commented
1 year ago PodSecurityPolicy is the issue, yes Please use something else to allow hostnetwork or hostport as you wish. I would also recommend to use the deployment instead of daemonset.
szuecs
commented
1 year ago @vcozzolino can you try to drop the psp definition from docs/kubernetes/deploy/daemonset/rbac.yaml?
Here what I would guess you need to have in rbac.yaml:
apiVersion: v1
kind: ServiceAccount
metadata:
name: skipper-ingress
namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: skipper-ingress
rules:
- apiGroups:
- networking.k8s.io
resources:
- ingresses
verbs:
- get
- list
- apiGroups: [""]
resources:
- namespaces
- services
- endpoints
- pods
verbs:
- get
- list
- apiGroups:
- zalando.org
resources:
- routegroups
verbs:
- get
- list
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: skipper-ingress
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: skipper-ingress
subjects:
- kind: ServiceAccount
name: skipper-ingress
namespace: kube-systemmodified daemonset to install a recent version:
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: skipper-ingress
namespace: kube-system
labels:
application: skipper-ingress
version: v0.16.94
component: ingress
spec:
selector:
matchLabels:
application: skipper-ingress
updateStrategy:
type: RollingUpdate
template:
metadata:
name: skipper-ingress
labels:
application: skipper-ingress
version: v0.16.94
component: ingress
spec:
priorityClassName: system-node-critical
serviceAccountName: skipper-ingress
tolerations:
- key: dedicated
operator: Exists
nodeSelector:
kubernetes.io/role: worker
hostNetwork: true
containers:
- name: skipper-ingress
image: registry.opensource.zalan.do/teapot/skipper:v0.16.94
ports:
- name: ingress-port
containerPort: 9999
hostPort: 9999
- name: metrics-port
containerPort: 9911
args:
- "skipper"
- "-kubernetes"
- "-kubernetes-in-cluster"
- "-kubernetes-path-mode=path-prefix"
- "-address=:9999"
- "-wait-first-route-load"
- "-proxy-preserve-host"
- "-serve-host-metrics"
- "-enable-ratelimits"
- "-experimental-upgrade"
- "-metrics-exp-decay-sample"
- "-reverse-source-predicate"
- "-lb-healthcheck-interval=3s"
- "-metrics-flavour=codahale,prometheus"
- "-enable-connection-metrics"
- "-max-audit-body=0"
- "-histogram-metric-buckets=.01,.025,.05,.075,.1,.2,.3,.4,.5,.75,1,2,3,4,5,7,10,15,20,30,60,120,300,600"
resources:
requests:
cpu: 150m
memory: 150Mi
readinessProbe:
httpGet:
path: /kube-system/healthz
port: 9999
initialDelaySeconds: 5
timeoutSeconds: 5
securityContext:
readOnlyRootFilesystem: true
runAsNonRoot: true
runAsUser: 1000If you want to have routegroups enabled (error logs look like this is missing) apply this:
# This is a copy of https://github.com/szuecs/routegroup-client/blob/master/zalando.org_routegroups.yaml
# DO NOT EDIT.
---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.8.0
creationTimestamp: null
name: routegroups.zalando.org
spec:
group: zalando.org
names:
categories:
- all
kind: RouteGroup
listKind: RouteGroupList
plural: routegroups
shortNames:
- rg
- rgs
singular: routegroup
scope: Namespaced
versions:
- additionalPrinterColumns:
- description: Hosts defined for the RouteGroup
jsonPath: .spec.hosts
name: Hosts
type: string
- description: Address of the Load Balancer for the RouteGroup
jsonPath: .status.loadBalancer
name: Address
type: string
name: v1
schema:
openAPIV3Schema:
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type: string
kind:
description: 'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
metadata:
type: object
spec:
properties:
backends:
description: List of backends that can be referenced in the routes.
items:
properties:
address:
description: Address is required for Type network
type: string
algorithm:
description: Algorithm is required for Type lb
enum:
- roundRobin
- random
- consistentHash
- powerOfRandomNChoices
type: string
endpoints:
description: Endpoints is required for Type lb
items:
type: string
minItems: 1
type: array
name:
description: Name is the BackendName that can be referenced
as RouteGroupBackendReference
type: string
serviceName:
description: ServiceName is required for Type service
type: string
servicePort:
description: ServicePort is required for Type service
type: integer
type:
description: Type is one of "service|shunt|loopback|dynamic|lb|network"
enum:
- service
- shunt
- loopback
- dynamic
- lb
- network
type: string
required:
- name
- type
type: object
type: array
defaultBackends:
description: DefaultBackends is a list of default backends defined
if no explicit backend is defined for a route.
items:
properties:
backendName:
description: BackendName references the skipperBackend by name
type: string
weight:
description: Weight defines the traffic weight, if there are
2 or more default backends
minimum: 0
type: integer
required:
- backendName
type: object
type: array
hosts:
description: List of hostnames for the RouteGroup.
items:
pattern: "^[a-z0-9]([-a-z0-9]*[a-z0-9])?([.][a-z0-9]([-a-z0-9]*[a-z0-9])?)*$"
type: string
type: array
routes:
items:
properties:
backends:
description: RouteGroupBackendReference specifies the list of
backendReference that should be applied to override the defaultBackends
items:
properties:
backendName:
description: BackendName references the skipperBackend
by name
type: string
weight:
description: Weight defines the traffic weight, if there
are 2 or more default backends
minimum: 0
type: integer
required:
- backendName
type: object
type: array
filters:
description: Filters specifies the list of filters applied to
the routeSpec
items:
type: string
type: array
methods:
description: Methods defines valid HTTP methods for the specified
routeSpec
items:
description: HTTPMethod is a valid HTTP method in uppercase.
enum:
- GET
- HEAD
- POST
- PUT
- PATCH
- DELETE
- CONNECT
- OPTIONS
- TRACE
type: string
type: array
path:
description: Path specifies Path predicate, only one of Path
or PathSubtree is allowed
type: string
pathRegexp:
description: PathRegexp can be added additionally
type: string
pathSubtree:
description: PathSubtree specifies PathSubtree predicate, only
one of Path or PathSubtree is allowed
type: string
predicates:
description: Predicates specifies the list of predicates applied
to the routeSpec
items:
type: string
type: array
type: object
minItems: 1
type: array
required:
- backends
type: object
status:
properties:
loadBalancer:
description: LoadBalancer is similar to ingress status, such that
external-dns has the same style as in ingress
properties:
routegroup:
description: RouteGroup is similar to Ingress in ingress status.LoadBalancer.
items:
properties:
hostname:
description: Hostname is the hostname of the load balancer
and is empty if IP is set
type: string
ip:
description: IP is the IP address of the load balancer and
is empty if Hostname is set
type: string
type: object
type: array
required:
- routegroup
type: object
required:
- loadBalancer
type: object
required:
- spec
type: object
served: true
storage: true
subresources:
status: {}
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []Related #2343
Describe the bug Following the instructions found at this link https://opensource.zalando.com/skipper/kubernetes/ingress-controller/ do not lead to successfully deploying the Skipper ingress controller on Kubernetes version 1.27. The Skipper binary generated with
make buildworks when launched from a shell.To Reproduce
kubectl create -f docs/kubernetes/deploy/daemonsetExpected behavior
Skipper YAML files created correctly and ingress-controller pods are running.
Observed behavior
Running
kubectl create -f docs/kubernetes/deploy/daemonsetalready generates an error due to the deprecation ofPodSecurityPolicyas of version v1.25 of Kubernetes (https://kubernetes.io/docs/reference/using-api/deprecation-guide/#psp-v125):On top of that, Skipper fails to run and generates the following output: