Closed jaskiratr closed 4 years ago
hbagdi
commented
4 years ago Do note that any code running inside your k8s cluster will have access to kong's admin api without authn/authz with this setup.
jaskiratr
commented
4 years ago Thanks for looking into this @hbagdi
Port 8001 and 8444 are not exposed in the ingress-kong deployment. You will have to do that.
Just to be clear, are you suggesting to modify the docker image for ingress-kong or simple expose the ports in yaml? I'm exposing the container ports for the ingress-kong deployment and enabling those in the kong-admin service.
apiVersion: v1
kind: Service
metadata:
name: kong-admin
namespace: kong
labels:
app: kong-admin
spec:
type: ClusterIP
ports:
- name: admin
port: 8001
protocol: TCP
targetPort: 8001
- name: admin-ssl
port: 8444
targetPort: 8444
protocol: TCP
selector:
app: ingress-kong
---
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: ingress-kong
name: ingress-kong
namespace: kong
spec:
replicas: 1
selector:
matchLabels:
app: ingress-kong
template:
metadata:
annotations:
kuma.io/gateway: enabled
prometheus.io/port: "8100"
prometheus.io/scrape: "true"
traffic.sidecar.istio.io/includeInboundPorts: ""
labels:
app: ingress-kong
spec:
containers:
- env:
- name: KONG_PROXY_LISTEN
value: 0.0.0.0:8000, 0.0.0.0:8443 http2 ssl
- name: KONG_ADMIN_LISTEN
value: 0.0.0.0:8001, 0.0.0.0:8444 ssl
- name: KONG_STATUS_LISTEN
value: 0.0.0.0:8100
- name: KONG_DATABASE
value: "off"
- name: KONG_NGINX_WORKER_PROCESSES
value: "1"
- name: KONG_ADMIN_ACCESS_LOG
value: /dev/stdout
- name: KONG_ADMIN_ERROR_LOG
value: /dev/stderr
- name: KONG_PROXY_ERROR_LOG
value: /dev/stderr
image: kong:2.1
...
name: proxy
ports:
- containerPort: 8001
name: admin
protocol: TCP
- containerPort: 8444
name: admin-ssl
protocol: TCP
- containerPort: 8000
name: proxy
protocol: TCP
- containerPort: 8443
name: proxy-ssl
protocol: TCP
- containerPort: 8100
name: metrics
protocol: TCP
...I also tried implementing the following env parameter for ingress-kong deployment.
- name: KONG_ADMIN_LISTEN
value: 0.0.0.0:8001, 0.0.0.0:8444 sslIf it helps troubleshoot, when accessing the loopback, it responded with 404.
{ "message": "Not found" }
And the admin API endpoint - {{kong-admin-url}}/services/ responded with
{
"next": null,
"data": [
{
"host": "kong-admin.kong.8001.svc",
"id": "a1aec58d-f0ee-5d2e-a258-c4e3a263f255",
"protocol": "http",
"read_timeout": 60000,
"tls_verify_depth": null,
"port": 80,
"updated_at": 1597697760,
"ca_certificates": null,
"created_at": 1597697760,
"connect_timeout": 60000,
"write_timeout": 60000,
"name": "kong.kong-admin.8001",
"retries": 5,
"path": "/",
"tls_verify": null,
"client_certificate": null,
"tags": null
}
]
}Also, just wanted to clarify that I'm only trying to access the Admin API in DB-less mode for read-only purposes.
Thank you again for your help!
hbagdi
commented
4 years ago And the admin API endpoint - {{kong-admin-url}}/services/ responded with
Where are you invoking this from? Reading this line in your message makes me wonder if you have achieved what you set out to do and we can close this issue.
If not, what is the error you receive after the changes you did in your last message?
jaskiratr
commented
4 years ago Where are you invoking this from?
To access the Admin endpoint, I port-forwarded svc/kong-admin to my localhost
kubectl port-forward -n kong svc/kong-admin 8080:8001Then I made a GET request to http://localhost:8080/services
{
"next": null,
"data": [
{
"host": "kong-admin.kong.8001.svc",
...
}
]
}If not, what is the error you receive after the changes you did in your last message?
I'm accessing the Loopback endpoint via proxy, I port-forwarded svc/kong-proxy to my localhost
kubectl port-forward -n kong svc/kong-proxy 8081:8443I made the GET request to http://localhost:8081/kong-admin/services and it threw the following error. Accessing it via the AWS loadbalancer also provided the same response.
{
"message": "Not found"
}Can you do kubectl port-forward -n kong svc/kong-admin 8080:8001 and share the output of http://localhost:8080/routes?
jaskiratr
commented
4 years ago Sure. http://localhost:8080/routes
{
"next": null,
"data": [
{
"id": "fc583924-ecef-5687-94e6-fa6b384631f3",
"path_handling": "v0",
"paths": [
"/kong-admin"
],
"destinations": null,
"headers": null,
"protocols": [
"http",
"https"
],
"created_at": 1597793646,
"snis": null,
"service": {
"id": "a1aec58d-f0ee-5d2e-a258-c4e3a263f255"
},
"name": "kong.kong-loopback.00",
"hosts": null,
"preserve_host": true,
"regex_priority": 0,
"methods": null,
"sources": null,
"updated_at": 1597793646,
"https_redirect_status_code": 426,
"tags": null,
"strip_path": false
}
]
}Ah, add konghq.com/strip-path: "true" annotation to the Ingress resource kong-loopback. That should fix this problem.
jaskiratr
commented
4 years ago @hbagdi Oh! That worked. Thank you so much for taking the time to help with this.
Summary
I'm trying to implement Kong 2.1 in DB Less mode on Kubernetes. I can access the admin API through port forwarding but not via proxy loopback.
Kong Ingress controller version image: "kong-docker-kubernetes-ingress-controller.bintray.io/kong-ingress-controller:0.9.1"
Kong version image: kong:2.1
Kubernetes version
paste
kubectl versionoutputClient Version: version.Info{Major:"1", Minor:"16+", GitVersion:"v1.16.6-beta.0", GitCommit:"e7f962ba86f4ce7033828210ca3556393c377bcc", GitTreeState:"clean", BuildDate:"2020-01-15T08:26:26Z", GoVersion:"go1.13.5", Compiler:"gc", Platform:"windows/amd64"} Server Version: version.Info{Major:"1", Minor:"16", GitVersion:"v1.16.10", GitCommit:"f3add640dbcd4f3c33a7749f38baaac0b3fe810d", GitTreeState:"clean", BuildDate:"2020-05-20T13:51:56Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"}
Environment
What happened
When I access the URL, following response is returned
The logs for ingress-kong pod show the following
Expected behavior
Steps To Reproduce
Apply the following yaml files.
all-in-one-dbless.yaml
```yaml apiVersion: v1 kind: Namespace metadata: name: kong --- apiVersion: apiextensions.k8s.io/v1beta1 kind: CustomResourceDefinition metadata: name: kongclusterplugins.configuration.konghq.com spec: additionalPrinterColumns: - JSONPath: .plugin description: Name of the plugin name: Plugin-Type type: string - JSONPath: .metadata.creationTimestamp description: Age name: Age type: date - JSONPath: .disabled description: Indicates if the plugin is disabled name: Disabled priority: 1 type: boolean - JSONPath: .config description: Configuration of the plugin name: Config priority: 1 type: string group: configuration.konghq.com names: kind: KongClusterPlugin plural: kongclusterplugins shortNames: - kcp scope: Cluster validation: openAPIV3Schema: properties: config: type: object configFrom: properties: secretKeyRef: properties: key: type: string name: type: string namespace: type: string required: - name - namespace - key type: object type: object disabled: type: boolean plugin: type: string protocols: items: enum: - http - https - grpc - grpcs - tcp - tls type: string type: array run_on: enum: - first - second - all type: string required: - plugin version: v1 --- apiVersion: apiextensions.k8s.io/v1beta1 kind: CustomResourceDefinition metadata: name: kongconsumers.configuration.konghq.com spec: additionalPrinterColumns: - JSONPath: .username description: Username of a Kong Consumer name: Username type: string - JSONPath: .metadata.creationTimestamp description: Age name: Age type: date group: configuration.konghq.com names: kind: KongConsumer plural: kongconsumers shortNames: - kc scope: Namespaced validation: openAPIV3Schema: properties: credentials: items: type: string type: array custom_id: type: string username: type: string version: v1 --- apiVersion: apiextensions.k8s.io/v1beta1 kind: CustomResourceDefinition metadata: name: kongcredentials.configuration.konghq.com spec: additionalPrinterColumns: - JSONPath: .type description: Type of credential name: Credential-type type: string - JSONPath: .metadata.creationTimestamp description: Age name: Age type: date - JSONPath: .consumerRef description: Owner of the credential name: Consumer-Ref type: string group: configuration.konghq.com names: kind: KongCredential plural: kongcredentials scope: Namespaced validation: openAPIV3Schema: properties: consumerRef: type: string type: type: string required: - consumerRef - type version: v1 --- apiVersion: apiextensions.k8s.io/v1beta1 kind: CustomResourceDefinition metadata: name: kongingresses.configuration.konghq.com spec: group: configuration.konghq.com names: kind: KongIngress plural: kongingresses shortNames: - ki scope: Namespaced validation: openAPIV3Schema: properties: proxy: properties: connect_timeout: minimum: 0 type: integer path: pattern: ^/.*$ type: string protocol: enum: - http - https - grpc - grpcs - tcp - tls type: string read_timeout: minimum: 0 type: integer retries: minimum: 0 type: integer write_timeout: minimum: 0 type: integer type: object route: properties: headers: additionalProperties: items: type: string type: array type: object https_redirect_status_code: type: integer methods: items: type: string type: array path_handling: enum: - v0 - v1 type: string preserve_host: type: boolean protocols: items: enum: - http - https - grpc - grpcs - tcp - tls type: string type: array regex_priority: type: integer strip_path: type: boolean upstream: properties: algorithm: enum: - round-robin - consistent-hashing - least-connections type: string hash_fallback: type: string hash_fallback_header: type: string hash_on: type: string hash_on_cookie: type: string hash_on_cookie_path: type: string hash_on_header: type: string healthchecks: properties: active: properties: concurrency: minimum: 1 type: integer healthy: properties: http_statuses: items: type: integer type: array interval: minimum: 0 type: integer successes: minimum: 0 type: integer type: object http_path: pattern: ^/.*$ type: string timeout: minimum: 0 type: integer unhealthy: properties: http_failures: minimum: 0 type: integer http_statuses: items: type: integer type: array interval: minimum: 0 type: integer tcp_failures: minimum: 0 type: integer timeout: minimum: 0 type: integer type: object type: object passive: properties: healthy: properties: http_statuses: items: type: integer type: array interval: minimum: 0 type: integer successes: minimum: 0 type: integer type: object unhealthy: properties: http_failures: minimum: 0 type: integer http_statuses: items: type: integer type: array interval: minimum: 0 type: integer tcp_failures: minimum: 0 type: integer timeout: minimum: 0 type: integer type: object type: object threshold: type: integer type: object host_header: type: string slots: minimum: 10 type: integer type: object version: v1 --- apiVersion: apiextensions.k8s.io/v1beta1 kind: CustomResourceDefinition metadata: name: kongplugins.configuration.konghq.com spec: additionalPrinterColumns: - JSONPath: .plugin description: Name of the plugin name: Plugin-Type type: string - JSONPath: .metadata.creationTimestamp description: Age name: Age type: date - JSONPath: .disabled description: Indicates if the plugin is disabled name: Disabled priority: 1 type: boolean - JSONPath: .config description: Configuration of the plugin name: Config priority: 1 type: string group: configuration.konghq.com names: kind: KongPlugin plural: kongplugins shortNames: - kp scope: Namespaced validation: openAPIV3Schema: properties: config: type: object configFrom: properties: secretKeyRef: properties: key: type: string name: type: string required: - name - key type: object type: object disabled: type: boolean plugin: type: string protocols: items: enum: - http - https - grpc - grpcs - tcp - tls type: string type: array run_on: enum: - first - second - all type: string required: - plugin version: v1 --- apiVersion: apiextensions.k8s.io/v1beta1 kind: CustomResourceDefinition metadata: name: tcpingresses.configuration.konghq.com spec: additionalPrinterColumns: - JSONPath: .status.loadBalancer.ingress[*].ip description: Address of the load balancer name: Address type: string - JSONPath: .metadata.creationTimestamp description: Age name: Age type: date group: configuration.konghq.com names: kind: TCPIngress plural: tcpingresses scope: Namespaced subresources: status: {} validation: openAPIV3Schema: properties: apiVersion: type: string kind: type: string metadata: type: object spec: properties: rules: items: properties: backend: properties: serviceName: type: string servicePort: format: int32 type: integer type: object host: type: string port: format: int32 type: integer type: object type: array tls: items: properties: hosts: items: type: string type: array secretName: type: string type: object type: array type: object status: type: object version: v1beta1 status: acceptedNames: kind: "" plural: "" conditions: [] storedVersions: [] --- apiVersion: v1 kind: ServiceAccount metadata: name: kong-serviceaccount namespace: kong --- apiVersion: rbac.authorization.k8s.io/v1beta1 kind: ClusterRole metadata: name: kong-ingress-clusterrole rules: - apiGroups: - "" resources: - endpoints - nodes - pods - secrets verbs: - list - watch - apiGroups: - "" resources: - nodes verbs: - get - apiGroups: - "" resources: - services verbs: - get - list - watch - apiGroups: - networking.k8s.io - extensions - networking.internal.knative.dev resources: - ingresses verbs: - get - list - watch - apiGroups: - "" resources: - events verbs: - create - patch - apiGroups: - networking.k8s.io - extensions - networking.internal.knative.dev resources: - ingresses/status verbs: - update - apiGroups: - configuration.konghq.com resources: - tcpingresses/status verbs: - update - apiGroups: - configuration.konghq.com resources: - kongplugins - kongclusterplugins - kongcredentials - kongconsumers - kongingresses - tcpingresses verbs: - get - list - watch - apiGroups: - "" resources: - configmaps verbs: - create - get - update --- apiVersion: rbac.authorization.k8s.io/v1beta1 kind: ClusterRoleBinding metadata: name: kong-ingress-clusterrole-nisa-binding roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: kong-ingress-clusterrole subjects: - kind: ServiceAccount name: kong-serviceaccount namespace: kong --- apiVersion: v1 kind: Service metadata: annotations: service.beta.kubernetes.io/aws-load-balancer-backend-protocol: "tcp" service.beta.kubernetes.io/aws-load-balancer-ssl-ports: "8443" service.beta.kubernetes.io/aws-load-balancer-connection-idle-timeout: "60" service.beta.kubernetes.io/aws-load-balancer-ssl-cert: "arn:aws:acm:xxxxxxx" name: kong-proxy namespace: kong labels: app: kong-proxy spec: externalTrafficPolicy: Local ports: # - name: proxy # port: 80 # protocol: TCP # targetPort: 8000 - name: proxy-ssl port: 8443 protocol: TCP targetPort: 8000 selector: app: ingress-kong type: LoadBalancer --- apiVersion: v1 kind: Service metadata: name: kong-admin namespace: kong labels: app: kong-admin spec: type: ClusterIP ports: - name: admin port: 8001 protocol: TCP targetPort: 8001 - name: admin-ssl port: 8444 targetPort: 8444 protocol: TCP selector: app: ingress-kong --- apiVersion: v1 kind: Service metadata: name: kong-validation-webhook namespace: kong spec: ports: - name: webhook port: 443 protocol: TCP targetPort: 8080 selector: app: ingress-kong --- apiVersion: apps/v1 kind: Deployment metadata: labels: app: ingress-kong name: ingress-kong namespace: kong spec: replicas: 1 selector: matchLabels: app: ingress-kong template: metadata: annotations: kuma.io/gateway: enabled prometheus.io/port: "8100" prometheus.io/scrape: "true" traffic.sidecar.istio.io/includeInboundPorts: "" labels: app: ingress-kong spec: containers: - env: - name: KONG_PROXY_LISTEN value: 0.0.0.0:8000, 0.0.0.0:8443 http2 ssl - name: KONG_ADMIN_LISTEN value: 127.0.0.1:8001, 127.0.0.1:8444 http2 ssl - name: KONG_STATUS_LISTEN value: 0.0.0.0:8100 - name: KONG_DATABASE value: "off" - name: KONG_NGINX_WORKER_PROCESSES value: "1" - name: KONG_ADMIN_ACCESS_LOG value: /dev/stdout - name: KONG_ADMIN_ERROR_LOG value: /dev/stderr - name: KONG_PROXY_ERROR_LOG value: /dev/stderr image: kong:2.1 lifecycle: preStop: exec: command: - /bin/sh - -c - kong quit livenessProbe: failureThreshold: 3 httpGet: path: /status port: 8100 scheme: HTTP initialDelaySeconds: 5 periodSeconds: 10 successThreshold: 1 timeoutSeconds: 1 name: proxy ports: - containerPort: 8001 name: admin protocol: TCP - containerPort: 8444 name: admin-ssl protocol: TCP - containerPort: 8000 name: proxy protocol: TCP - containerPort: 8443 name: proxy-ssl protocol: TCP - containerPort: 8100 name: metrics protocol: TCP readinessProbe: failureThreshold: 3 httpGet: path: /status port: 8100 scheme: HTTP initialDelaySeconds: 5 periodSeconds: 10 successThreshold: 1 timeoutSeconds: 1 securityContext: runAsUser: 1000 - env: - name: CONTROLLER_KONG_ADMIN_URL value: https://127.0.0.1:8444 - name: CONTROLLER_KONG_ADMIN_TLS_SKIP_VERIFY value: "true" - name: CONTROLLER_PUBLISH_SERVICE value: kong/kong-proxy - name: POD_NAME valueFrom: fieldRef: apiVersion: v1 fieldPath: metadata.name - name: POD_NAMESPACE valueFrom: fieldRef: apiVersion: v1 fieldPath: metadata.namespace image: kong-docker-kubernetes-ingress-controller.bintray.io/kong-ingress-controller:0.9.1 imagePullPolicy: IfNotPresent livenessProbe: failureThreshold: 3 httpGet: path: /healthz port: 10254 scheme: HTTP initialDelaySeconds: 5 periodSeconds: 10 successThreshold: 1 timeoutSeconds: 1 name: ingress-controller ports: - containerPort: 8080 name: webhook protocol: TCP readinessProbe: failureThreshold: 3 httpGet: path: /healthz port: 10254 scheme: HTTP initialDelaySeconds: 5 periodSeconds: 10 successThreshold: 1 timeoutSeconds: 1 serviceAccountName: kong-serviceaccount ```admin-loopback.yaml
```yaml --- # Key Auth Plugin apiVersion: configuration.konghq.com/v1 kind: KongPlugin metadata: name: loopback-key-auth-plugin namespace: kong plugin: key-auth --- # ACL Plugin apiVersion: configuration.konghq.com/v1 kind: KongPlugin metadata: name: loopback-acl-plugin namespace: kong plugin: acl disabled: false config: whitelist: - admin-group hide_groups_header: true --- # Ingress apiVersion: extensions/v1beta1 kind: Ingress metadata: name: kong-loopback namespace: kong annotations: plugins.konghq.com: loopback-key-auth-plugin, loopback-acl-plugin spec: rules: - http: paths: - path: /kong-admin backend: serviceName: kong-admin servicePort: 8001 --- # Create Admin Consumer apiVersion: configuration.konghq.com/v1 kind: KongConsumer metadata: name: my-admin username: my-admin custom_id: my-admin-id --- # ACL credentials apiVersion: configuration.konghq.com/v1 kind: KongCredential metadata: name: admin-group consumerRef: my-admin type: acl config: group: admin-group --- # Key-auth apikey credentials apiVersion: configuration.konghq.com/v1 kind: KongCredential metadata: name: my-admin-apikey consumerRef: my-admin type: key-auth config: key: xxxx --- ```Port forward or go to loadbalancer URL
The following URL should redirect to admin route.
http://localhost:8081/kong-admin/?apikey=xxxxxLet me know if you have any suggestions on this issue. Thank you