Access to the ClusterIP service from the host

dee-kryvenko commented 2 years ago

Rancher Desktop Version

0.7.1

Rancher Desktop K8s Version

1.20.14

What operating system are you using?

macOS

Operating System / Build Version

11.6.2 (20G314)

What CPU architecture are you using?

x64

Windows User Only

No response

Actual Behavior

I want to be able to access ClusterIP service from my host and it seems like it is not automatically forwarded/routed.

Steps to Reproduce

kubectl run -it --rm $(basename $(pwd)) --image=$(basename $(pwd)):latest --restart=Never --image-pull-policy=Never --port 5000 --expose

Produces:

apiVersion: v1
kind: Service
metadata:
  creationTimestamp: "2022-01-18T18:19:02Z"
  name: foo
  namespace: default
  resourceVersion: "1121"
  uid: 1ee138dc-efbf-435d-9603-70cc4f49c710
spec:
  clusterIP: 10.43.72.11
  clusterIPs:
  - 10.43.72.11
  ports:
  - port: 5000
    protocol: TCP
    targetPort: 5000
  selector:
    run: foo
  sessionAffinity: None
  type: ClusterIP
status:
  loadBalancer: {}

Result

> curl -L http://localhost:5000/
curl: (7) Failed to connect to localhost port 5000: Connection refused
> curl -L http://10.43.72.11:5000/
curl: (7) Failed to connect to 10.43.72.11 port 5000: Operation timed out
> curl -L http://foo:5000/
curl: (7) Failed to connect to foo port 5000: Operation timed out

Expected Behavior

I was expecting to be able to access such a service. I understand there is traefik and how to make it work with ingress - but kubectl run does not support adding ingress automatically. I am just trying to figure out easiest way to test containers locally (just containers, not deployments - so trying to stay away from helm/kustomize for that level of testing).

Additional Information

No response

jandubois commented 2 years ago

Could you include a repro example in the report? I need an actual image, and the commands and/or config files to deploy it, in case the issue is with the image.

dee-kryvenko commented 2 years ago

The steps are all the same for public image. Something obvious like nginx uses port 80 unfortunately which usually inaccessible, but tomcat image uses 8080:

kubectl run -it --rm test --image=tomcat --port 5000 --expose

Takes a while to start - Java...

apiVersion: v1
kind: Service
metadata:
  creationTimestamp: "2022-01-19T01:07:52Z"
  name: test
  namespace: default
  resourceVersion: "18343"
  uid: ab5a96d9-d001-4206-93f4-3bf653514380
spec:
  clusterIP: 10.43.4.254
  clusterIPs:
  - 10.43.4.254
  ports:
  - port: 8080
    protocol: TCP
    targetPort: 8080
  selector:
    run: test
  sessionAffinity: None
  type: ClusterIP
status:
  loadBalancer: {}

jandubois commented 2 years ago

kubectl run -it --rm test --image=tomcat --port 5000 --expose

You have to use port 8080, but I assume it is just a cut&pasto.

I can talk to the service from a different pod as expected, e.g.

root@nginx:/# curl 10.43.227.181:8080
<!doctype html><html lang="en"><head><title>HTTP Status 404 – Not Found</title><style type="text/css">body {font-family:Tahoma,Arial,sans-serif;} h1, h2, h3, b {color:white;background-color:#525D76;} h1 {font-size:22px;} h2 {font-size:16px;} h3 {font-size:14px;} p {font-size:12px;} a {color:black;} .line {height:1px;background-color:#525D76;border:none;}</style></head><body><h1>HTTP Status 404 – Not Found</h1><hr class="line" /><p><b>Type</b> Status Report</p><p><b>Description</b> The origin server did not find a current representation for the target resource or is not willing to disclose that one exists.</p><hr class="line" /><h3>Apache Tomcat/10.0.14</h3></body></html>root@nginx:/#

I want to be able to access ClusterIP service from my host and it seems like it is not automatically forwarded/routed.

If you want to access the service from outside the cluster, you need to switch to a LoadBalancer type, as ClusterIP is strictly for connections within the cluster. It is not reachable from the guest VM, so can't be forwarded to the host either.

I've edited the service to change the type and could then access it from the host via the external IP on the bridged network.

dee-kryvenko commented 2 years ago

Yeah, it was a typo - I did use --port 8080, and yes - I meant the host to pod and not pod to pod connections. Unfortunately kubectl run --expose creates ClusterIP and not a LoadBalancer, so the user has no control over it. It is not a problem for me personally, as we figured out in Slack why nerdctl run didn't work (it wanted -n k8s.io), so I am all good here. kubectl run is positioned as a "docker run alternative for beginners" so it is not expected the user will know how to change service type or what is the service, and makes me think the intention of that command is to expose the port to the host/externally, and it doesn't seem to work with RD out of the box. Although it probably doesn't work with Docker for Desktop neither and I probably misinterpreting what --expose means. If you think that it works as expected - feel free to close this issue, and thanks for helping me out with nerdctl run in slack!

rancher-sandbox / rancher-desktop