search

akash-network / support

Akash Support and Issue Tracking
5 stars 4 forks source link

akash limits expose.accept http uri to a single container port #47

Open arno01 opened 2 years ago

arno01 commented 2 years ago

One cannot expose multiple container ports as http/https (i.e. non-nodePorts) as akash limits expose.accept http uri to a single container port.

multi-ports container

I've created a container which would listen on 8080, 8081, 8082 ports and reply app1, app2, app3 messages to HTTP GET (browser or curl) requests respectively.

$ docker run --rm -ti andrey01/multi-ports:0.1

$ docker inspect $(docker ps -lq) | grep IPAdd
            "SecondaryIPAddresses": null,
            "IPAddress": "172.17.0.2",
                    "IPAddress": "172.17.0.2",

$ curl 172.17.0.2:8080
app1
$ curl 172.17.0.2:8081
app2
$ curl 172.17.0.2:8082
app3

Akash SDL manifest

---
version: "2.0"

services:
  multiports:
    image: andrey01/multi-ports:0.1 
    expose:
      - port: 8080
        as: 80
        accept:
          - "app1.ingress.nixaid.com"
          - "test-a.ingress.nixaid.com"
        to:
          - global: true
      - port: 8081
        as: 80
        accept:
          - "app2.ingress.nixaid.com"
          - "test-b.ingress.nixaid.com"
        to:
          - global: true
      - port: 8082
        as: 80
        accept:
          - "app3.ingress.nixaid.com"
          - "test-c.ingress.nixaid.com"
        to:
          - global: true

profiles:
  compute:
    multiports:
      resources:
        cpu:
          units: 1.0
        memory:
          size: 128Mi 
        storage:
          size: 128Mi
  placement:
    akash:
      pricing:
        multiports:
          denom: uakt
          amount: 100

deployment:
  multiports:
    akash:
      profile: multiports
      count: 1

it only responds for the first container's port 8080, but not for 8081, 8082

$ curl app1.ingress.nixaid.com
app1
$ curl app2.ingress.nixaid.com
app1
$ curl app3.ingress.nixaid.com
app1

$ curl test-a.ingress.nixaid.com
app1
$ curl test-b.ingress.nixaid.com
app1
$ curl test-c.ingress.nixaid.com
app1

akash provider lease-status

{
  "services": {
    "multiports": {
      "name": "multiports",
      "available": 1,
      "total": 1,
      "uris": [
        "app1.ingress.nixaid.com",
        "app2.ingress.nixaid.com",
        "app3.ingress.nixaid.com",
        "na57qnd8e59lh2nfeesdsuidn4.ingress.nixaid.com",
        "test-a.ingress.nixaid.com",
        "test-b.ingress.nixaid.com",
        "test-c.ingress.nixaid.com"
      ],
      "observed_generation": 1,
      "replicas": 1,
      "updated_replicas": 1,
      "ready_replicas": 1,
      "available_replicas": 1
    }
  },
  "forwarded_ports": {}
}

from provider PoV

$ kubectl -n lease get manifest $NS -o yaml
...
spec:
  group:
    name: akash
    services:
    - count: 1
      expose:
      - external_port: 80
        global: true
        hosts:
        - app1.ingress.nixaid.com
        - test-a.ingress.nixaid.com
        http_options:
          max_body_size: 1048576
          next_cases:
          - error
          - timeout
          next_tries: 3
          read_timeout: 60000
          send_timeout: 60000
        port: 8080
        proto: TCP
      - external_port: 80
        global: true
        hosts:
        - app2.ingress.nixaid.com
        - test-b.ingress.nixaid.com
        http_options:
          max_body_size: 1048576
          next_cases:
          - error
          - timeout
          next_tries: 3
          read_timeout: 60000
          send_timeout: 60000
        port: 8081
        proto: TCP
      - external_port: 80
        global: true
        hosts:
        - app3.ingress.nixaid.com
        - test-c.ingress.nixaid.com
        http_options:
          max_body_size: 1048576
          next_cases:
          - error
          - timeout
          next_tries: 3
          read_timeout: 60000
          send_timeout: 60000
        port: 8082
        proto: TCP
      image: andrey01/multi-ports:0.1
      name: multiports
      unit:
        cpu: 1000
        memory: "134217728"
        storage: "134217728"
...

root@akash-master-01:~# kubectl -n $NS get ing -o wide
NAME                                            CLASS                 HOSTS                                           ADDRESS                                    PORTS   AGE
app1.ingress.nixaid.com                         akash-ingress-class   app1.ingress.nixaid.com                         161.97.178.239,167.86.73.47,185.211.5.95   80      2m31s
app2.ingress.nixaid.com                         akash-ingress-class   app2.ingress.nixaid.com                         161.97.178.239,167.86.73.47,185.211.5.95   80      2m31s
app3.ingress.nixaid.com                         akash-ingress-class   app3.ingress.nixaid.com                         161.97.178.239,167.86.73.47,185.211.5.95   80      2m31s
na57qnd8e59lh2nfeesdsuidn4.ingress.nixaid.com   akash-ingress-class   na57qnd8e59lh2nfeesdsuidn4.ingress.nixaid.com   161.97.178.239,167.86.73.47,185.211.5.95   80      2m32s
test-a.ingress.nixaid.com                       akash-ingress-class   test-a.ingress.nixaid.com                       161.97.178.239,167.86.73.47,185.211.5.95   80      2m31s
test-b.ingress.nixaid.com                       akash-ingress-class   test-b.ingress.nixaid.com                       161.97.178.239,167.86.73.47,185.211.5.95   80      2m31s
test-c.ingress.nixaid.com                       akash-ingress-class   test-c.ingress.nixaid.com                       161.97.178.239,167.86.73.47,185.211.5.95   80      2m31s

root@akash-master-01:~# kubectl -n $NS get svc -o wide
NAME         TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)   AGE     SELECTOR
multiports   ClusterIP   10.233.22.126   <none>        80/TCP    2m40s   akash.network/manifest-service=multiports,akash.network/namespace=e0n294nn1o9siaguq0no83hfcddp1t0scrkb3s5pf6gg2,akash.network=true

root@akash-master-01:~# kubectl -n $NS get pods -o wide
NAME                          READY   STATUS    RESTARTS   AGE     IP               NODE                         NOMINATED NODE   READINESS GATES
multiports-58d65dfd8c-54pkg   1/1     Running   0          3m23s   10.233.114.180   akash-worker-03.nixaid.com   <none>           <none>

root@akash-master-01:~# kubectl -n $NS describe svc
Name:              multiports
Namespace:         e0n294nn1o9siaguq0no83hfcddp1t0scrkb3s5pf6gg2
Labels:            akash.network=true
                   akash.network/manifest-service=multiports
                   akash.network/namespace=e0n294nn1o9siaguq0no83hfcddp1t0scrkb3s5pf6gg2
Annotations:       <none>
Selector:          akash.network/manifest-service=multiports,akash.network/namespace=e0n294nn1o9siaguq0no83hfcddp1t0scrkb3s5pf6gg2,akash.network=true
Type:              ClusterIP
IP Family Policy:  SingleStack
IP Families:       IPv4
IP:                10.233.22.126
IPs:               10.233.22.126
Port:              0-80  80/TCP
TargetPort:        8080/TCP
Endpoints:         10.233.114.180:8080
Session Affinity:  None
Events:            <none>
arno01 commented 2 years ago

cc @boz @dmikey

hydrogen18 commented 2 years ago

This is possibly a bug in the hostname operator as it may be erroneously assuming that all containers have only 1 public facing HTTP service. Fixing this may not be a high priority unless it is blocking a use case we are focusing on.

boz commented 2 years ago

Thanks @arno01!

andy108369 commented 1 year ago

I've checked the current provider-services 0.2.1. The issue persists.