NVIDIA / aistore

AIStore: scalable storage for AI applications
https://aistore.nvidia.com
MIT License
1.21k stars 160 forks source link

Can the target node in Kubernetes not require a public IP address #135

Closed yingca1 closed 1 year ago

yingca1 commented 1 year ago

Can we access the content directly through the public address of the proxy LB endpoint, instead of being redirected to the PublicNet service port of the target node with a 301 when using ais object get <object-uri>?

If I want to proxy public network access to a target node through Ingress, do I only need to handle the public_net in all endpoints of the target node? do intra_control_net and intra_data_net not need to be exposed to the public network?

{
    "tmap": {
        "PWARUOrw": {
            "intra_control_net": {
                "direct_url": "http://aistore1-target-0.aistore1-target.ais.svc.cluster.local:51082",
                "node_ip_addr": "aistore1-target-0.aistore1-target.ais.svc.cluster.local",
                "daemon_port": "51082"
            },
            "public_net": {
                "direct_url": "http://10.72.4.6:51081",
                "node_ip_addr": "10.72.4.6",
                "daemon_port": "51081"
            },
            "daemon_id": "PWARUOrw",
            "daemon_type": "target",
            "intra_data_net": {
                "direct_url": "http://aistore1-target-0.aistore1-target.ais.svc.cluster.local:51083",
                "node_ip_addr": "aistore1-target-0.aistore1-target.ais.svc.cluster.local",
                "daemon_port": "51083"
            },
            "flags": 0
        },
    }
}
saiprashanth173 commented 1 year ago

intra_control_net and intra_data_net are only for internal communication and you need not expose them externally.

To be able access data outside your cluster, you can set and it should work without any issue on GKE:

spec:
  ...
  enableExternalLB: true

see: https://github.com/NVIDIA/ais-k8s/tree/6ace0207028226796a55094a6dad5a6e40c447eb/operator#enabling-external-access

yingca1 commented 1 year ago

enableExternalLB: true No problem doing it this way, but I'm just wondering if we could expose fewer public IP+ports since we have a proxy. Why not handle all external requests through the proxy-lb?

alex-aizman commented 1 year ago

performance. The answer is, performance and scalability. Of course, we could proxy back and forth and reverse all day long. But the whole premise of aistore is different. No extra hops, not metadata servers - linear scale.