Support LB attached to private network without enabling route controller

[jtackaberry]

TL;DR

I am using Calico on microk8s for networking and IPAM. I'd like to be able to attach a hccm-provisioned LB to a private network without having to enable full networking.

Expected behavior

My goal is to use Calico for networking/IPAM but deploy a LB attached to a private network, addressing the backend servers via private IPs.

The README says:

If you manage the network yourself it might still be required to let the CCM know about private networks. You can do this by adding the environment variable with the network name/ID in the CCM deployment.

I do manage the network myself, so ok, I included the network name (unimaginatively default) in the hcloud secret, and added this stanza to my Helm chart values file:

env:
  HCLOUD_NETWORK:
    valueFrom:
      secretKeyRef:
        name: hcloud
        key: network

Unfortunately, even though I have left networking.enabled as false with the chart, merely defining HCLOUD_NETWORK as above seems to auto-enable the route controller, and I get the error:

E1012 01:19:20.429965 1 controllermanager.go:322] Error starting "node-route-controller" F1012 01:19:20.429992 1 controllermanager.go:224] error running controllers: invalid CIDR[0]: (invalid CIDR address: )

So I next turned to load_balancer.go for annotations (BTW are these annotations documented anywhere or is the source code the only place to see what's available?) and was hoping to see some means of attaching an LB to a private network.

I see load-balancer.hetzner.cloud/use-private-ip which is obviously going to be a prerequisite, but this alone isn't sufficient:

I1012 01:30:25.321003 1 event.go:389] "Event occurred" object="ingress-nginx/ingress-nginx-controller" fieldPath="" kind="Service" apiVersion="v1" type="Warning" reason="SyncLoadBalancerFailed" message="Error syncing load balancer: failed to ensure load balancer: hcloud/loadBalancers.EnsureLoadBalancer: hcops/LoadBalancerOps.ReconcileHCLBTargets: use private ip: missing network id"

Makes sense, because HCCM doesn't know the network id. But nor do I see any way to actually tell it the id without enabling the route controller. Ultimately that's the nature of this FR: allow configuring the network id with HCCM, preferably using the HCLOUD_NETWORK env var approach, without implying the enablement of the route controller.

(Apologies if this ends up being more a request for cluebatting than a feature request.)

[mehrdadpfg]

we have the same issue ( we also happen to have robot servers in our cluster), I went the exact same path but there is no luck to pass network id to controller without it automatically want to run routes controller. I believe this is more a bug rather than being an enhancement.

main.go:67] Cloud provider could not be initialized: could not init cloud provider "hcloud": using Routes with Robot is not supported

update: tried with these two settings that I have found but still no luck

--configure-cloud-routes=false
--controllers=-node-route-controller

[mehrdadpfg]

@jtackaberry I have figured it out you should set this variable in your deployment and then you can pass the networkID without crash

HCLOUD_NETWORK_ROUTES_ENABLED: false

[jtackaberry]

@mehrdadpfg ah wonderful! I somehow missed that nugget in my trawling of GitHub issues. Tested that out and it works. Thanks!

If only it was documented. Perhaps this issue is worth leaving open from a documentation perspective (which is perhaps my real only complaint about this project in general). I can submit a PR for the README.

[mehrdadpfg]

yeah i totally agree, actually i did not find it in issues until you mentioned i crawled through the source code to find it. its very confusing and ambiguous .