Only using a default network named 'default' works

[kinghrothgar]

The firewall rule fails to be created because our default network is not named 'default.' This would be fixed by either of these 2 options:

1. Adding an argument to allow specifying the network
2. Retrieving the default network from GCE

[G-Harmon]

All else being equal, I would say it's better not to add an argument if the tool can figure it out.

[nikhiljindal]

I am curious if this works fine for a single cluster ingress? Does the ingress-gce controller retrieve the right default network before creating the firewall rule?

If yes, then we should do the same here as well.

We want to share more code with ingress-gce and hence better to have behavior parity.

cc @nicksardo

[nicksardo]

ingress-gce knows the correct network via gce.conf on the master. If you know the cluster name/location, you call the GKE GET cluster endpoint and see the configured network

[nikhiljindal]

Thanks @nicksardo That works for GKE clusters. Is there a way to get that for GCE clusters as well? Anything in VM metadata?

We have also discussed delegating firewall rules management to the in cluster ingress-gce controller, which will alleviate this problem: https://github.com/GoogleCloudPlatform/k8s-multicluster-ingress/issues/122

[nicksardo]

It exists in VM metadata: https://github.com/kubernetes/kubernetes/blob/master/pkg/cloudprovider/providers/gce/gce.go#L763, but would you SSH into a machine to get that info? Wouldn't it be easier to call GCE API and describe the instance?

[nikhiljindal]

Right. I meant the metadata returned in GCE API's GET call for instance. (Maybe metadata is not the right term for that?)

Yes I see that the network and subnetwork names are listed there. So we can GET an instance and extract the network name from it, assuming that all nodes are on the same network. That will work for both GCE and GKE clusters.

Here is similar code to extract network tags from an instance: https://github.com/GoogleCloudPlatform/k8s-multicluster-ingress/blob/50b7f6c590c3c08b4dab88cc2afc4d6d3fdcb4c1/app/kubemci/pkg/gcp/networktags/networktags.go#L59.

The code to extract network name will look similar. This is where we compute the desired firewall rule: https://github.com/GoogleCloudPlatform/k8s-multicluster-ingress/blob/50b7f6c590c3c08b4dab88cc2afc4d6d3fdcb4c1/app/kubemci/pkg/gcp/firewallrule/firewallrulesyncer.go#L139.

I see a TODO there to set network for non-default network names :)

[nicksardo]

Right, looks good to me.

[G-Harmon]

Nikhil, you posted the same line of code twice. I think you meant https://github.com/GoogleCloudPlatform/k8s-multicluster-ingress/blob/50b7f6c590c3c08b4dab88cc2afc4d6d3fdcb4c1/app/kubemci/pkg/gcp/firewallrule/firewallrulesyncer.go for the 2nd link?

[nikhiljindal]

Oops :) Thanks for pointing out. Updated.

[nikhiljindal]

Sent https://github.com/GoogleCloudPlatform/k8s-multicluster-ingress/pull/137 to fix this.

[nikhiljindal]

As discussed in https://github.com/GoogleCloudPlatform/k8s-multicluster-ingress/pull/137#issuecomment-367441646, https://github.com/GoogleCloudPlatform/k8s-multicluster-ingress/pull/137 fixed the issue when all clusters are in the same network.

We still need to fix the multiple networks case. Will file a separate issue if a user asks for it.