Azure: public IP sku of nodes does not match sku of LB

[bentastic27]

We enabled the selection of the Load Balancer sku in this GitHub issue:

https://github.com/rancher/terraform-provider-rancher2/issues/580

But when a Standard LB is created, the public IP SKU of the nodes do not match, they default to basic which breaks the public IP/LB relationship. So when no_public_ip is set to false (which is default) the SKU should be calculated from the LB SKU.

Error creating machine: Error in driver during machine creation: network.PublicIPAddressesClient#CreateOrUpdate: Failure sending request: StatusCode=400 -- Original Error: Code="SkuCannotBeChangedOnUpdate" Message="Sku property is set at creation time and cannot be changed from Standard to Basic on resource update for resource /subscriptions/subscription-id-here/resourceGroups/resource-group-name/providers/Microsoft.Network/publicIPAddresses/node-name-here." Details=[]; Timeout waiting for ssh key

gz#15404

[rawmind0]

Not fully understanding what that means as i'm not a big azure expert. Could you please explain, what would be the expected behaviour here??

[junkiebev]

Not fully understanding what that means as i'm not a big azure expert. Could you please explain, what would be the expected behaviour here??

• It is not possible to mix IP SKUs in an availability set.
• Azure load balancers target availability sets.
• Standard tier load balancers can only target Standard SKU IPs
• Basic tier load balancers can only target Basic SKU IPs

The issue is that the terraform provider provisions the compute w/ basic tier IPs, instead of using the SKUs of the load balancer which was selected.

RePro steps 1) Spin up cluster with standard tier LB choice 2) Create internal load balancer service

This results in the networking for all cluster compute breaking.

Expected behavior: The SKU chosen for the loadbalancer must be identical to the SKU chosen for the compute's IP which is provisioned by the provider

[rawmind0]

Thanks for the clarification @junkiebev . We are investigating this, but it seems that selecting ip SKU option is not available at rancher-machine nor docker-machine , so not supported by Rancher nor this provider.

[junkiebev]

Thanks for the clarification @junkiebev . We are investigating this, but it seems that selecting ip SKU option is not available at rancher-machine nor docker-machine , so not supported by Rancher nor this provider.

@rawmind0 - It's just giving it a public IP with that SKU. You do it in AKS already and it (allegedly) works in the GUI.

[rawmind0]

@rawmind0 - It's just giving it a public IP with that SKU. You do it in AKS already and it (allegedly) works in the GUI.

If it's just giving a public IP to the nodes, setting no_public_ip to false should be enough, isn't it?? There are 2 azure configs: node_template azure driver (rancher-machine) and rke azure cloud_provider, (k8s) and it seems that one is affecting the other one, but don't know how to config them to use specific sku kind. We are investigating if this is working from Rancher ui.

AKS is a different story, it's using other drivers than azure.

[junkiebev]

It isn't sufficient - Public IP addresses have skus (though I believe the default is Standard and the ARM you are feeding specifies Basic):

[junkiebev]

Docs

https://docs.microsoft.com/en-us/azure/load-balancer/skus

Microsoft recommends Standard load balancer. Standalone VMs, availability sets, and virtual machine scale sets can be connected to only one SKU, never both. Load balancer and the public IP address SKU must match when you use them with public IP addresses. Load balancer and public IP SKUs aren't mutable.

[bentastic27]

@junkiebev It appears that this may be broken in Rancher and not just in Terraform. To confirm, are you able to run into the same behavior when creating the cluster via the Rancher UI? If so, I'll need to create another GitHub issue on rancher/rancher for this. I don't have access to functioning Azure credentials at the moment for me to try on my end. I sent a booking link in the ticket if you'd like to do this over a screenshare.

In either case, let us know either here or in the ticket.

[rawmind0]

It isn't sufficient - Public IP addresses have skus

@junkiebev Indeed, that's what i'm trying to explain. node_template is using rancher-machine to deploy vms from Rancher. rancher-machine (nor docker-machine) doesn't seem to provide support to configure ip sku's on azure. If so, the support should be added at rancher-machine, then at rancher and finally here.

[HarrisonWAffel]

This new field has been implemented in this PR: https://github.com/rancher/terraform-provider-rancher2/pull/1023

The associated rancher/machine work was done in this PR: https://github.com/rancher/machine/pull/198

Because these terraform changes have already been validated in a separate issue, I'll be closing this issue out as effectively a duplicate.