MicrosoftDocs / azure-docs

Open source documentation of Microsoft Azure
https://docs.microsoft.com/azure
Creative Commons Attribution 4.0 International
10.28k stars 21.47k forks source link

Cluster IP address #8184

Closed MauricioZa closed 6 years ago

MauricioZa commented 6 years ago

Hello guys

In the section: "Set the cluster IP address" > Step 3 it says: "Select Static IP Address and specify an available address from subnet where the SQL Server is in the Address text box. Then, click OK."

I did exactly that. But I have a problem. I took the next address that was available in my subnet and assigned it to the cluster. Then, I finished the rest of the configuration, and everything is up and running. All good !

Now, 2 weeks later, I created 10 Web Server VMs in the same subnet. I noticed one of these Web Servers grabbed the same IP that I assigned to my cluster previously. The cluster is still alive and running (and the web server with the duplicated IP is running OK too).

It makes me nervous that I have the same IP allocated twice. I am scared my Always On cluster may have a problem due to this.

I investigated and there's no way to reserve a local IP within Azure so it is not taken by anything else. How can I reserve the cluster IP in the Azure subnet so it is not taken by anything else? Or, is it just that there's no problem if it is duplicated ?

Thanks !


Document Details

Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

mimckitt commented 6 years ago

@jorgefava Thanks for the question! We are investigating and will update you shortly.

mimckitt commented 6 years ago

@MikeRayMSFT can you advise on this one? I believe it is setup this way as the second cluster would be the failover cluster. Hence, the same IPs would make sense. But I know duplicate IPs in Azure can cause problems.

MauricioZa commented 6 years ago

Sorry for bother, but, we are trying to implement this. Is there any response to my issue ? Thank you !

mimckitt commented 6 years ago

@jorgefava sorry for the delay! I am checking with @MikeRayMSFT offline now.

MikeRayMSFT commented 6 years ago

@jorgefava - I also am sorry for the delay. From my side I'm contacting the appropriate Azure teams to find out how to correct the information in the technical content and will update as soon as possible.

Here are some resources for technical support immediately:

MikeRayMSFT commented 6 years ago

@jorgefava -

To avoid this situation, the cluster address can use APIPA (usually used when no DHCP server is available). Choose an address from the address range 169.254.0.1 to 169.254.255.254. The cluster address is already not connectable in the Azure VM WSFC so, an APIPA address can be used. This is what the Microsoft Azure resource manager ARM templates do for the WSFC hosting the availability group.

We will update the documents.

Thank you for your comments.

KJLive commented 6 years ago

No matter what I tried, (GUI and Powershell) I can't enter the APIPA 169.254.x.x address as the cluster address because it isn't in the same subnet as the NIC's assigned IP - what seems to be missing?

mimckitt commented 6 years ago

Thanks @kjsteven. I am working with @MikeRayMSFT offline to see what the issue is.

Related MSDN thread for reference: https://social.msdn.microsoft.com/Forums/azure/en-US/60d2a3db-3ef7-423b-8291-5aad3aedcc8a/cannot-assign-apipa-failover-cluster-ip-when-creating-sql-2016-alwayson-on-azure-vms?forum=WAVirtualMachinesforWindows

MikeRayMSFT commented 6 years ago

@kjsteven this sounds very frustrating for you and @jorgefava - in separate situations. @MicahMcKittrick-MSFT and I have the solution.

  1. Go to the Azure load balancer (that you are using for the SQL Server AG load balancer) and add a new Frontend IP address. a) Under Settings, click Frontend IP configuration b) Click Add c) Specify the subnet that hosts the SQL Server and a dynamic IP address. Click create. (You really need a static address, but I use the dynamic one to create it so that it can grab a address from the correct subnet. d) After the front end IP address is created, change it to "static". Write down this IP address.
  2. Add a health probe to the SQL Server AG load balancer. a) In the Azure portal, locate the load balancer settings, and click Health probes. b) Click add, give the new health probe a name, use the TCP protocol. Set the port for an usused port. I used 58888. It's just an example. c) Use the default ssettings for interval (5) and unhealthy threshold (2). Click OK to create the health probe. 3.. Create a new load balancing rule. Use the new front end IP address, the port (in my case 58888) for both the Port and the back end port. Use the same backend pool as the AG load balancing rule. Use the health probe that you just created. Very Important: Enable Floating IP (direct server return) sometimes called DSR.

====== Now RDP into one of the the SQL Server AG cluster nodes.

Open the firewall. Create a new inbound rule for the health probe. Allow TCP 58888 the firewall. Do this for both cluster nodes.

===

Configure the IP address on the cluster

In failover cluster manager, click the cluster. Locate Cluster Core Resources

Under Cluster Core Resources -> Server Name, right-click IP Address and click Properties

Copy the value from the Name of the IP address. Probably something like Cluster IP address. You will need this to set the cluster properties.

Under IP Address Set the IP address to the IP address you configured for the New FRont End IP address on the load balancer.

Click OK.

Failover cluster manager will prompt you to restart the cluster resource.

=====

Open Powershell

Run Get-ClusterNetwork. Copy the name of the cluster network. Probably something like Cluster Network 1.

Update the following script for your environment.

$ClusterNetworkName = "<ClusterNetworkName" # the cluster network name (Use Get-ClusterNetwork on Windows Server 2012 of higher to find the name)
$IPResourceName = "<Cluster IP Address Name>" # the IP Address resource name
$ILBIP = "x.x.x.x" # the IP Address of the Internal Load Balancer (ILB). This is the static IP address for the load balancer you configured in the Azure portal.
[int]$ProbePort = <58888> #Replace with your port

Import-Module FailoverClusters

Get-ClusterResource $IPResourceName | Set-ClusterParameter -Multiple @{"Address"="$ILBIP";"ProbePort"=$ProbePort;"SubnetMask"="255.255.255.255";"Network"="$ClusterNetworkName";"EnableDhcp"=0}

Run the script to set the cluster parameter for your environment.

That should solve the problem and prevent any IP address issues with the WSFC name.

Technical Reason: For an IP address to float easily from one host to another in on premises networks it uses a gratuitous ARP (GARP). GARP is not available on cloud provider networks, so the load balancer holds the front end IP address (so it won’t get reassigned) and probes the cluster nodes to see which node owns the resource. Then calls to the IP address are routed to the LB which forwards them to whichever node is responding to the health probe.

I'm working to update the content.

Thank you for submitting this request.

mimckitt commented 6 years ago

Thanks @MikeRayMSFT!

@jorgefava we have updated the doc. Please try to work through it again and let us know if you have issues. We setup an environment and verified the steps work.