dsccommunity / FailoverClusterDsc

This module contains DSC resources for deployment and configuration of Windows Server Failover Cluster.
MIT License
60 stars 54 forks source link

xCluster: Adding IgnoreNetwork Parameter #145

Closed TraGicCode closed 6 years ago

TraGicCode commented 7 years ago

Pull Request (PR) description Creating PR early so people can see the progress and offer help/advice @johlju

This Pull Request (PR) fixes the following issues: Fixes #143

Task list:


This change is Reviewable

codecov-io commented 7 years ago

Codecov Report

Merging #145 into dev will not change coverage. The diff coverage is n/a.

Impacted file tree graph

@@        Coverage Diff         @@
##           dev   #145   +/-   ##
==================================
  Coverage   99%    99%           
==================================
  Files        7      7           
  Lines      571    571           
==================================
  Hits       570    570           
  Misses       1      1
johlju commented 7 years ago

Keep in mind that when opening a PR this early, each push to your fork will "spam" all those watching this repo. So either keep pushes to a minimum, or close the PR temporary and reopen the PR each time you need help or comments. As long as you don't force push it will reopen and get all commits from your working branch. I'm uncertain, but think a force push will render the PR unable to reopen if done when closed.

TraGicCode commented 7 years ago

Okay Sounds Good thanks :)

TraGicCode commented 7 years ago

@johlju This feature is complete. Can you verify a couple of things on your end.

1.) This parameter is only used at cluster creation therefore only attached to the Set-TargetResource method. After pushing i see the test on appveyor are failing because Test-TargetResource differs from parameters of Set-TargetResource. What is the proper solution? Add this parameter to Get and Set and Test Methods and don't use the parameter inside Get and Set?

2.) Make sure the unit tests make sense and identify if i'm missing anything.

johlju commented 7 years ago

Please see the BestPractices.md in DscResources repo, especially the sections Get-TargetResource should not contain unused non-mandatory parameters and Use Identical Parameters for Set-TargetResource and Test-TargetResource (unfortunately this section has not ben completed).

I will add a more complete contributions section in the README.md as I did in xSQLServer README.md Contribution section

johlju commented 7 years ago

As of number 2, I will review this one, then I will check tests and code. :)

johlju commented 7 years ago

Test looks good so far, just a few comments there. But you will need to add more tests when you add code for the other review comments.

In addition to the review comments, please resolve this:


Reviewed 1 of 2 files at r2, 2 of 2 files at r3. Review status: all files reviewed at latest revision, 14 unresolved discussions.


DSCResources/MSFT_xCluster/MSFT_xCluster.psm1, line 17 at r3 (raw file):

cluster. Typically

Seem you got two spaces here after '.'.

Throughout where this text is used, and in schema.mof as well.


DSCResources/MSFT_xCluster/MSFT_xCluster.psm1, line 37 at r3 (raw file):

[Parameter(Mandatory = $false)]

Please change to [Parameter()].


DSCResources/MSFT_xCluster/MSFT_xCluster.psm1, line 39 at r3 (raw file):

        [Parameter(Mandatory = $false)]
        [System.String[]]
        $IgnoreNetwork,

Non-mandatory parameters should only in rare occasions be needed in the Get-TargetResource. Maybe this is one if it is not possible to get this information from the cluster to return in the hash table (see next comment).


DSCResources/MSFT_xCluster/MSFT_xCluster.psm1, line 78 at r3 (raw file):

    }

    @{

The new IgnoreNetwork should be returned here with the correct value. This is used by Get-DscConfiguration to view current status. Get-TargetResource could also be called by Test-/Set-TargetResource where this information can be used. In the code above you should get the values of IgnoreNetwork from the cluster, if possible. If not possible, then you need to add the parameter to the Get-TargetResource function so you can return the values to Get-DscConfiguration.


DSCResources/MSFT_xCluster/MSFT_xCluster.psm1, line 126 at r3 (raw file):

[Parameter(Mandatory = $false)]

Please change to [Parameter()].


DSCResources/MSFT_xCluster/MSFT_xCluster.psm1, line 172 at r3 (raw file):

if ($IgnoreNetwork.Count -ne 0)

Maybe instead use if ($PSBoundParameters.ContainsKey('IgnoreNetwork'))? Would that work?


DSCResources/MSFT_xCluster/MSFT_xCluster.psm1, line 179 at r3 (raw file):

New-Cluster @newClusterParameters -NoStorage -Force -ErrorAction Stop

Let's splat the rest of the parameters as well.

            $newClusterParameters = @{
              Name          = $Name
              Node          = $env:COMPUTERNAME
              StaticAddress = $StaticIPAddress
              NoStorage     = $true
              Force         = $true
              ErrorAction   = 'Stop'
            }

DSCResources/MSFT_xCluster/MSFT_xCluster.psm1, line 269 at r3 (raw file):

[Parameter(Mandatory = $false)]

Please change to [Parameter()].


DSCResources/MSFT_xCluster/MSFT_xCluster.psm1, line 271 at r3 (raw file):

        [Parameter(Mandatory = $false)]
        [System.String[]]
        $IgnoreNetwork,

In Test-TargetResource you need to evaluate if the networks listed is actually ignored, and return $false if they are not, and $true if they are.

This is used if the desired state changes. For example a ignored network is activated/enabled then this resource should return the state to desired state. Returning $false the LCM will run Set-TargetResource, so Set-TargetResource should make sure the network is ignored again (inactivate/disable).


Tests/Unit/MSFT_xCluster.Tests.ps1, line 262 at r3 (raw file):


                        Context 'When IgnoreNetwork is passed as a single value' {
                            it 'Should call New-Cluster cmdlet with IgnoreNetwork parameter' {

Please change to It (upper 'I').


Tests/Unit/MSFT_xCluster.Tests.ps1, line 263 at r3 (raw file):

$withIgnoreNetworkParameter = $mockDefaultParameters + @{ IgnoreNetwork = '10.0.2.0/24' }

I had problems with not cloning as hash tables are referenced (very different from what PowerShell usually does). So I used this safer method below. Maybe the method you use is safe, never tried it. Is it as safe as the below code, it's cloning the hash table when adding?

$mockTestParameters = $mockDefaultParameters.Clone()
$mockTestParameters['IgnoreNetwork'] = '10.0.2.0/24'

If you change this, then change throughout.


Tests/Unit/MSFT_xCluster.Tests.ps1, line 275 at r3 (raw file):


                        Context 'When IgnoreNetwork is passed as an array' {
                            it 'Should call New-Cluster cmdlet with IgnoreNetwork parameter' {

Please change to It (upper 'I').


Tests/Unit/MSFT_xCluster.Tests.ps1, line 280 at r3 (raw file):


                                Assert-MockCalled -CommandName New-Cluster -Exactly -Times 1 -Scope It -ParameterFilter {
                                    $IgnoreNetwork -eq ('10.0.2.0/24') -and

Do you need () around the value here?


Tests/Unit/MSFT_xCluster.Tests.ps1, line 289 at r3 (raw file):


                        Context 'When IgnoreNetwork is not passed' {
                            it 'Should call New-Cluster cmdlet without IgnoreNetwork parameter' {

Please change to It (upper 'I').


Comments from Reviewable

TraGicCode commented 7 years ago

Can you shed some quick insight on what i need to do when i resolve the issues from reviewable.io so you can see the outstanding problems? @johlju

TraGicCode commented 7 years ago

Here are the outstanding issues currently. Maybe you are able to help assist me.

Tests/Unit/MSFT_xCluster.Tests.ps1, line 263 at r3 (raw file):

Not sure what you are talking about on this. The test works so i assume what i did is good.

DSCResources/MSFT_xCluster/MSFT_xCluster.psm1, line 271 at r3 (raw file):

DSCResources/MSFT_xCluster/MSFT_xCluster.psm1, line 78 at r3 (raw file):

Need more help on the above 2. I'm very new to clustering and from my understanding it sounds like this is only used at creation time and not persisted in any state anywhere. Can you help me verify this?

I have also allowed editing of the maintainers if you need to make changes to my code.

johlju commented 7 years ago

@TraGicCode Just type 'Done' (or click the little 'Done'-button) on each comment you have fixed. If you need to discuss the comment further, just continue the discussion in the comment in Reviewable. Each comment is saved as draft so you can send all comments back to GitHub as a single comment when you press the green "Publish" button at the top of the Reviewable page.

johlju commented 7 years ago

Tests/Unit/MSFT_xCluster.Tests.ps1, line 263 at r3 (raw file):

Previously, johlju (Johan Ljunggren) wrote…
> $withIgnoreNetworkParameter = $mockDefaultParameters + @{ IgnoreNetwork = '10.0.2.0/24' } I had problems with not cloning as hash tables are referenced (very different from what PowerShell usually does). So I used this safer method below. Maybe the method you use is safe, never tried it. Is it as safe as the below code, it's cloning the hash table when adding? ``` $mockTestParameters = $mockDefaultParameters.Clone() $mockTestParameters['IgnoreNetwork'] = '10.0.2.0/24' ``` If you change this, then change throughout.

I will try to explain this better. The reason for writing the above comment is because I could not verify it on my own last night.

The below code will result in $b and $a having the same two properties, not just $b as one would think.

$a = @{
    Name = 'John'
}

$b = $a
$b.Add('LastName','Smith')

But your code will actually result in a entirely new hash table. So in this case, $a will have one property and $b will have two properties. I did not know this! :smile:

$a = @{
    Name = 'John'
}

$b = $a + @{ LastName = 'Smith' }

Before I have solved this by using the following. This will also result in $a having one property and $b having two properties.

$a = @{
    Name = 'John'
}

$b = $a.Clone
$b['LastName'] = 'Smith'

Comments from Reviewable

johlju commented 7 years ago

Review status: 0 of 6 files reviewed at latest revision, 14 unresolved discussions.


DSCResources/MSFT_xCluster/MSFT_xCluster.psm1, line 78 at r3 (raw file):

Previously, johlju (Johan Ljunggren) wrote…
The new IgnoreNetwork should be returned here with the correct value. This is used by Get-DscConfiguration to view current status. Get-TargetResource could also be called by Test-/Set-TargetResource where this information can be used. In the code above you should get the values of IgnoreNetwork from the cluster, if possible. If not possible, then you need to add the parameter to the Get-TargetResource function so you can return the values to Get-DscConfiguration.

Will try to explain better.

When running Get-DscConfiguration on the target node, it calls the Get-TargetResource in each configuration. Get-TargetResource will run an evaluate the cluster and return the correct values for each property, and Get-DscConfiguration will return the hash table to the user. But since the property IgnoreNetwork is not part of the returned hash table, then the user will not be able to determine which networks are ignored (or none are ignore = value is $null).

You should add IgnoreNetwork to this hash table, and the value should be the result of you (preferably) evaluating the cluster and returning the networks that are ignored, or if it is not possible to evaluate this value from the cluster, then just pass back the value that was set in the configuration (the value in the IgnoreNetwork parameter).


DSCResources/MSFT_xCluster/MSFT_xCluster.psm1, line 271 at r3 (raw file):

Previously, johlju (Johan Ljunggren) wrote…
In Test-TargetResource you need to evaluate if the networks listed is actually ignored, and return `$false` if they are not, and `$true` if they are. This is used if the desired state changes. For example a ignored network is activated/enabled then this resource should return the state to desired state. Returning $false the LCM will run Set-TargetResource, so Set-TargetResource should make sure the network is ignored again (inactivate/disable).

You need to add test code here in the Test-TargetResource function that make sure the networks in the IgnoreNetwork parameter is actually being ignored in each consecutive run.

Let say, for the sake of less writting, that we have two networks A and B. The configuration is set to IgnoreNetwork = B

  1. Test-TargetResource is run and determine there is no cluster.
  2. Set-TargetResource is run and creates the cluster and created to ignore network B.
  3. Test-TargetResource is run on the next consecutive run (default every 15 minutes for Local Configuration Manager) and should evaluate if network B is ignored (this code is missing).
  4. A user adds network B, from outside of DSC - GUI or PowerShell, to the list of networks that should be used.
  5. Test-TargetResource is run on the next consecutive run and see that network B is actually enabled and not being ignored, Test-TargetResource should return $false at this time, because state is no longer in desired state. Desired state is that network B should be ignored - not to be used (this code is missing).
  6. LCM runs Set-TargetResource to get the state in desired state by disable/removing network B again (this code is missing).

Tests/Unit/MSFT_xCluster.Tests.ps1, line 263 at r3 (raw file):

$withIgnoreNetworkParameter = $mockDefaultParameters + @{ IgnoreNetwork = '10.0.2.0/24' }

Since this was correct, please just change the style to this

$withIgnoreNetworkParameter = $mockDefaultParameters + @{
    IgnoreNetwork = '10.0.2.0/24'
}

or this

$ignoreNetworkParameter = @{
    IgnoreNetwork = '10.0.2.0/24'
}

$mockTestParameters = $mockDefaultParameters + $ignoreNetworkParameter

Comments from Reviewable

TraGicCode commented 7 years ago

Review status: 0 of 6 files reviewed at latest revision, 14 unresolved discussions.


DSCResources/MSFT_xCluster/MSFT_xCluster.psm1, line 17 at r3 (raw file):

Previously, johlju (Johan Ljunggren) wrote…
> cluster. Typically Seem you got two spaces here after '.'. Throughout where this text is used, and in schema.mof as well.

Done


DSCResources/MSFT_xCluster/MSFT_xCluster.psm1, line 37 at r3 (raw file):

Previously, johlju (Johan Ljunggren) wrote…
> [Parameter(Mandatory = $false)] Please change to `[Parameter()]`.

Done.


DSCResources/MSFT_xCluster/MSFT_xCluster.psm1, line 126 at r3 (raw file):

Previously, johlju (Johan Ljunggren) wrote…
> [Parameter(Mandatory = $false)] Please change to `[Parameter()]`.

Done.


DSCResources/MSFT_xCluster/MSFT_xCluster.psm1, line 172 at r3 (raw file):

Previously, johlju (Johan Ljunggren) wrote…
> if ($IgnoreNetwork.Count -ne 0) Maybe instead use `if ($PSBoundParameters.ContainsKey('IgnoreNetwork'))`? Would that work?

Done.


DSCResources/MSFT_xCluster/MSFT_xCluster.psm1, line 179 at r3 (raw file):

Previously, johlju (Johan Ljunggren) wrote…
> New-Cluster @newClusterParameters -NoStorage -Force -ErrorAction Stop Let's splat the rest of the parameters as well. ``` $newClusterParameters = @{ Name = $Name Node = $env:COMPUTERNAME StaticAddress = $StaticIPAddress NoStorage = $true Force = $true ErrorAction = 'Stop' } ```

Done.


DSCResources/MSFT_xCluster/MSFT_xCluster.psm1, line 269 at r3 (raw file):

Previously, johlju (Johan Ljunggren) wrote…
> [Parameter(Mandatory = $false)] Please change to `[Parameter()]`.

Done.


Tests/Unit/MSFT_xCluster.Tests.ps1, line 262 at r3 (raw file):

Previously, johlju (Johan Ljunggren) wrote…
Please change to `It` (upper 'I').

Done.


Tests/Unit/MSFT_xCluster.Tests.ps1, line 275 at r3 (raw file):

Previously, johlju (Johan Ljunggren) wrote…
Please change to `It` (upper 'I').

Done.


Tests/Unit/MSFT_xCluster.Tests.ps1, line 280 at r3 (raw file):

Previously, johlju (Johan Ljunggren) wrote…
Do you need () around the value here?

Done.


Tests/Unit/MSFT_xCluster.Tests.ps1, line 289 at r3 (raw file):

Previously, johlju (Johan Ljunggren) wrote…
Please change to `It` (upper 'I').

Done.


Comments from Reviewable

TraGicCode commented 7 years ago

Thanks @johlju . I have marked the issues i have resolved as fixed on reviewable.io. Hopefully my next PR will be less messy. I now understand about the Get-TargetResource thing needing to either pass the value through or retrive the information from the specified cluster that is having Get called on it. I will see if i can find the ignore network stored anywhere in the cluster. If not i'll pass it through.

TraGicCode commented 7 years ago

Review status: 0 of 6 files reviewed at latest revision, 14 unresolved discussions.


Tests/Unit/MSFT_xCluster.Tests.ps1, line 263 at r3 (raw file):

Previously, johlju (Johan Ljunggren) wrote…
> $withIgnoreNetworkParameter = $mockDefaultParameters + @{ IgnoreNetwork = '10.0.2.0/24' } Since this was correct, please just change the style to this ``` $withIgnoreNetworkParameter = $mockDefaultParameters + @{ IgnoreNetwork = '10.0.2.0/24' } ``` or this ``` $ignoreNetworkParameter = @{ IgnoreNetwork = '10.0.2.0/24' } $mockTestParameters = $mockDefaultParameters + $ignoreNetworkParameter ```

Done.


Comments from Reviewable

TraGicCode commented 7 years ago

Review status: 0 of 6 files reviewed at latest revision, 14 unresolved discussions.


DSCResources/MSFT_xCluster/MSFT_xCluster.psm1, line 78 at r3 (raw file):

Previously, johlju (Johan Ljunggren) wrote…
Will try to explain better. When running `Get-DscConfiguration` on the target node, it calls the Get-TargetResource in each configuration. `Get-TargetResource` will run an evaluate the cluster and return the correct values for each property, and `Get-DscConfiguration` will return the hash table to the user. But since the property `IgnoreNetwork` is not part of the returned hash table, then the user will not be able to determine which networks are ignored (or none are ignore = value is $null). You should add IgnoreNetwork to this hash table, and the value should be the result of you (preferably) evaluating the cluster and returning the networks that are ignored, or if it is not possible to evaluate this value from the cluster, then just pass back the value that was set in the configuration (the value in the IgnoreNetwork parameter).

I creating a cluster and inspected Get-Cluster along with other get-cluster* cmdlets and could not find anywhere in which the ignorenetwork was being persisted. Therefore this seems to be only used during creation time just as the documentation specifies here.

https://technet.microsoft.com/en-us/itpro/powershell/windows/failoverclusters/new-cluster

Therefore, I'm simply passing the passed in $IgnoreNetwork parameter to the return value of Get-TargetResource


Comments from Reviewable

TraGicCode commented 7 years ago

The xsqlserversetup has a similar a scenario with its Setupprocesstimeout parameter. This parameter is not queryable and only used at the time of creation. Therefore I'm only going to apply this parameter to set-targetresource and NOT test-targetresource and NOT get-targetresource. Sound good @johlju ?

johlju commented 7 years ago

The SetupProcessTimeout parameter is not comparable since it used to make sure the setup process don't hang, it is not a value used to change the configuration of SQL Server. That's why it's not have any logic in the Test-TargetResource function. That said, I'm looking into if it's a bug that it is not returned by Get-TargetResource, I think it must.

So I still see it is necessary to implement the logic I mention in the review comments.

johlju commented 7 years ago

Only time this logic could not be added is if it is impossible to add a network that was previously ignored, to the cluster.

TraGicCode commented 7 years ago

If you get a chance can you peek around. I cannot see anywhere after creation where you can see the ignored network or even change this. Going to look again and see what i can find and compare the 2 vm's closely while testing. Sorry for the inconvience! I'm still really new and learning.

johlju commented 7 years ago

Check out this if it can be of any help. Won't have time to check this for a while myself unfortunately

https://technet.microsoft.com/en-us/library/cc725775(v=ws.11).aspx

TraGicCode commented 7 years ago

Thanks Looking now!

TraGicCode commented 7 years ago

I created 1 cluster without the -ignorenetwork and another with the -ignorenetwork and they had EXACTLY the same cluster configuration. Compared using GUI ( following https://technet.microsoft.com/en-us/library/cc725775(v=ws.11).aspx ) along with the following cmdlets

Get-Cluster | fl *
Get-ClusterNetwork | fl *

Considering this blocked until someone is able to help.

johlju commented 7 years ago

@TraGicCode It seems it's not possible to ignore networks that are assigned IP address thru DHCP (not using -IgnoreNetwork at least). They seem to always be added for cluster communication. -IgnoreNetwork can be used to ignore networks set to a static IP address. The reason for not able to ignore these networks, I guess, is because it is unknown how the network look like prior of assignment.

To remove DHCP networks for cluster communication that need to be added to the xClusterNetwork which should handle the networks once the cluster is created.

So I think this PR should focus on networks with static IP adress.

TraGicCode commented 6 years ago

@johlju Sorry for getting back to this so late. You are indeed correct. I was able to ignorenetworks that i specified once i disabled DHCP for the NIC's i wanted ignored.

johlju commented 6 years ago

Can you rebase this PR? If you don't have time, and you want me to help you, then please enable "Allow edits from mainatiners" to the right.


Reviewed 4 of 6 files at r4, 2 of 2 files at r5. Review status: all files reviewed at latest revision, all discussions resolved, some commit checks failed.


Comments from Reviewable

TraGicCode commented 6 years ago

The allows edits from maintainers checkbox is already checked so you should be good to assist.

msftclas commented 6 years ago

CLA assistant check
All CLA requirements met.

johlju commented 6 years ago

Reviewed 5 of 5 files at r6. Review status: all files reviewed at latest revision, 3 unresolved discussions, some commit checks failed.


CHANGELOG.md, line 15 at r6 (raw file):

  - Get-TargetResource now correctly returns the IP address instead of throwing
    and error ([issue #28](https://github.com/PowerShell/xFailOverCluster/issues/28)).
  - Added -IgnoreNetwork parameter

This should have a reference to an issue and end with full stop (.).


README.md, line 67 at r6 (raw file):

(Optional)

Should be (Write).


DSCResources/MSFT_xCluster/MSFT_xCluster.schema.mof, line 9 at r6 (raw file):

    [Write, Description("StaticIPAddress of the Cluster")] String StaticIPAddress;
    [Required, EmbeddedInstance("MSFT_Credential"), Description("Credential to create the cluster")] String DomainAdministratorCredential;
    [Write, Description("One or more networks to ignore when creating the cluster. Typically used to ignore networks with DHCP enabled since they are always included by default.")] String IgnoreNetwork[];

This description should be changed stating that it can only be used on Static IP networks. This should be used as the description in the README.md and comment-based help as well.


Comments from Reviewable

johlju commented 6 years ago

Review status: 1 of 6 files reviewed at latest revision, 3 unresolved discussions.


CHANGELOG.md, line 15 at r6 (raw file):

Previously, johlju (Johan Ljunggren) wrote…
This should have a reference to an issue and end with full stop (.).

Done


README.md, line 67 at r6 (raw file):

Previously, johlju (Johan Ljunggren) wrote…
> (Optional) Should be (Write).

Done


DSCResources/MSFT_xCluster/MSFT_xCluster.schema.mof, line 9 at r6 (raw file):

Previously, johlju (Johan Ljunggren) wrote…
This description should be changed stating that it can only be used on Static IP networks. This should be used as the description in the README.md and comment-based help as well.

Done


Comments from Reviewable

johlju commented 6 years ago

@TraGicCode I was certain you had sign the CLA - maybe the BOT is acting up? The way the bot works was changed recently so maybe this PR was in between changes. Could you please sign the CLA again just to make sure it's done? 😄 If the CLA is not signed I cannot merge this code, and we are so close now :/

johlju commented 6 years ago

@TraGicCode See this comment on how to sign the CLA; https://github.com/PowerShell/xFailOverCluster/pull/145#issuecomment-353939120

TraGicCode commented 6 years ago

@johlju Signed.

johlju commented 6 years ago

@TraGicCode Thank you very much! 😄

johlju commented 6 years ago

Reviewed 5 of 5 files at r7. Review status: all files reviewed at latest revision, 1 unresolved discussion.


DSCResources/MSFT_xCluster/MSFT_xCluster.psm1, line 200 at r7 (raw file):

            New-Cluster @newClusterParameters

            if ( -not (Get-Cluster))

Remove space in front of -not just to be consistent.


Comments from Reviewable

johlju commented 6 years ago

Review status: 5 of 6 files reviewed at latest revision, 1 unresolved discussion.


DSCResources/MSFT_xCluster/MSFT_xCluster.psm1, line 200 at r7 (raw file):

Previously, johlju (Johan Ljunggren) wrote…
Remove space in front of `-not` just to be consistent.

Done


Comments from Reviewable

johlju commented 6 years ago
:lgtm:

Reviewed 1 of 1 files at r8. Review status: all files reviewed at latest revision, all discussions resolved.


Comments from Reviewable

johlju commented 6 years ago

@TraGicCode so this one is over the finish line, just merged. Thanks for your work! Hope I see you again here. 😄