solidfire / PowerShell

Collection of scripts, functions, and examples using the SolidFire Tools for PowerShell
MIT License
27 stars 20 forks source link

When naming a single volume with SFVolumePair cmdlet, it loops through every volume and is time-consuming #79

Closed ddansf closed 4 years ago

ddansf commented 5 years ago

When naming a single volume with SFVolumePair cmdlet, it loops through and appears to enumerate every volume on the cluster as part of this function, which can be very time-consuming on a production cluster with hundreds of volumes.

Observed in:

Powershell Version 1.5.0.119 Cluster version 11.0.0.781

Example:

Set-SFVolumePair -VolumeID 8 -PausedManual:$true -Confirm:$false

One customer recently tried this on a cluster with 800 volumes as part of a cluster migration and the associated delay was extremely long when invoking the cmdlet in a loop and calling specific volume id's.

However, if an array of specific volume id's is passed to the cmdlet directly, performance is much faster.

scaleoutsean commented 5 years ago

What happens if you pass it an array with 1 element (1 volume ID)?

scaleoutsean commented 5 years ago

When naming a single volume with SFVolumePair cmdlet, it loops through and appears to enumerate every volume on the cluster as part of this function

Isn't this enumeration is done by the API?

the associated delay was extremely long

With 50 replication pairs Set-SFVolumePair takes 0.28s to run (list of 1 or loop of 1, same result). With 1 replication pair, the same Set-SFVolumePair takes 0.24s - only 16% faster, which sounds reasonable.

It takes only 0.04s seconds to enumerate 50 volumes. Unless the cmdlet or server-side stops to scale at some point, 800 volumes would still take less than 1 second. What does "extremely long" mean?

ddansf commented 5 years ago

Clarification around the observed delay:

With 800+ volumes [on the cluster], the code "Set-SFVolumePair -VolumeID $s -PausedManual:$true -SFConnection $SourceCluster" would enumerate all 800+ volumes even though I am sending it 1 volume ID at a time.

Accompanying details:

Source Cluster

$SourceCluster = Connect-SFCluster -Target "" -credential $cred -ErrorAction Stop #

Destination Cluster

$DestinationCluster = Connect-SFCluster -Target "" -credential $cred -ErrorAction Stop #

Get all of the VolumePairs VolumeId on the Source Cluster

$SourceVolumePairIds = (Get-SFVolumePair).volumeid foreach ($s in $SourceVolumePairIds) { Set-SFVolumePair -VolumeID $s -PausedManual:$true -SFConnection $SourceCluster }

Get all of the Volumes on the Destination Cluster with Volume Access set to ReplicationTarget and set to readWrite

$DestinationRepliVolumes = (Get-SFVolume -SFConnection $DestinationCluster | ? {$_.access -eq 'ReplicationTarget'}).volumeid foreach ($d in $DestinationRepliVolumes){ Set-SFVolume -volumeid $d -Access readWrite -SFConnection $DestinationCluster -Confirm:$false }

scaleoutsean commented 5 years ago

@ddansf with foreach ($d in $DestinationRepliVolumes){Set-SFVolume ..} they make hundreds of API calls to the API server. What I had suggested is to first build a list from $DestinationRepliVolumes, and then just run Set-SFVolume -volumeid $list.

I don't have 2 clusters to try this (and the other day I only had Demo VM which is limited to 50 volumes), but when I try another (unrelated, because I don't have any pairs) cmdlet on singleton cluster, the first approach (to call it many times) is much slower:

Maybe they'd see an improvement with the second approach if ithey can use it - all volumes get enumerated and matched just once, on the API server, and it's just 1 call.

ntavendale commented 4 years ago

Closing this issue, since it does not appear to be a bug, rather the issue is with the way the PS command is being used..