joel74 / POSH-LTM-Rest

PowerShell module and scripts for working with F5's LTM REST API
MIT License
78 stars 48 forks source link

Get-PoolsforMember not working as Get-Help states #220

Closed shawns1234 closed 2 years ago

shawns1234 commented 2 years ago

I am trying to write some code to search through all nodes "like" the test supplied, and ultimately return all pools or apps associated with a node. The purpose is to more quickly update a few dozen pools in seconds as opposed to hours manually. I cannot figure out how to get Get-PoolsforMember to work. Below is two different things I've tried.

Get-PoolMember -F5session $F5Session | Where-Object -Property Name -like -Value "5webiwf" | Get-PoolsforMember -f5session $f5session

$f5nodes = Get-PoolMember -F5session $F5Session | Where-Object -Property Name -like -Value "5webiwf" foreach ($f5node in $f5nodes) { Get-PoolsforMember -f5session $f5session }

If I use either of those, I return all pools on the F5. If I do not use -F5Session, I receive: Write-Error: C:\Users\adminss\Documents\PowerShell\Modules\F5-LTM\1.4.315\Public\Get-Pool.ps1:32 Line | 32 | … $JSON = Invoke-F5RestMethod -Method Get -Uri $URI -F5Session $F5S … | ~~~~~~~~~~~~~ | "401 :

joel74 commented 2 years ago

Hi, Instead of: Where-Object -Property Name -like -Value "5webiwf" please try Where-Object {$_.name -match "5webiwf"}

The reason the first option returns all pools is because nothing is matching your Where clause. Once you get a match and pass the object on to Get-PoolsforMember, you should get the results you want. As you found, you have to use an F5session, otherwise you can't connect to your F5.

shawns1234 commented 2 years ago

Thank you for your assistance. I have tried your recommendation, but it results in the same outcome. It produces a repeating list of all pools on the F5 that has to be manually cancelled. Below is the command I am running: Get-PoolMember -F5session $F5Session | Where-Object {$_.name -match "5webiwf"} | Get-PoolsforMember -f5session $f5session I have tried -like, -match, and -contains. like & contains produces no result & no error. match produces the loop of all pools. Adding * on each side of the word produces the loop of all pools. Lastly, for clarity, the (2) nodes are named PDC-5WEBIWF01 & 02. The old nodes also contain 5WEBIWF in their name. My goal would be to "find all pools that have the old nodes", then update the pools with "new nodes". Unfortunately, I cannot simply use the node name itself because the "name" is the node name:portnumber. I don't know these without looking at them which defeats the purpose of automation. As an additional point, I'll need to be able to capture the port number itself for the second part of this exercise, but I'm assuming some powershell magic can grab that from the name itself; storing it as a temporary variable.

joel74 commented 2 years ago

Hi. -match is definitely what you should be using. Since we don't know exactly where the command is breaking, let's simplify.

When you execute the following, does it output the two pool members: Get-PoolMember -F5session $F5Session | Where-Object {$_.name -match "5webiwf"}

Also, if you're calling New-F5Session at the start, you don't need to pass in the session variable to each call. It's stored in the code context and used automatically.

shawns1234 commented 2 years ago

2022-08-16 16_28_51-PSCommands Screenshot attached to show output. So, that command by itself seems to produce all instances where the node is attached to something. It is when I pass the last pipe and command that I receive the constant loop. Based on your comment about not needing to pass the session to the 2nd pipe, I can only assume that it is treating it as a fresh get-poolsformember with effectively no member name; resulting in a constant loop of all pools. I receive the following error when executing: Get-PoolMember -F5session $F5Session | Where-Object {$_.name -match "5webiwf"} | Get-PoolsForMember

Write-Error: C:\Users...\Documents\PowerShell\Modules\F5-LTM\1.4.315\Public\Get-Pool.ps1:32 Line | 32 | … $JSON = Invoke-F5RestMethod -Method Get -Uri $URI -F5Session $F5S … | ~~~~~~~~~~~~~ | "401 :

Write-Error: C:\Users...\Documents\PowerShell\Modules\F5-LTM\1.4.315\Public\Get-Pool.ps1:32 Line | 32 | … $JSON = Invoke-F5RestMethod -Method Get -Uri $URI -F5Session $F5S … | ~~~~~~~~~~~~~ | "401 :

Write-Error: C:\Users...\Documents\PowerShell\Modules\F5-LTM\1.4.315\Public\Get-Pool.ps1:32 Line | 32 | … $JSON = Invoke-F5RestMethod -Method Get -Uri $URI -F5Session $F5S … | ~~~~~~~~~~~~~ | "401 :

Write-Error: C:\Users...\Documents\PowerShell\Modules\F5-LTM\1.4.315\Public\Get-Pool.ps1:32 Line | 32 | … $JSON = Invoke-F5RestMethod -Method Get -Uri $URI -F5Session $F5S … | ~~~~~~~~~~~~~ | "401 :

Thank you for trying to help me sort this out. It seems simple, but I'm missing something somewhere.

joel74 commented 2 years ago

Hi. Good, that's what we wanted to ensure that device on its own is doing. How many instances is that command finding?

You said that when you pass all those results to the Get-PoolsForMember, it goes into a constant loop. I'm wondering if it's just that there are so many pool members that it's searching all pools for, that it ends up being a very long execution time.

If you change the pattern in Get-PoolMember so we only get 1 or 2 pool members back and then pipe that to Get-PoolsForMember, does that complete with the expected results? (And if you need to add the F5session param back to Get-PoolsForMember so that it works, please do. I'm not clear on what the error at line 401 is.)

shawns1234 commented 2 years ago

So, the loop is cycling through all pools on the F5. It's 156 pools. No errors, just displays 156 pools, and then starts back over at the beginning until you cancel execution. These two nodes are only attached to maybe 6 pools. 2022-08-16 16_28_51-PSCommands_2

Passing the full name with the port resulted in the same behavior in every scenario I've been trying. There should only be 1 pool displayed using the fullname:port. In regards to the 401, I assume it is the API behind the scenes executing the rest command against the F5. It's like the URI & F5Session are not populating the command correctly. Therefore, one might expect to get a 401 (not found) when executing a GET for example. All speculation... I'm awful at programming. :-)

shawns1234 commented 2 years ago

Wait.. I'm confusing 401 with 404... at any rate, I think you know what I'm trying to say.

Edit: Looks like 401 is an authorization error. That would maybe explain why I don't get this when passing the $F5Session at the end. It seems like that third command is not "inheriting" the session authorization.

joel74 commented 2 years ago

Hi. I think part of the confusion here has to do with pool members vs. nodes. In the screenshot with the list you uploaded, I counted 23 pool members. For every pool member that first command returns, it is going to search all pools to see if those pool members are in them. So the statement that there are only 2 nodes used in 6 pools doesn't seem to jive with the pool member results you're getting.

At some point, it would make more sense and be a better use of time to print out all members for all pools and decide based on that data which ones you want to change.

shawns1234 commented 2 years ago

You're right. I made that statement from memory. There would be half the 23 number. 1 pool likely not having both members. There are actually only 6-7 being used which is why I said that mistakenly. I'm not sure what to do at this point. I'm not sure what you mean by print them all out, but it seems like that would require a manual "look" at the lines to make some decision. That defeats the purpose of the entire exercise. According to the help info, Get-Poolsformember should work in the way I have used it.
One last thing I find confusing is that we are passing some variation of "name" to this command. There is no option for -Name. The only options that appear to be relevant are -address & -Inputobject.

joel74 commented 2 years ago

Hi, let's try this again: When you execute this command, how many lines of pool members does that list out? Get-PoolMember -F5session $F5Session | Where-Object {$_.name -match "5webiwf"}

shawns1234 commented 2 years ago

Line 31 of Get-Pool.ps1 $URI = $F5Session.BaseURL + 'pool/{0}' -f (Get-ItemPath -Name $itemname -Application $Application -Partition $Partition)

I see Get-ItemPath in a private folder, but I'm unsure if it is getting loaded or used properly. When attempting to use Get-ItemPath by itself, I receive an error that the module isn't found. I manually imported a few of the modules in the private folder to get past some of this. I'm not sure what is what anymore because I'm braindead looking at this code.
What I "think" I found is that, at least in my case, I needed to add the name of the iApp before the pool name itself. Not sure if the Get-ItemPath module should be doing this or not, but I can't seem to perform a basic Get-Pool -name without specifying the app as well. Example below: get-pool -F5Session $F5Session -Name 'sub.domain.com.app~sub.domain.com._pool' This command returns the appropriate pool information. Hopefully something I said can help you determine what is broken either in the code or my head. :-)

Edit-- I was typing that as you sent the last message. I'll try that now.

shawns1234 commented 2 years ago

Get-PoolMember -F5session $F5Session | Where-Object {$_.name -match "5webiwf"}

23 line items. The same (2) nodes, but listed on different ports depending on the pool they are associated with. Node 1 is a member of 11 pools Node 2 is a member of 12 pools. ( Not sure what is up with that, but that is appdev's problem :-) )

joel74 commented 2 years ago

Ok. So the Get-PoolsForMember command takes those 23 pool member entries and looks at every single pool on the F5 to see if any of those 23 are members of that pool. I think the process is just taking a long time.

You also had a question / were confused about passing -name to Get-PoolsForMember. We're not passing -name. It's being used in the preceding Where filter to limit the number of pool member objects, and then the resulting array of objects is passed to Get-PoolsForMember.

I'm not sure that this helps or clears anything up, but I'm also not yet convinced that the code is working other than expected.

Lastly, you keep mentioning 'nodes' but this code is focused on 'pool members' not 'nodes.' They are two different types of entities.

shawns1234 commented 2 years ago

I apologize for using the term node. Nodes are what I'm ultimately replacing. The node name is contained in the pool member name, so.. I'm treating them conceptually as the same thing. I understand what you are saying. I'll try to stop using the term. I'll go back to one of my original points.. Running the command above with | Get-Poolsformember returns the 401 error per my screenshot. My theory is the full path isn't being created properly because it relies on the path containing information about the iApp as well. If I add the session information with the | Get-Poolsformember, it enumerates all pools. It does this right away. There is no "waiting" other than the time it takes to get the information and write it to the screen. A full iteration of roughly 150 pools takes about 10 seconds before starting back at the beginning of the list of pools. When I get back to my computer, I'm going to try to alter the get-pool module to write out all the variables so I can see exactly what is trying to be sent via rest commands.

On Wed, Aug 17, 2022, 10:11 AM joel @.***> wrote:

Ok. So the Get-PoolsForMember command takes those 23 pool member entries and looks at every single pool on the F5 to see if any of those 23 are members of that pool. I think the process is just taking a long time.

You also had a question / were confused about passing -name to Get-PoolsForMember. We're not passing -name. It's being used in the preceding Where filter to limit the number of pool member objects, and then the resulting array of objects is passed to Get-PoolsForMember.

I'm not sure that this helps or clears anything up, but I'm also not yet convinced that the code is working other than expected.

Lastly, you keep mentioning 'nodes' but this code is focused on 'pool members' not 'nodes.' They are two different types of entities.

— Reply to this email directly, view it on GitHub https://github.com/joel74/POSH-LTM-Rest/issues/220#issuecomment-1218143375, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAOO73PDI6B3YAXBAQ7MITLVZT6L7ANCNFSM56FLXTXQ . You are receiving this because you authored the thread.Message ID: @.***>

joel74 commented 2 years ago

Thanks. I thought the 401 error only started when we removed the F5Session variable from the second command. If we add it back, then we should be back to where you are seeing all the pool names being listed out. I.e.

Get-PoolMember -F5session $F5Session | Where-Object {$_.name -match "5webiwf"} | Get-PoolsforMember -f5session $f5session

So you're saying that there are pools in the list that do not contain one of the 23 pool members?

shawns1234 commented 2 years ago

You are correct regarding "when" we receive the 401 error. In regards to the second question, the repeating list of pools DOES include the 11 (or 12) pools that only contain the pool members. For clarity, there are (23) interesting pool members comprised of the same two server objects with varying TCP port information. (1) pool only contains "pool member 2". The issue is that it includes ALL pools on the F5 regardless of pool member name. The repeating list is (156) pool objects instead of the (12) I would expect to see. To further our dive into this, I decided to let that command run for a bit to see what would happen. I outputted to a text file so I could count how many times the loop repeated. I'm sure you're already guessing how many times it looped. (23) times. Printed out into a text file, it was 3591 lines of text. 23 x 156 pools + 3 lines of header info. Each loop contains exactly the same data. So, it knows to loop through 23 different pool members, but instead of writing out the ones only related to the pool member itself, it writes out all pools.

shawns1234 commented 2 years ago

2022-08-16 16_28_51-PSCommands_3

FWIW, this is the output when running the above command with -Verbose at the end.

joel74 commented 2 years ago

Ok. I may have a lead. In looking at the pool member output, I noticed that the pool members listed don't have specific IP addresses - they have a value of "any6" for Address.

Therefore, when Get-PoolsForMember parses the pool member objects passed to it, it doesn't see this as a valid IP address and basically calls Get-PoolMember with no parameters at all. As a result, all pool members get returned every time.

shawns1234 commented 2 years ago

Ok. I may have a lead. In looking at the pool member output, I noticed that the pool members listed don't have specific IP addresses - they have a value of "any6" for Address.

Therefore, when Get-PoolsForMember parses the pool member objects passed to it, it doesn't see this as a valid IP address and basically calls Get-PoolMember with no parameters at all. As a result, all pool members get returned every time.

That makes perfect sense. I was wondering why the command used -address when, at least for me, they are all the same. I think you are onto something! Thank you for sticking with me on this.

joel74 commented 2 years ago

Hi. I'm glad we were finally able to make some progress on this. I'll make an update to Get-Help that clarifies we expect pool members to have IP addresses assigned in order to successfully use Get-PoolsforMember. It seems like, in this scenario since IPs aren't assigned, you may just need to list out all the pools and pool members and visually determine which are assigned where, unless you came up with a different way to get the info.

shawns1234 commented 2 years ago

I appreciate all the effort. Unfortunately, doing as you say provides no value. As I stated in the beginning, the whole reason for diving into this API was to eliminate manual intervention. Using these methods to simply "look" at a list printed on the scene is actually more effort than logging into the F5, selecting a node, then displaying associated pools. I was hoping to provide value to my development team during migrations and roll outs. Since you didn't mention adding additional capability to this specific module, I can only assume that the underlying F5 Rest API just isn't capable of accepting a pool member name as an identifier.

On Fri, Aug 19, 2022, 9:38 AM joel @.***> wrote:

Hi. I'm glad we were finally able to make some progress on this. I'll make an update to Get-Help that clarifies we expect pool members to have IP addresses assigned in order to successfully use Get-PoolsforMember. It seems like, in this scenario since IPs aren't assigned, you may just need to list out all the pools and pool members and visually determine which are assigned where, unless you came up with a different way to get the info.

— Reply to this email directly, view it on GitHub https://github.com/joel74/POSH-LTM-Rest/issues/220#issuecomment-1220756877, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAOO73LUVXP6QDZJVWFNYWDVZ6L5JANCNFSM56FLXTXQ . You are receiving this because you authored the thread.Message ID: @.***>

joel74 commented 2 years ago

Fair enough. The F5 API does support using a name value in place of an address value as an identifier. It's been a number of years since this function was created, so I can't exactly recall the reasoning for why IP address was the chosen identifier when a pool member object is passed in. Pool member name seems like a better choice overall since I believe it should be unique.

I've updated and tested locally a version of Get-PoolsForMember that uses the Name value when pool member objects are passed in. I'd want to test it further before committing it to the main code branch but I could share the file with you to update your local copy and test with.

shawns1234 commented 2 years ago

That's wonderful news. I really appreciate it. I'd be happy to test it out!

joel74 commented 2 years ago

Cool. Here you go: https://gist.github.com/joel74/b453ceffa240d8b0d752c113faa40c9d

shawns1234 commented 2 years ago

2022-08-16 16_28_51-PSCommands_4

Took me a minute to realize I needed to unload the previous module, etc.... Scratched my head for a bit. But, I'm a happy camper now! It seems to work perfectly. Since it does take a minute to run through all the pools, I decided to add a progress bar with it as you can see in the screenshot. Seriously, thank you for sticking with this.

joel74 commented 2 years ago

You're very welcome. I'm glad we finally got there. I created a branch with the update for Get-PoolsForMember and will most likely end up just merging that into the main code line. Thanks for using the module and for helping me improve it.