search

brocade / ansible

55 stars 33 forks source link

Feature request: Would like to gather nameserver information #80

Open munokar opened 3 years ago

munokar commented 3 years ago

I would like to gather the information provided by RestAPI leave brocade-name-server as facts. I need to be able to have a PID (SID / DID) translation to WWPN / alias. Since we have plenty of NPIV device using the fcid_hex from brocade_interface_fibrechannel is not sufficient.

daniel-chung-broadcom commented 3 years ago

Do you mind clarifying a bit? Are you looking to provide PID of the device in the playbook and have the play return Port WWN of the associated device & zoning alias name?

btw, I have find_ns_by_pid.yml that takes in pid and display the ns entry found. Is this kind of along the line of what you are looking for but you want port_name to be isolated and find zoning alias based on it?

dc400196@iaas-l-2206-611:~/ansible$ ansible-playbook -i san-inventory find_ns_by_pid.yml

PLAY [san_eng_zone_seed_san_a] *****

TASK [gather facts] **** ok: [san_eng_zone_seed_san_a]

TASK [print ansible_facts gathered] **** ok: [san_eng_zone_seed_san_a] => { "ansible_facts": { "discovered_interpreter_python": "/usr/bin/python3", "fibrechannel_name_server": [ { "cascaded_ag": "no", "class_of_service": "class-2 class-3", "connected_through_ag": "no", "fabric_port_name": "20:04:c4:f5:7c:00:a3:f0", "fc4_features": null, "fc4_type": null, "fcoe_device": "no", "frame_redirection": "no", "link_speed": "32G", "lsan": "no", "name_server_device_type": "Physical Unknown(initiator/target)", "node_name": "20:04:c4:f5:7c:00:a3:f0", "node_symbolic_name": null, "partial": "no", "permanent_port_name": "20:04:c4:f5:7c:00:a3:f0", "port_id": "0x020400", "port_index": "4", "port_name": "20:04:c4:f5:7c:00:a3:f0", "port_properties": "SIM Port", "port_symbolic_name": null, "port_type": "n-port", "real_device_behind_ag": "no", "slow_drain_device_quarantine": "no", "state_change_registration": "None" } ] } }

PLAY RECAP ***** san_eng_zone_seed_san_a : ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0

munokar commented 3 years ago

Thanks for pointing to the find_ns_by_pid.yml - this is what I needed to get the information. We have MAPS configured to send ASIC flow information for F-ports. The events generated contains source- and destination-IDs which need to be translated - thats way I need a full list of the nameserver which I can receive slightly modifying your example.

daniel-chung-broadcom commented 3 years ago

I saw this in the email copy of your earlier posting "Next topic would be creating / modifying MAPS rule, policies and configs using ansible. I have not seen any examples for that or any python code. Is that something that is in focus?" But I don't see it in the actual thread. Not sure what is going on.

Anyhow, assuming that is still something that you need some help with, we have brocade_maps_maps_config.py module that you can use. For others, we don't have "named" modules, but we can use brocade_list_obj or brocade_singleton_obj to create playbooks to deal with those areas of rest that we don't have "named" modules.

For example, https://github.com/brocade/ansible/issues/79 thread shows another user making use of brocade_list_obj* module to create a playbook to set maps-policy and reporting a defect. In the zip file attached to the thread, you can see an example. If you have a specific example of what you are trying to accomplish (if you want to attach a cli commands used for example), I could provide some reference examples of how to create playbooks using the generic object handler modules. Below is a link to the readme location for more information.

https://github.com/brocade/ansible#template-based-ansible-modules

munokar commented 3 years ago

I have changed the post since I have found also #79 and want to give it a try using the bocade_list_obj and brocade_singleton_obj. For sure a more comfortable "named" module would be very helpful and simplify the approach. The goal we want to achieve is that MAPS is entirely managed through Ansible. Actions like:

Currently all of this stuff is done through scripts via SSH since MAPS for 150 SAN switches / directors are hard to handle via BNA.

daniel-chung-broadcom commented 3 years ago

I see.

Would you be comfortable sharing the script with me or send me the specific commands as examples? I could check to see if I could provide you with some reference playbook examples and try to identify gaps if any.

munokar commented 3 years ago

I got managed to write a playbook that create a policy and some rules. For the policy I also experienced the issues described in #79 - since I'm using the collection from the ansible galaxy I expect this to be "normal". How would from you implement a playbook that removes all rules that are not predefined and are not in any policy anymore? Because we want to have some information about the rule itself in the name we end up in new rules being created and "old" rule being deleted if we "change" a rule. Our rule names normally look like MARG_16GSWL_SFP_TXP_lt200n or WARN_E_PORTS_UTIL_gt75m.

So for example if I change a threshold for CRC errors on E-ports from 0 to 1 out script will generate and execute the following command:

mapsrule --create WARN_E_PORTS_C3TXTO_gt1m -group ALL_E_PORTS -monitor C3TXTO -timebase Min -op g -value 1 -action raslog -policy Company_6510_Policy mapsrule --delete WARN_E_PORTS_C3TXTO_gt0m -force mapspolicy --enable Company_6510_Policy

daniel-chung-broadcom commented 3 years ago

Thanks for the info.

Which of these actions were you able to create playbooks using the existing list module? I think you would be able to create based on your earlier post. But I think you don't have ability to delete a specific policy that you just created. Would ability to do something like this satisfy your requirement?

As for enabling the policy, I'll need to do bit more research on it myself.

munokar commented 3 years ago

To be honest deleting a policy is something that is done very seldom since we do not change the policy name. It normal just contains the type of the switch and the word policy ;-) As written it would be good to have the possibility to delete all rules that are not in a policy and that are not default rules. So the ability to delete a single rule would help somehow. Another option would that it actually works like the the policy stuff where I specify a list of rules for the policy and everything that is not specified gets deleted. This would allow us to create a list of rules (incl. all def-rules) which need to be created (if doesn't exist) or updated (if changed) and everything else gets deleted.

daniel-chung-broadcom commented 3 years ago

Just to clarify, the example that I gave was using policy for my own convenience and the new option should apply to any list-based objects, including maps rules by specifying "rule" for list_name. Also, delete_entries is an array, just like "entries" is. So, the example can be extended to delete multiple entries in the list. So, the example can be extended to delete a list of rules and a replay of the playbook should confirm no change - meaning the rules specified are not present.

As for the ability to delete all rules that are not specified by the policy, I think that might be difficult to do straight up. maps-policy and rule are two different lists. I am told that removing a rule from a policy does not remove the rule from the rule list. So, the way you can enforce such implementation is to use playbooks that contain the full database with all_entries attributes set to True for and rule. If all_entries are set to True, entries that are not specified are automatically deleted. But the complication here would be that the full list must contain all the default entries and I believe that will trigger the play to fail. So, technically possible, this doesn't sound like a good solution.

Let me know what you think

munokar commented 3 years ago

I also though about the solution by using the all_entries: True also it would be necessary to populate the list with all default rules as you have described. The issue is that this has to be done dynamically since from FOS version to version the default rules are extended or changed and I don't want to update the vars in the playbook for every FOS version - it should only include our custom rules. Having the possibility for a delete_entries on any kind of brocade_list_obj sounds like a good idea. I think I'll try to go ahead with the all_entries and see how it can be done this way...

daniel-chung-broadcom commented 3 years ago

just thinking outloud here, I think I might be able to add specific logic to basically remove comparison on default is_predefined entries as part of comparison. Let me give it some thought.

Basically, when you specify all_entries to be true, the list of entries listed will be considered full entries of all non-default (user defined) entries. Would that make this use case bit easier to deal with? You would still need to specify all the rules that you created in the playbook, but the list won't need to included default rules.

munokar commented 3 years ago

That would really help a lot! Since you can't delete or modify default-rules I think it is good to exclude them for all rule based actions. The only thing that need to be possible is to add default rules to custom policies (which is mandatory for some default rules) - but I think this has nothing to do with creating / deleting the rules...

daniel-chung-broadcom commented 3 years ago

policy list is a separate thing and it has to contain the full list of all the rules (default or custom). So, I think the current implementation should still work. I think I'll need to exclude the default policies there as well.

Getting back to rules for now, here is the playbook that I was trying out. I started out with two rules and all_entries not specified (true by default). I verified that two rules are created. I updated the playbook to add rule3. That worked. I update the playbook again to remove rule3. And that worked ok as well. So, the playbook will basically confirm that the specified rules are in the database. If not, create the missing entries. If any are not specified, they will be removed.

Please let me know if this is along the line of what you were looking for.

munokar commented 3 years ago

So this example is already done with code changes on your side? When I try to run the following playbook:

It only works if I set all_entries: False otherwise I get the error: "error-message": "Specified rule is a default rule" which was expected from me... With all_entries: False it creates the rules but as expected it don't delete a rule if I remove it from the playbook.

daniel-chung-broadcom commented 3 years ago

no. :-) I am making changes as we discussed but nothing has been pushed to github.com/brocade/ansible yet. If I have the changes pushed to a side branch, will you be able to download from github and try it?

munokar commented 3 years ago

Sure just let me know I can test this tomorrow...

daniel-chung-broadcom commented 3 years ago

I just pushed my changes to the branch named documentation_attribute_content_change. This branch also contains the earlier fix related to issue #79.

I have

facts_maps.yml to print policy and rules - including all the default entries at the moment maps_rule.yaml to create two rules. I verified removing one of the rules and removing all the rules (leave entries to be empty) maps_policy.yaml to create one inactive policy containing these two rules created. maps_policy_delete.yml to use delete_entries option.

You can activate the policy by updating maps_policy.yaml to is_active_policy to True.

One thing to note here is that if you update the underlying rule, apply the change and re-run maps_policy.yaml with is_active_policy set to true, reactivation doesn't take place since ansible does not see any diffs for the policy. As a workaround, you may want to check for scenarios where rules have changed. And if you see any change, you can toggle the active policy to one of the default entry and before running the portion of playbook with policy.

daniel-chung-broadcom commented 3 years ago

also created named modules: brocade_maps_maps_policy and brocade_maps_rule.

munokar commented 3 years ago

I had a chance to test it and it works fine and as expected. But I found an "issue" I'm not sure how to get around it.

Only rules can be removed that are not member of a policy. This can be an issue if I need to create and delete a rule within one task. Let's say I have a policy named MARG_16GSWL_SFP_RXP_lt80n which will trigger if the receive power of a 16 GBit/s SFP drops below 80 mW. If I change this threshold to 100 mW the rule need to be changed but also renamed. Since I don't see how renaming a rule is possible we will normally delete the rule MARG_16GSWL_SFP_RXP_lt80n and create a new rule with the name MARG_16GSWL_SFP_RXP_lt100n. Using all_entries: True will fail since the rule MARG_16GSWL_SFP_RXP_lt80n is still on the policy. If I try to update the policy first he already tries to add the rule MARG_16GSWL_SFP_RXP_lt100n to the policy which will fail because the rule is not there yet... On the CLI there is an easy solution for that. The -force switch when doing a mapsrule --delete which removes the rule from all policies. Checking the RestAPI I can't see anything like there :-(

I attached my playbook I used for testing. If you run it once it will work perfectly but if you change the name of the rule and rerun it will fail with the error:

"error-message": "Specified rule is associated with a policy"

create_maps_rule_policy.yml.txt

daniel-chung-broadcom commented 3 years ago

Probably not the best solution but just something top of my head. I could basically clean out the rules from the policy before rules. But the problem here is that the first and the third task will always apply the change. This only be changed if the name of the rule changed in the middle and the third task. Doesn't feel smooth regardless. Let me give some additional thought here.

daniel-chung-broadcom commented 3 years ago

I just pushed a change in maps_rule_policy.yml on that same branch. Do you mind checking it out and see if something like that would work? Basically, I ran the rule task first in checkmode. If it returns true, I'll clear out the policy, delete/create rules for real, and update the policy. If the checkmode returns false, we'll skip the task that clears the policy and just run the rest of the playbook normally. I ran few scenarios and everything runs as expected including dry-run.

It is still not entirely fine targeted since we don't know if the rule change would trigger a deletion of an existing rule or maybe the change was just an addition to rules list and didn't require policy update. So, dry run wouldn't be 100% correct depending on some scenarios.

Let me know how you feel about it.

daniel-chung-broadcom commented 3 years ago

btw, added benefit with this sequence is that if anything changed within the rule (for example threshold) but naming didn't change, the changes will be applied because policy will delete/add the rules back. Previously, policy would not have been updated and the rule change would not be applied.

munokar commented 3 years ago

I had a look at the solution and recreated your playbook in our environment. There I two things 1 can see here:

  1. I think there is a small issue in the maps_rule_policy.yml because the default for the policy is is_active_policy: False so the policy never gets enabled when I run it.
  2. If I change to 'is_active_policy: True' it creates everything and get the job done. But if I check the switch afterwards and execute the command mapspolicy --show -all I see the following message in the last line: WARNING: The policy configuration for the active policy has changed because of addition/deletion/modification of rule(s). The active policy has to be re-enabled to apply these new changes. Also the policy is active and the rules are in there it looks like there is a change after the policy gets enabled.

In general you approach should be OK but I think it has two major disadvantages:

  1. If you disable the policy to clear off all rules from it the default policy gets active with might trigger some event and perhaps send messages. traps etc to the monitoring systems from events you don't want to have there...
  2. I'm not sure if something like quiet times will work for rules when they are removed from the policy and across enabling different policies

So I think whether the solution you provided is good or not depends a bit on the frequency of changes and how the infrastructure is reacting to the enabling of default policiy.

Fir reference I attached my playbook. create_maps_policy_rule.yml.txt

daniel-chung-broadcom commented 3 years ago

Thanks for the info.

Would setting the policy to remain enable but clearing out the rules address some aspects of concern with generating events with default rules. The concern will still be that real events are not generated based on the rules that should still be in effect between the clearing of the rules, rules update, and policy enable.

As for the pending changes warning, let me find out what is going on there. I am also able to repro in my setup.

daniel-chung-broadcom commented 3 years ago

Turns out that maps policy cannot be disabled by setting is_active_policy to be False. The only way to disable an active policy is to enable something else. So, I updated the playbook to create a dummy policy with empty rules if the rules database is changing. This removes the warning that you were seeing. I think it avoids the problem of generating events based on some default or existing policies. But it does still have the issue of potentially missing events while the configuration is happening.

I also requested that we open a defect to properly document the is_active_policy behavior.

Let me know what you think.

Thanks.

daniel-chung-broadcom commented 3 years ago

one more update. I just pushed some changes to the module & maps_rule_policy.yml.

I updated the brocade_maps_rule to return remaining_brocade_maps_rule, which is the list of rules that are going to remain. So, if you changed from

rule1/rule2

to

rule3/rule2,

the list being returned is rule2. And I used this rule to enable the dummy policy. So, the sequence of the play should be.

1) check to see if any rule db is changing 2) if rule db is changing, enable the dummy policy with the final rules minus the rules being delete and rules being added 3) update the rule db 4) update the policy

So, this effectively allow rules that are not deleted to remain effective before #3 & #4.

Let me know what you think.

munokar commented 3 years ago

Hello, sorry for the late response! I had a look and it sounds like a good solution to work with the interim config only containing remaining rules. I will make some more detailed tests and try to implement a solution for our environment based on the described procedure.

daniel-chung-broadcom commented 3 years ago

No problem. Let me know how the latest works out. If all looks good, I'll plan to test, merge and update galaxy collection. Thanks.

munokar commented 2 years ago

Hello Daniel,

although the solution is not very elegant solution it works fine working with an interim config.

Thanks for your support.

munokar commented 2 years ago

Hi, have the MAPS related changes we discussed made it to the galaxy? Or can you tell me which branch I could use to continue testing in that area?

Thanks.

prabhu-broadcom commented 2 years ago

Hi, The side branch documentation_attribute_content_change has many features which are not part of the main branch yet. As galaxy main branch requires to go through the full testing cycle before the release, its planned to be made available in the next major release. Please continue to use this side branch (documentation_attribute_content_change) till then for this maps feature.

Thanks,

prasad-valmeti-broadcom commented 1 year ago

This is just to give an update. Plan is in progress to merge documentation branch changes into main/galaxy. Will update once the plan is finalized.

prasad-valmeti-broadcom commented 1 year ago

The issue is already addressed in Ansible FOS 1.3.3. If there are no further issues, please suggest if we can close the issue.

prasad-valmeti-broadcom commented 1 year ago

Since there is no response and the issue is already addressed in Ansible FOS 1.3.3, closing this issue. Please let us know your opinion on this.

prasad-valmeti-broadcom commented 5 months ago

If there are no further issues, we would like to close this issue. Please suggest your opinion on this.