Closed Niltak closed 2 years ago
File running the module:
rule_list = [
{'ports': 'HTTPS', 'network': 'obj1'},
{'ports': 'obj_tcp_5100', 'network': 'obj2'},
{'ports': 'obj_tcp_1521', 'network': 'obj3'},
{'ports': 'obj_tcp_1525', 'network': 'obj4'},
{'ports': 'obj_tcp_1529', 'network': 'obj5'},
{'ports': 'obj_tcp_1530', 'network': 'obj6'}]
with Connection(
username=fmc_user,
password=fmc_pwd,
timeout=30) as test:
count = 1
for rule in rule_list:
test.rule_add(RuleTemplate(
f'test_rule_soa_{count}',
'automation_testing',
source_zones='test1',
source_networks='target_obj',
destination_networks=rule['network'],
destination_ports=rule['ports'],
new_comments='test_rule_api'))
count += 1
This appears to be an incomplete example of the problem. What is RuleTemplate
?
Sorry I was trying to reduce the complexity since the code works outside using the same fmc object.
class Rule(dict):
def __init__(
self, name, access_policy,
action=None, enabled=None, sendEventsToFMC=None,
logFiles=None, logBegin=None, logEnd=None,
intrusion_policy=None, variable_set=None,
vlan_tags=None, urls_info=None,
source_zones=None, destination_zones=None,
source_ports=None, destination_ports=None,
source_networks=None, destination_networks=None,
remove_source_zones=None, remove_destination_zones=None,
remove_source_ports=None, remove_destination_ports=None,
remove_source_networks=None, remove_destination_networks=None,
new_comments=None, id=None):
self.name = name
self.access_policy = access_policy
self.action = action
self.enabled = enabled
self.sendEventsToFMC = sendEventsToFMC
self.logFiles = logFiles
self.logBegin = logBegin
self.logEnd = logEnd
self.intrusion_policy = intrusion_policy
self.variable_set = variable_set
self.vlan_tags = vlan_tags
self.urls_info = urls_info
self.source_zones = source_zones
self.destination_zones = destination_zones
self.source_ports = source_ports
self.destination_ports = destination_ports
self.source_networks = source_networks
self.destination_networks = destination_networks
self.remove_source_zones = remove_source_zones
self.remove_destination_zones = remove_destination_zones
self.remove_source_ports = remove_source_ports
self.remove_destination_ports = remove_destination_ports
self.remove_source_networks = remove_source_networks
self.remove_destination_networks = remove_destination_networks
self.new_comments = new_comments
self.id = id
def info(self):
'''
Returns a dictionary of the class.
'''
return self.__dict__
class RuleTemplate(Rule):
def __init__(
self, name, access_policy,
action='ALLOW', enabled=False, sendEventsToFMC=True,
logFiles=False, logBegin=False, logEnd=True,
intrusion_policy='t_intrusion_dc1', variable_set='t_variableset',
**kwargs):
super().__init__(
name, access_policy,
action=action, enabled=enabled, sendEventsToFMC=sendEventsToFMC,
logFiles=logFiles, logBegin=logBegin, logEnd=logEnd,
intrusion_policy=intrusion_policy, variable_set=variable_set,
**kwargs)
example of a work around:
rule_list = [
{'ports': 'HTTPS', 'network': 'obj1'},
{'ports': 'obj_tcp_5100', 'network': 'obj2'},
{'ports': 'obj_tcp_1521', 'network': 'obj3'},
{'ports': 'obj_tcp_1525', 'network': 'obj4'},
{'ports': 'obj_tcp_1529', 'network': 'obj5'},
{'ports': 'obj_tcp_1530', 'network': 'obj6'}]
count = 1
for rule in rule_list:
with Connection(
username=fmc_user,
password=fmc_pwd,
timeout=30) as test:
test.rule_add(RuleTemplate(
f'test_rule_soa_{count}',
'automation_testing',
source_zones='test1',
source_networks='target_obj',
destination_networks=rule['network'],
destination_ports=rule['ports'],
new_comments='test_rule_api'))
count += 1
I do not see the problem you are reporting in either the FMCAPI code that was released this week or the previous version. Here are my FMCAPI logs:
2022-09-16 08:05:38,999 257:apiclasstemplate INFO POST success. Object with name: "marksull-dup" and id: "0050568E-B2F8-0ed3-0000-176094164300" created in FMC.
2022-09-16 08:06:01,204 170:apiclasstemplate INFO GET success. Object with name: "marksull-dup" and id: "0050568E-B2F8-0ed3-0000-176094164300" fetched from FMC.
2022-09-16 08:06:10,113 257:apiclasstemplate INFO POST success. Object with name: "test_rule_soa_1" and id: "0050568E-B2F8-0ed3-0000-000268849882" created in FMC.
---
2022-09-16 08:06:20,185 170:apiclasstemplate INFO GET success. Object with name: "marksull-dup" and id: "0050568E-B2F8-0ed3-0000-176094164300" fetched from FMC.
2022-09-16 08:06:21,699 257:apiclasstemplate INFO POST success. Object with name: "test_rule_soa_2" and id: "0050568E-B2F8-0ed3-0000-000268849883" created in FMC.
---
2022-09-16 08:06:25,587 170:apiclasstemplate INFO GET success. Object with name: "marksull-dup" and id: "0050568E-B2F8-0ed3-0000-176094164300" fetched from FMC.
2022-09-16 08:06:27,175 257:apiclasstemplate INFO POST success. Object with name: "test_rule_soa_3" and id: "0050568E-B2F8-0ed3-0000-000268849884" created in FMC.
---
2022-09-16 08:06:30,198 170:apiclasstemplate INFO GET success. Object with name: "marksull-dup" and id: "0050568E-B2F8-0ed3-0000-176094164300" fetched from FMC.
2022-09-16 08:06:31,679 257:apiclasstemplate INFO POST success. Object with name: "test_rule_soa_4" and id: "0050568E-B2F8-0ed3-0000-000268849885" created in FMC.
---
2022-09-16 08:06:36,085 170:apiclasstemplate INFO GET success. Object with name: "marksull-dup" and id: "0050568E-B2F8-0ed3-0000-176094164300" fetched from FMC.
2022-09-16 08:06:37,578 257:apiclasstemplate INFO POST success. Object with name: "test_rule_soa_5" and id: "0050568E-B2F8-0ed3-0000-000268849886" created in FMC.
---
2022-09-16 08:06:40,808 170:apiclasstemplate INFO GET success. Object with name: "marksull-dup" and id: "0050568E-B2F8-0ed3-0000-176094164300" fetched from FMC.
2022-09-16 08:06:42,307 257:apiclasstemplate INFO POST success. Object with name: "test_rule_soa_6" and id: "0050568E-B2F8-0ed3-0000-000268849887" created in FMC.
2022-09-16 08:06:42,308 163:fmc INFO Auto deploy changes set to False. Use the Deploy button in FMC to push changes to FTDs.
---
Because it was an incomplete example, I simply set the rule name and action the same on each rule, so I don't know what other activities you were performing in the rule_add
method.
rule_details.name = rule.name
rule_details.action = "ALLOW"
The best way forward is to provide the most minimal example of the problem you are experiencing without any additional complexities, just the minimum raw FMCAPI calls that recreate your problem. Also, include your FMCAPC logs as above.
If I were to guess your issue, it would be a Pass By Reference issue, where you are inadvertently passing around a list between methods, and it is being appended to and reused on successive iterations.
Sorry for not getting back sooner. Emergency at work.
Here is a less complex example:
import fmcapi
from userlist import fmc_user, fmc_pwd
class Connection(fmcapi.FMC):
def __init__(
self,
host="redacted",
username="admin",
password="Admin123",
domain='test-it',
autodeploy=False,
file_logging=None,
logging_level="INFO",
debug=False,
limit=1000,
timeout=15):
super().__init__(
host, username, password,
domain, autodeploy,
file_logging, logging_level,
debug, limit, timeout)
def test_loop(self):
'''
'''
rule_list = [
{'ports': 'HTTPS', 'network': 'any'},
{'ports': 'obj_tcp_5100', 'network': 'any-ipv4'},
{'ports': 'obj_tcp_1521', 'network': 'any-ipv4-test'}]
count = 0
for rule in rule_list:
rule_details = fmcapi.AccessRules(
fmc=self, acp_name='automation_testing',
category='Automation', action='ALLOW')
count += 1
rule_details.name = f'test_rule_{count}'
rule_details._enabled = True
rule_details._sendEventsToFMC = True
rule_details._logEnd = True
rule_details.intrusion_policy('set', name='test_intrusion_dc1')
rule_details.variable_set('set', name='test_variableset')
rule_details.new_comments('add', value='test_rule_api')
rule_details.destination_port('add', name=rule['ports'])
rule_details.source_network('add', name=rule['network'])
rule_details.post()
with Connection(
username=fmc_user,
password=fmc_pwd,
timeout=30) as test:
test.test_loop()
INFO:root:Requesting new tokens from https://{hostname redacted}/api/fmc_platform/v1/auth/generatetoken.
INFO:root:Domain name entered not found in FMC, falling back to Global
INFO:root:Domain set to Global/test-it
INFO:root:Building base to URLs.
INFO:root:Populating vdbVersion, sruVersion, serverVersion, and geoVersion FMC instance variables.
INFO:root:This FMC's version is 7.0.1.1 (build 11)
INFO:root:GET success. Object with name: "automation_testing" and id: "3C5731FF-5F3A-0ed3-0000-228856506384" fetched from FMC.
INFO:root:GET success. Object with name: "test_intrusion_dc1" and id: "0e3c1b4e-7291-11ec-895d-23b674325230" fetched from FMC.
INFO:root:Intrusion Policy set to "test_intrusion_dc1" for this AccessRules object.
INFO:root:GET success. Object with name: "test_variableset" and id: "7a98350e-5f58-11eb-9a7f-9024cd0a25b7" fetched from FMC.
INFO:root:VariableSet set to "test_variableset" for this AccessRules object.
INFO:root:GET success. Object with name: "HTTPS" and id: "1834bd00-38bb-11e2-86aa-62f0c593a59a" fetched from FMC.
WARNING:root: GET query for HTTPS is not found.
INFO:root:Adding "HTTPS" to destinationPorts for this AccessRules.
INFO:root:Adding "any" to sourceNetworks for this AccessRules.
INFO:root:POST success. Object with name: "test_rule_1" and id: "3C5731FF-5F3A-0ed3-0000-000268459256" created in FMC.
---
INFO:root:GET success. Object with name: "automation_testing" and id: "3C5731FF-5F3A-0ed3-0000-228856506384" fetched from FMC.
INFO:root:GET success. Object with name: "test_intrusion_dc1" and id: "0e3c1b4e-7291-11ec-895d-23b674325230" fetched from FMC.
INFO:root:Intrusion Policy set to "test_intrusion_dc1" for this AccessRules object.
INFO:root:GET success. Object with name: "test_variableset" and id: "7a98350e-5f58-11eb-9a7f-9024cd0a25b7" fetched from FMC.
INFO:root:VariableSet set to "test_variableset" for this AccessRules object.
INFO:root:GET success. Object with name: "obj_tcp_5100" and id: "3C5731FF-5F3A-0ed3-0000-008589942277" fetched from FMC.
WARNING:root: GET query for obj_tcp_5100 is not found.
INFO:root:Adding "obj_tcp_5100" to destinationPorts for this AccessRules.
INFO:root:Adding "any-ipv4" to sourceNetworks for this AccessRules.
INFO:root:POST success. Object with name: "test_rule_1" and id: "3C5731FF-5F3A-0ed3-0000-000268459257" created in FMC.
---
INFO:root:GET success. Object with name: "automation_testing" and id: "3C5731FF-5F3A-0ed3-0000-228856506384" fetched from FMC.
INFO:root:GET success. Object with name: "test_intrusion_dc1" and id: "0e3c1b4e-7291-11ec-895d-23b674325230" fetched from FMC.
INFO:root:Intrusion Policy set to "test_intrusion_dc1" for this AccessRules object.
INFO:root:GET success. Object with name: "test_variableset" and id: "7a98350e-5f58-11eb-9a7f-9024cd0a25b7" fetched from FMC.
INFO:root:VariableSet set to "test_variableset" for this AccessRules object.
INFO:root:GET success. Object with name: "obj_tcp_1521" and id: "3C5731FF-5F3A-0ed3-0000-008589942450" fetched from FMC.
WARNING:root: GET query for obj_tcp_1521 is not found.
INFO:root:Adding "obj_tcp_1521" to destinationPorts for this AccessRules.
INFO:root:Adding "any-ipv4-test" to sourceNetworks for this AccessRules.
INFO:root:POST success. Object with name: "test_rule_1" and id: "3C5731FF-5F3A-0ed3-0000-000268459259" created in FMC.
---
INFO:root:Auto deploy changes set to False. Use the Deploy button in FMC to push changes to FTDs.
To add to the weirdness of this bug, sometimes (mostly when I run it for the first time of the day) it runs this script without the bug. I have restarted my IDE to see if it was some issue with vars getting cached but it doesn't seem to be the case.
You are not making my life easy ;-)
A couple of things:
I ran your code 4 times with varying complexity (excluding some of the dependencies hoping to recreate the issue in the simplest way possible), but I always would only see 3 records created.
In the example above that you provided, what did the ACP look like on the FMC?
My bad for not removing more complexities. My brain that morning thought I needed those vars to create any rule.
To be clear, my last post has the logs from the above code, but it was edited to take out some redacted vars. I added the dashes and added a print dash line to the code after to make it more readable.
The above logs show that it created 'test_rule_1' 3 times. It doesn't log the output for the other rules created.
Here is my current code with the least amount of complexities for my issue:
import fmcapi
from aa_global import fmc_user, fmc_pwd, fmc_host, fmc_domain
class Connection(fmcapi.FMC):
def __init__(
self,
host="redacted",
username="admin",
password="Admin123",
domain='test-it',
autodeploy=False,
file_logging=None,
logging_level="INFO",
debug=False,
limit=1000,
timeout=15):
super().__init__(
host, username, password,
domain, autodeploy,
file_logging, logging_level,
debug, limit, timeout)
def test_loop(self):
count = 0
while count < 3:
rule_details = fmcapi.AccessRules(
fmc=self, acp_name='automation_testing', action='ALLOW')
count += 1
rule_details.name = f'test_rule_{count}'
rule_details.post()
print('---')
with Connection(
host=fmc_host,
username=fmc_user,
password=fmc_pwd,
domain=fmc_domain,
timeout=30) as test:
test.test_loop()
Logging of the Bug
INFO:root:Requesting new tokens from https://prdfmc01/api/fmc_platform/v1/auth/generatetoken.
INFO:root:Domain name entered not found in FMC, falling back to Global
INFO:root:Domain set to {{REDACTED}}
INFO:root:Building base to URLs.
INFO:root:Populating vdbVersion, sruVersion, serverVersion, and geoVersion FMC instance variables.
INFO:root:This FMC's version is 7.0.1.1 (build 11)
INFO:root:GET success. Object with name: "automation_testing" and id: "3C5731FF-5F3A-0ed3-0000-128856506384" fetched from FMC.
INFO:root:POST success. Object with name: "test_rule_1" and id: "3C5731FF-5F3A-0ed3-0000-000268459283" created in FMC.
---
INFO:root:GET success. Object with name: "automation_testing" and id: "3C5731FF-5F3A-0ed3-0000-128856506384" fetched from FMC.
INFO:root:POST success. Object with name: "test_rule_1" and id: "3C5731FF-5F3A-0ed3-0000-000268459284" created in FMC.
---
INFO:root:GET success. Object with name: "automation_testing" and id: "3C5731FF-5F3A-0ed3-0000-128856506384" fetched from FMC.
INFO:root:POST success. Object with name: "test_rule_1" and id: "3C5731FF-5F3A-0ed3-0000-000268459286" created in FMC.
---
INFO:root:Auto deploy changes set to False. Use the Deploy button in FMC to push changes to FTDs.
Logging when it randomly works:
INFO:root:Requesting new tokens from https://prdfmc01/api/fmc_platform/v1/auth/generatetoken.
INFO:root:Domain name entered not found in FMC, falling back to Global
INFO:root:Building base to URLs.
INFO:root:Populating vdbVersion, sruVersion, serverVersion, and geoVersion FMC instance variables.
INFO:root:This FMC's version is 7.0.1.1 (build 11)
INFO:root:GET success. Object with name: "automation_testing" and id: "3C5731FF-5F3A-0ed3-0000-128856506384" fetched from FMC.
INFO:root:POST success. Object with name: "test_rule_1" and id: "3C5731FF-5F3A-0ed3-0000-000268459280" created in FMC.
---
INFO:root:GET success. Object with name: "automation_testing" and id: "3C5731FF-5F3A-0ed3-0000-128856506384" fetched from FMC.
INFO:root:POST success. Object with name: "test_rule_2" and id: "3C5731FF-5F3A-0ed3-0000-000268459281" created in FMC.
---
INFO:root:GET success. Object with name: "automation_testing" and id: "3C5731FF-5F3A-0ed3-0000-128856506384" fetched from FMC.
INFO:root:POST success. Object with name: "test_rule_3" and id: "3C5731FF-5F3A-0ed3-0000-000268459282" created in FMC.
---
INFO:root:Auto deploy changes set to False. Use the Deploy button in FMC to push changes to FTDs.
Picture of before and after:
Honestly, don't know. I just tested it 4 different times, and it always works fine. We use this code in production many times daily and haven't seen this issue in the wild either.
Thinking aloud...
1) Any chance you can initiate the fmcapi
connection with debug=true
and capture the logs and attach it as a file here (make sure you removed any potential private information)?
2) What version of FMC are you using? Has there been a recent change to your FMC version, and if so, do you have a previous version to test on to see if the problem follows the FMC or follows the code?
Here you go: fmcapi_logs.txt
FMC v7.0.1.1 I am attempting to get ahold of a test server for v7.0.4 to see if that works. There is a bug report that is similar to what I am experiencing, but it was 'fixed' on v7.0.1 . https://bst.cloudapps.cisco.com/bugsearch/bug/CSCvy03907
I am ok saying this isn't an API issue on fmcapi. Everything seems to be good when it gets handed off to FMC. The only thing that is odd is this bug hasn't happen if I create a new connection object every time I make a new rule.
IMHO CSCvy03907 does not appear related to this issue.
The log definitely highlights an anomaly. Check this out:
I honestly don't know what is going on there.
My next troubleshooting step (I would try this myself, but I cannot recreate the issue) would be to take the URL/JSON Data from that debug log and use postman (or similar) and post the exact same data using a single session key and see if you get the same response (i.e. get back a different rule name). Then repeat the tests with a fresh session key for each post (which should be the equivalent of creating a new FMCAPI connection object each time) and see if this works without issue.
If you see similar results to the above anomaly, then we know this is not FMCAPI package related. If you do not see the same issue, then I am going to further confused ;-)
Confirmed with Postman that this issue is on the FMC side. Thanks for the help with this.
Very unusual problem. If you end up opening a case with TAC and get the bug id, if you could drop it in here so we have reference to it that would be helpful.
Describe the bug While using the same fmc object in a loop and posting a set new access rule, it will post every rule before it.
If I am looping 6 new rules, it will post 1 - 12 -123 -1234 -12345 - 123456 for a total of 21 rules created.
To Reproduce Loop creation of access rules while using a with statement of a fmc object.
Expected behavior Only posting 6 rules when supplied 6 rules.
Python environment
Additional context Add any other context about the problem here.