search

StackStorm / st2

StackStorm (aka "IFTTT for Ops") is event-driven automation for auto-remediation, incident responses, troubleshooting, deployments, and more for DevOps and SREs. Includes rules engine, workflow, 160 integration packs with 6000+ actions (see https://exchange.stackstorm.org) and ChatOps. Installer at https://docs.stackstorm.com/install/index.html
https://stackstorm.com/
Apache License 2.0
5.96k stars 741 forks source link

Enable completely offline st2-rule-tester #6208

Open jk464 opened 1 month ago

jk464 commented 1 month ago

To enable offline testing of StackStorm rules (i.e. in a CI pipeline) this PR allows st2-rule-tester to run w/o a MongoDB connection (which was the only external requirement the script required)

st2-rule-tester only requires a connection to Mongo if you use the rule-ref and/or trigger-instance-id flags - as they reference entries in the MongoDB. If you only want to provide rule and trigger-instance yaml files (for example in CI testing pipeline of rules w/in a pack) then the Mongo connection is never required.

This PR adds an --offline flag, which if provided, skips the setting up of the mongo connection - otherwise (and by default) the connection is still setup.

There's also a check that if the --offline flag is provide that neither rule-ref and/or trigger-instance-id flags are provided - and if so exits with an error informing the user they can't be used in conjunction with the --offline flag.

This enables the following usecase https://stackstorm-community.slack.com/archives/C066APT88/p1716561254326499

jk464 commented 1 month ago

I tested to make sure that w/o --offline a mongo connection is attempt, with it one isnt setup, and that providing any combination of rule-ref and/or trigger-instance-id with offline causes the script to error out gracefully:

root@845ca895c5c0:/opt/stackstorm/test# st2-rule-tester --rule ./my_rule.yaml --trigger-instance ./trigger_instance_1.yaml
2024-05-24 16:49:32,449 INFO [-] Connecting to database "st2" @ "127.0.0.1:27017" as user "None".
2024-05-24 16:49:35,495 ERROR [-] Failed to connect to database "st2" @ "127.0.0.1:27017" as user "None": 127.0.0.1:27017: [Errno 111] Connection refused, Timeout: 3.0s, Topology Description: <TopologyDescription id: 6650c51cca5e714f5a64df6d, topology_type: Single, servers: [<ServerDescription ('127.0.0.1', 27017) server_type: Unknown, rtt: None, error=AutoReconnect('127.0.0.1:27017: [Errno 111] Connection refused')>]>
Traceback (most recent call last):
  File "/usr/bin/st2-rule-tester", line 21, in <module>
    rule_tester.main()
  File "/opt/stackstorm/st2/lib/python3.8/site-packages/st2reactor/cmd/rule_tester.py", line 66, in main
    db_setup()
  File "/opt/stackstorm/st2/lib/python3.8/site-packages/st2common/database_setup.py", line 55, in db_setup
    connection = db_init.db_setup_with_retry(**db_cfg)
  File "/opt/stackstorm/st2/lib/python3.8/site-packages/st2common/persistence/db_init.py", line 79, in db_setup_with_retry
    return db_func_with_retry(
  File "/opt/stackstorm/st2/lib/python3.8/site-packages/st2common/persistence/db_init.py", line 58, in db_func_with_retry
    return retrying_obj.call(db_func, *args, **kwargs)
  File "/opt/stackstorm/st2/lib/python3.8/site-packages/retrying.py", line 257, in call
    return attempt.get(self._wrap_exception)
  File "/opt/stackstorm/st2/lib/python3.8/site-packages/retrying.py", line 301, in get
    six.reraise(self.value[0], self.value[1], self.value[2])
  File "/opt/stackstorm/st2/lib/python3.8/site-packages/six.py", line 719, in reraise
    raise value
  File "/opt/stackstorm/st2/lib/python3.8/site-packages/retrying.py", line 251, in call
    attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
  File "/opt/stackstorm/st2/lib/python3.8/site-packages/st2common/models/db/__init__.py", line 243, in db_setup
    connection = _db_connect(
  File "/opt/stackstorm/st2/lib/python3.8/site-packages/st2common/models/db/__init__.py", line 217, in _db_connect
    raise e
  File "/opt/stackstorm/st2/lib/python3.8/site-packages/st2common/models/db/__init__.py", line 208, in _db_connect
    connection.admin.command("ping")
  File "/opt/stackstorm/st2/lib/python3.8/site-packages/pymongo/database.py", line 757, in command
    with self.__client._socket_for_reads(
  File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__
    return next(self.gen)
  File "/opt/stackstorm/st2/lib/python3.8/site-packages/pymongo/mongo_client.py", line 1387, in _socket_for_reads
    server = self._select_server(read_preference, session)
  File "/opt/stackstorm/st2/lib/python3.8/site-packages/pymongo/mongo_client.py", line 1346, in _select_server
    server = topology.select_server(server_selector)
  File "/opt/stackstorm/st2/lib/python3.8/site-packages/pymongo/topology.py", line 244, in select_server
    return random.choice(self.select_servers(selector,
  File "/opt/stackstorm/st2/lib/python3.8/site-packages/pymongo/topology.py", line 202, in select_servers
    server_descriptions = self._select_servers_loop(
  File "/opt/stackstorm/st2/lib/python3.8/site-packages/pymongo/topology.py", line 218, in _select_servers_loop
    raise ServerSelectionTimeoutError(
pymongo.errors.ServerSelectionTimeoutError: 127.0.0.1:27017: [Errno 111] Connection refused, Timeout: 3.0s, Topology Description: <TopologyDescription id: 6650c51cca5e714f5a64df6d, topology_type: Single, servers: [<ServerDescription ('127.0.0.1', 27017) server_type: Unknown, rtt: None, error=AutoReconnect('127.0.0.1:27017: [Errno 111] Connection refused')>]>
root@845ca895c5c0:/opt/stackstorm/test# st2-rule-tester --offline  --rule ./my_rule.yaml --trigger-instance ./trigger_instance_1.yaml
2024-05-24 16:49:39,769 INFO [-] Validating rule irc.relayed_matched_irc_message for pubmsg.
2024-05-24 16:49:39,777 INFO [-] 1 rule(s) found to enforce for pubmsg.
2024-05-24 16:49:39,778 INFO [-] Failed to retrieve config for pack <Mock name='mock.pack' id='140737394697888'> and user stanley: expected string or bytes-like object
2024-05-24 16:49:39,780 INFO [-] Action parameters resolved to:
2024-05-24 16:49:39,780 INFO [-]    message: Kami_ on #stackstorm: stackstorm is cool!
2024-05-24 16:49:39,780 INFO [-]    channel: #irc-relay
2024-05-24 16:49:39,780 INFO [-] === RULE MATCHES ===
root@845ca895c5c0:/opt/stackstorm/test# st2-rule-tester --offline  --rule-ref foo
2024-05-24 16:49:54,048 CRITICAL [-] 'rule-ref' and/or 'trigger-instance-id' cannot be used in 'offline' mode
root@845ca895c5c0:/opt/stackstorm/test# st2-rule-tester --offline  --trigger-instance-id foo
2024-05-24 16:50:02,121 CRITICAL [-] 'rule-ref' and/or 'trigger-instance-id' cannot be used in 'offline' mode
root@845ca895c5c0:/opt/stackstorm/test# st2-rule-tester --offline  --trigger-instance-id foo --rule-ref foo
2024-05-24 16:50:08,788 CRITICAL [-] 'rule-ref' and/or 'trigger-instance-id' cannot be used in 'offline' mode