splunk / pytest-splunk-addon

A Dynamic test tool for Splunk Technology Add-ons
https://splunk.github.io/pytest-splunk-addon/
Apache License 2.0
57 stars 20 forks source link

Requirement test sample files should be validated as XMLs before used #548

Open hsekowski-splunk opened 2 years ago

hsekowski-splunk commented 2 years ago

If element is not formatted correctly (eg. <field name="dest" value="so1" \> instead of <field name="dest" value="so1" />), tests are not executed and misleading information is given to the user: WARNING pytest-splunk-addon:splunk.py:831 Could not connect to Universal Forwarder Instance. Will try again. exception=[Errno 0] Error INFO pytest-splunk-addon:splunk.py:817 Trying to connect Universal Forwarder instance... splunk={"uf_host": "127.0.0.1", "uf_port": 61328, "uf_username": "admin", "uf_password": "[some password]"} INFO pytest-splunk-addon:splunk.py:825 Connected to Universal Forwarder instance. INFO pytest-splunk-addon:splunk.py:784 Events cleanup was disabled. DEBUG filelock:_api.py:169 Attempting to acquire lock 140391159921680 on ef875bad1f9e4cfc94db55edbf5a90d1_events.lock DEBUG filelock:_api.py:173 Lock 140391159921680 acquired on ef875bad1f9e4cfc94db55edbf5a90d1_events.lock DEBUG filelock:_api.py:203 Attempting to release lock 140391159921680 on ef875bad1f9e4cfc94db55edbf5a90d1_events.lock DEBUG filelock:_api.py:206 Lock 140391159921680 released on ef875bad1f9e4cfc94db55edbf5a90d1_events.lock INFO pytest-splunk-addon:hec_event_ingestor.py:120 Making a HEC event request with the following params: hec_uri:https://127.0.0.1:61242/services/collector headers:{'Authorization': 'Splunk 9b741d03-43e9-4164-908b-e09102327d22'} DEBUG urllib3.connectionpool:connectionpool.py:1005 Starting new HTTPS connection (1): 127.0.0.1:61242 DEBUG urllib3.connectionpool:connectionpool.py:465 https://127.0.0.1:61242 "POST /services/collector/event HTTP/1.1" 200 27 =========================================================================================================== short test summary info ============================================================================================================ ERROR tests/requirement_test/test_addon.py::Test_App::test_events_with_untokenised_values - File "<string>", line None =================================================================================================== 1 skipped, 1 error in 194.39s (0:03:14) ==================================================================================================== Desirable is to have information that "XML is not formatted correctly" and tests terminated.

nandinivij commented 2 years ago

We are doing this in the requirement test unit test - which is a prerequisite test to a requirement test. We ideally want to restrict running requirement tests if prereq tests fail. I will make a note to add that restriction in the CI.