search

remg427 / misp42splunk

A Splunk app to use MISP in background
GNU Lesser General Public License v3.0
109 stars 30 forks source link

4.0.1 rc1 python error in misp_common.py #186

Closed skalvaro closed 3 years ago

skalvaro commented 3 years ago

Installed 4.0.1 rc1 and mispgetioc returns a TypeError.

External search command 'mispgetioc' returned error code 1. Script output = "error_message=TypeError at "/opt/splunk/etc/apps/misp42splunk/bin/misp_common.py", line 51 : string indices must be integers ".

Tried on Splunk version 7.3.2 and 8.0.4.

remg427 commented 3 years ago

Hello, I tried to reproduce but it works on 8.1 and 7.2. Latest package include one logging at line 51 to check what you have as instance. normally you should get a dict that contains instance configurations

skalvaro commented 3 years ago

Hi, Thank you for looking into this. I may have found something. If I only configure a single misp instance I get the error, if I configure a second dummy instance everything works as expected.

As you can find in below logs we don't seem to get a correct dict if the configuration only contains a single misp instance.

single misp instance configured: search.log multiple misp instances configured: search2.log

So I added a log line to see what we get as misp_instances. single misp instance configured: search3.log multiple misp instances configured: search4.log

skalvaro commented 3 years ago

Hi,

Here is what works for me: If only a single instance is configured misp_instances is not a list so I found that 'response' also has 'totalResults' returned and used that to determine what part of the code to execute. During testing I also noticed that the [MC202] alert to indicate nothing is configered at all was never reached.

    response = helper.service.get('misp42splunk_account')
    if response.status == 200:
        data_body = splunklib.data.load(response.body.read())
    else:
        raise Exception("[MC2000] Unexpected status received %s", str(response.status))
        return None

    if int(data_body['feed']['totalResults']) == 0: # No misp instance configured
        raise Exception("[MC202] no misp instance configured. Please onfigure an entry for %s", str(misp_instance))
        return None
    elif int(data_body['feed']['totalResults']) == 1: # Single misp instance configured
        instance = data_body['feed']['entry']
        helper.log_debug("[MC1000] instance set: {}".format(instance))
        foundStanza = False
        if misp_instance == str(instance['title']):
            app_config = instance['content']
            foundStanza = True
    else: # Multiple misp instances configured
        misp_instances = data_body['feed']['entry']
        foundStanza = False
        for instance in list(misp_instances):
            helper.log_debug("[MC1000] instance set: {}".format(instance))
            if misp_instance == str(instance['title']):
                app_config = instance['content']
                foundStanza = True

    if not foundStanza:
        raise Exception("[MC201] no misp_instance with specified name found: %s ", str(misp_instance))
        return None

Below is a small part of the debug logs i gathered.

=== Nothing configured ===
[MC1002] response set: {'status': 200, 'reason': 'OK', 'headers': [('Date', 'Tue, 02 Feb 2021 11:15:41 GMT'), ('Expires', 'Thu, 26 Oct 1978 00:00:00 GMT'), ('Cache-Control', 'no-store, no-cache, must-revalidate, max-age=0'), ('Content-Type', 'text/xml; charset=UTF-8'), ('X-Content-Type-Options', 'nosniff'), ('Content-Length', '1775'), ('Vary', 'Authorization'), ('Connection', 'Close'), ('X-Frame-Options', 'SAMEORIGIN'), ('Server', 'Splunkd')], 'body': <splunklib.binding.ResponseReader object at 0x7ffbc5a4eb90>}
[MC1002] response type: <class 'splunklib.data.Record'>
[MC1002] response len: 4
[MC1001] data_body set: {'feed': {'title': None, 'id': 'https://127.0.0.1/servicesNS/nobody/search/misp42splunk_account', 'updated': '2021-02-02T11:15:41+00:00', 'generator': {'build': '767223ac207f', 'version': '8.0.4'}, 'author': {'name': 'Splunk'}, 'link': {'href': '/servicesNS/nobody/search/misp42splunk_account/_new', 'rel': 'create'}, 'totalResults': '0', 'itemsPerPage': '30', 'startIndex': '0', 'messages': None}}
[MC1002] data_body len: 1
[MC1002] data_body type: <class 'splunklib.data.Record'>

=== Single instance configured ===
[MC1002] response set: {'status': 200, 'reason': 'OK', 'headers': [('Date', 'Tue, 02 Feb 2021 11:20:03 GMT'), ('Expires', 'Thu, 26 Oct 1978 00:00:00 GMT'), ('Cache-Control', 'no-store, no-cache, must-revalidate, max-age=0'), ('Content-Type', 'text/xml; charset=UTF-8'), ('X-Content-Type-Options', 'nosniff'), ('Content-Length', '3888'), ('Vary', 'Authorization'), ('Connection', 'Close'), ('X-Frame-Options', 'SAMEORIGIN'), ('Server', 'Splunkd')], 'body': <splunklib.binding.ResponseReader object at 0x7fe103df4cd0>}
[MC1002] response type: <class 'splunklib.data.Record'>
[MC1002] response len: 4
[MC1001] data_body set: {'feed': {'title': None, 'id': 'https://127.0.0.1/servicesNS/nobody/search/misp42splunk_account', 'updated': '2021-02-02T11:20:03+00:00', 'generator': {'build': '767223ac207f', 'version': '8.0.4'}, 'author': {'name': 'Splunk'}, 'link': {'href': '/servicesNS/nobody/search/misp42splunk_account/_new', 'rel': 'create'}, 'totalResults': '1', 'itemsPerPage': '30', 'startIndex': '0', 'messages': None, 'entry': {'title': 'misp', 'id': 'https://127.0.0.1/servicesNS/nobody/misp42splunk/misp42splunk_account/misp', 'updated': '1970-01-01T00:00:00+00:00', 'link': [{'href': '/servicesNS/nobody/misp42splunk/misp42splunk_account/misp', 'rel': 'alternate'}, {'href': '/servicesNS/nobody/misp42splunk/misp42splunk_account/misp', 'rel': 'list'}, {'href': '/servicesNS/nobody/misp42splunk/misp42splunk_account/misp', 'rel': 'edit'}, {'href': '/servicesNS/nobody/misp42splunk/misp42splunk_account/misp', 'rel': 'remove'}], 'author': {'name': 'skalvaro'}, 'content': {'client_use_cert': '0', 'disabled': '0', 'eai:acl': {'app': 'misp42splunk', 'can_change_perms': '1', 'can_list': '1', 'can_share_app': '1', 'can_share_global': '1', 'can_share_user': '1', 'can_write': '1', 'modifiable': '1', 'owner': 'skalvaro', 'perms': {'read': ['*'], 'write': ['*']}, 'removable': '1', 'sharing': 'global'}, 'eai:appName': 'misp42splunk', 'eai:userName': 'nobody', 'misp_key': '********', 'misp_url': 'https://misp.skalvaro.lan', 'misp_use_proxy': '0', 'misp_verifycert': '0', 'type': 'text/xml'}}}}
[MC1002] data_body len: 1
[MC1002] data_body type: <class 'splunklib.data.Record'>
[MC1001] misp_instances set: {'title': 'misp', 'id': 'https://127.0.0.1/servicesNS/nobody/misp42splunk/misp42splunk_account/misp', 'updated': '1970-01-01T00:00:00+00:00', 'link': [{'href': '/servicesNS/nobody/misp42splunk/misp42splunk_account/misp', 'rel': 'alternate'}, {'href': '/servicesNS/nobody/misp42splunk/misp42splunk_account/misp', 'rel': 'list'}, {'href': '/servicesNS/nobody/misp42splunk/misp42splunk_account/misp', 'rel': 'edit'}, {'href': '/servicesNS/nobody/misp42splunk/misp42splunk_account/misp', 'rel': 'remove'}], 'author': {'name': 'skalvaro'}, 'content': {'client_use_cert': '0', 'disabled': '0', 'eai:acl': {'app': 'misp42splunk', 'can_change_perms': '1', 'can_list': '1', 'can_share_app': '1', 'can_share_global': '1', 'can_share_user': '1', 'can_write': '1', 'modifiable': '1', 'owner': 'skalvaro', 'perms': {'read': ['*'], 'write': ['*']}, 'removable': '1', 'sharing': 'global'}, 'eai:appName': 'misp42splunk', 'eai:userName': 'nobody', 'misp_key': '********', 'misp_url': 'https://misp.skalvaro.lan', 'misp_use_proxy': '0', 'misp_verifycert': '0', 'type': 'text/xml'}}
[MC1002] misp_instances len: 6
[MC1002] misp_instances type: <class 'splunklib.data.Record'>
[MC1000] instance set: title

=== Multiple instances configured ===
[MC1002] response set: {'status': 200, 'reason': 'OK', 'headers': [('Date', 'Tue, 02 Feb 2021 11:28:36 GMT'), ('Expires', 'Thu, 26 Oct 1978 00:00:00 GMT'), ('Cache-Control', 'no-store, no-cache, must-revalidate, max-age=0'), ('Content-Type', 'text/xml; charset=UTF-8'), ('X-Content-Type-Options', 'nosniff'), ('Content-Length', '6007'), ('Vary', 'Authorization'), ('Connection', 'Close'), ('X-Frame-Options', 'SAMEORIGIN'), ('Server', 'Splunkd')], 'body': <splunklib.binding.ResponseReader object at 0x7f4b60c31a10>}
[MC1002] response type: <class 'splunklib.data.Record'>
[MC1002] response len: 4
[MC1001] data_body set: {'feed': {'title': None, 'id': 'https://127.0.0.1/servicesNS/nobody/search/misp42splunk_account', 'updated': '2021-02-02T11:28:36+00:00', 'generator': {'build': '767223ac207f', 'version': '8.0.4'}, 'author': {'name': 'Splunk'}, 'link': {'href': '/servicesNS/nobody/search/misp42splunk_account/_new', 'rel': 'create'}, 'totalResults': '2', 'itemsPerPage': '30', 'startIndex': '0', 'messages': None, 'entry': [{'title': 'misp', 'id': 'https://127.0.0.1/servicesNS/nobody/misp42splunk/misp42splunk_account/misp', 'updated': '1970-01-01T00:00:00+00:00', 'link': [{'href': '/servicesNS/nobody/misp42splunk/misp42splunk_account/misp', 'rel': 'alternate'}, {'href': '/servicesNS/nobody/misp42splunk/misp42splunk_account/misp', 'rel': 'list'}, {'href': '/servicesNS/nobody/misp42splunk/misp42splunk_account/misp', 'rel': 'edit'}, {'href': '/servicesNS/nobody/misp42splunk/misp42splunk_account/misp', 'rel': 'remove'}], 'author': {'name': 'skalvaro'}, 'content': {'client_use_cert': '0', 'disabled': '0', 'eai:acl': {'app': 'misp42splunk', 'can_change_perms': '1', 'can_list': '1', 'can_share_app': '1', 'can_share_global': '1', 'can_share_user': '1', 'can_write': '1', 'modifiable': '1', 'owner': 'skalvaro', 'perms': {'read': ['*'], 'write': ['*']}, 'removable': '1', 'sharing': 'global'}, 'eai:appName': 'misp42splunk', 'eai:userName': 'nobody', 'misp_key': '********', 'misp_url': 'https://misp.skalvaro.lan', 'misp_use_proxy': '0', 'misp_verifycert': '0', 'type': 'text/xml'}}, {'title': 'misp2', 'id': 'https://127.0.0.1/servicesNS/nobody/misp42splunk/misp42splunk_account/misp2', 'updated': '1970-01-01T00:00:00+00:00', 'link': [{'href': '/servicesNS/nobody/misp42splunk/misp42splunk_account/misp2', 'rel': 'alternate'}, {'href': '/servicesNS/nobody/misp42splunk/misp42splunk_account/misp2', 'rel': 'list'}, {'href': '/servicesNS/nobody/misp42splunk/misp42splunk_account/misp2', 'rel': 'edit'}, {'href': '/servicesNS/nobody/misp42splunk/misp42splunk_account/misp2', 'rel': 'remove'}], 'author': {'name': 'skalvaro'}, 'content': {'client_use_cert': '0', 'disabled': '0', 'eai:acl': {'app': 'misp42splunk', 'can_change_perms': '1', 'can_list': '1', 'can_share_app': '1', 'can_share_global': '1', 'can_share_user': '1', 'can_write': '1', 'modifiable': '1', 'owner': 'skalvaro', 'perms': {'read': ['*'], 'write': ['*']}, 'removable': '1', 'sharing': 'global'}, 'eai:appName': 'misp42splunk', 'eai:userName': 'nobody', 'misp_key': '********', 'misp_url': 'https://misp.skalvaro.lan', 'misp_use_proxy': '0', 'misp_verifycert': '0', 'type': 'text/xml'}}]}}
[MC1002] data_body len: 1
[MC1002] data_body type: <class 'splunklib.data.Record'>
[MC1001] misp_instances set: [{'title': 'misp', 'id': 'https://127.0.0.1/servicesNS/nobody/misp42splunk/misp42splunk_account/misp', 'updated': '1970-01-01T00:00:00+00:00', 'link': [{'href': '/servicesNS/nobody/misp42splunk/misp42splunk_account/misp', 'rel': 'alternate'}, {'href': '/servicesNS/nobody/misp42splunk/misp42splunk_account/misp', 'rel': 'list'}, {'href': '/servicesNS/nobody/misp42splunk/misp42splunk_account/misp', 'rel': 'edit'}, {'href': '/servicesNS/nobody/misp42splunk/misp42splunk_account/misp', 'rel': 'remove'}], 'author': {'name': 'skalvaro'}, 'content': {'client_use_cert': '0', 'disabled': '0', 'eai:acl': {'app': 'misp42splunk', 'can_change_perms': '1', 'can_list': '1', 'can_share_app': '1', 'can_share_global': '1', 'can_share_user': '1', 'can_write': '1', 'modifiable': '1', 'owner': 'skalvaro', 'perms': {'read': ['*'], 'write': ['*']}, 'removable': '1', 'sharing': 'global'}, 'eai:appName': 'misp42splunk', 'eai:userName': 'nobody', 'misp_key': '********', 'misp_url': 'https://misp.skalvaro.lan', 'misp_use_proxy': '0', 'misp_verifycert': '0', 'type': 'text/xml'}}, {'title': 'misp2', 'id': 'https://127.0.0.1/servicesNS/nobody/misp42splunk/misp42splunk_account/misp2', 'updated': '1970-01-01T00:00:00+00:00', 'link': [{'href': '/servicesNS/nobody/misp42splunk/misp42splunk_account/misp2', 'rel': 'alternate'}, {'href': '/servicesNS/nobody/misp42splunk/misp42splunk_account/misp2', 'rel': 'list'}, {'href': '/servicesNS/nobody/misp42splunk/misp42splunk_account/misp2', 'rel': 'edit'}, {'href': '/servicesNS/nobody/misp42splunk/misp42splunk_account/misp2', 'rel': 'remove'}], 'author': {'name': 'skalvaro'}, 'content': {'client_use_cert': '0', 'disabled': '0', 'eai:acl': {'app': 'misp42splunk', 'can_change_perms': '1', 'can_list': '1', 'can_share_app': '1', 'can_share_global': '1', 'can_share_user': '1', 'can_write': '1', 'modifiable': '1', 'owner': 'skalvaro', 'perms': {'read': ['*'], 'write': ['*']}, 'removable': '1', 'sharing': 'global'}, 'eai:appName': 'misp42splunk', 'eai:userName': 'nobody', 'misp_key': '********', 'misp_url': 'https://misp.skalvaro.lan', 'misp_use_proxy': '0', 'misp_verifycert': '0', 'type': 'text/xml'}}]
[MC1002] misp_instances len: 2
[MC1002] misp_instances type: <class 'list'>
[MC1000] instance set: {'title': 'misp', 'id': 'https://127.0.0.1/servicesNS/nobody/misp42splunk/misp42splunk_account/misp', 'updated': '1970-01-01T00:00:00+00:00', 'link': [{'href': '/servicesNS/nobody/misp42splunk/misp42splunk_account/misp', 'rel': 'alternate'}, {'href': '/servicesNS/nobody/misp42splunk/misp42splunk_account/misp', 'rel': 'list'}, {'href': '/servicesNS/nobody/misp42splunk/misp42splunk_account/misp', 'rel': 'edit'}, {'href': '/servicesNS/nobody/misp42splunk/misp42splunk_account/misp', 'rel': 'remove'}], 'author': {'name': 'skalvaro'}, 'content': {'client_use_cert': '0', 'disabled': '0', 'eai:acl': {'app': 'misp42splunk', 'can_change_perms': '1', 'can_list': '1', 'can_share_app': '1', 'can_share_global': '1', 'can_share_user': '1', 'can_write': '1', 'modifiable': '1', 'owner': 'skalvaro', 'perms': {'read': ['*'], 'write': ['*']}, 'removable': '1', 'sharing': 'global'}, 'eai:appName': 'misp42splunk', 'eai:userName': 'nobody', 'misp_key': '********', 'misp_url': 'https://misp.skalvaro.lan', 'misp_use_proxy': '0', 'misp_verifycert': '0', 'type': 'text/xml'}}
[MC1000] instance set: {'title': 'misp2', 'id': 'https://127.0.0.1/servicesNS/nobody/misp42splunk/misp42splunk_account/misp2', 'updated': '1970-01-01T00:00:00+00:00', 'link': [{'href': '/servicesNS/nobody/misp42splunk/misp42splunk_account/misp2', 'rel': 'alternate'}, {'href': '/servicesNS/nobody/misp42splunk/misp42splunk_account/misp2', 'rel': 'list'}, {'href': '/servicesNS/nobody/misp42splunk/misp42splunk_account/misp2', 'rel': 'edit'}, {'href': '/servicesNS/nobody/misp42splunk/misp42splunk_account/misp2', 'rel': 'remove'}], 'author': {'name': 'skalvaro'}, 'content': {'client_use_cert': '0', 'disabled': '0', 'eai:acl': {'app': 'misp42splunk', 'can_change_perms': '1', 'can_list': '1', 'can_share_app': '1', 'can_share_global': '1', 'can_share_user': '1', 'can_write': '1', 'modifiable': '1', 'owner': 'skalvaro', 'perms': {'read': ['*'], 'write': ['*']}, 'removable': '1', 'sharing': 'global'}, 'eai:appName': 'misp42splunk', 'eai:userName': 'nobody', 'misp_key': '********', 'misp_url': 'https://misp.skalvaro.lan', 'misp_use_proxy': '0', 'misp_verifycert': '0', 'type': 'text/xml'}}
GET request to https://127.0.0.1:8089/servicesNS/nobody/search/storage/passwords/ (body: {'count': -1, 'offset': 0})
remg427 commented 3 years ago

Hi, Thank you very much for your trouble shooting and pointing the issue. I will take your code into misp_common.py Thanks again -- Sent with K-9 Mail.

remg427 commented 3 years ago

fixed in 4.0.1