MISP42 Getting Proxy Error with Splunk 9.0

[brwnskndgirl]

I can't seem to figure out the following error with MISP42 once we upgraded to Splunk 9.0.0. It appears the Python library changed also.

07-27-2022 22:15:26.024 ERROR script [31825 phase_1] - SearchMessage orig_component=script sid=1658960004.369_AEB75738-63D0-4F91-879F-AD62D0A7A6EC message_key=EXTERN:SCRIPT_NONZERO_RETURN_%s%d_%s message=External search command 'mispgetioc' returned error code 1. Script output = "error_message=ProxyError at "/opt/splunk/lib/python3.7/site-packages/requests/adapters.py", line 512 : HTTPSConnectionPool(host='xxxxx.me.com', port=443): Max retries exceeded with url: /attributes/restSearch (Caused by ProxyError('Cannot connect to proxy.', ConnectionResetError(104, 'Connection reset by peer')))\r\n\r\n".

[timo92700]

Hello, After upgrading to splunk 9.0, I have the same kind of error :

_07-28-2022 12:35:28.821 ERROR script [1564718 phase_1] - SearchMessage orig_component=script sid=1659004527.1855_96CB9E0B-21F7-47EB-8C09-391F40BE0E16 message_key=EXTERN:SCRIPT_NONZERO_RETURN_%s%d_%s message=External search command 'mispgetioc' returned error code 1. Script output = "error_message=SSLError at "/opt/splunk/lib/python3.7/site-packages/requests/adapters.py", line 514 : HTTPSConnectionPool(host='xxxxxx', port=443): Max retries exceeded with url: /attributes/restSearch (Caused by SSLError(SSLError(1, '[SSL: UNKNOWN_PROTOCOL] unknown protocol (ssl.c:1106)')))\r\n\r\n".

Notice that when running the query on the same search head using the curl command in the terminal, as the below command, it works :

_curl -k -X POST "https://xxxxxxxx/attributes/restSearch" -H "Content-type:application/json" -H "Authorization:" -H "Accept:application/json" -x http://xxxxx:8080 --connect-timeout 300_

[firm-0ne]

Same as above having issues with MISP42 command mispgetioc after upgrading to Splunk v9.0 (was v8.2.4).

07-29-2022 15:13:31.050 INFO PhaseNodeGenerationVisitor [15036 searchOrchestrator] - FallBackReason: Fallback to 2-phase mode because of empty split key of cmd: mispgetioc . . . 07-29-2022 15:13:31.348 ERROR ScriptRunner [15057 phase_1] - stderr from '/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/misp42splunk/bin/mispgetioc.py EXECUTE misp_instance=CSO_misp_test2 last=2d limit=1000': [IO-101] logging level is set to DEBUG 07-29-2022 15:13:31.348 ERROR ScriptRunner [15057 phase_1] - stderr from '/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/misp42splunk/bin/mispgetioc.py EXECUTE misp_instance=CSO_misp_test2 last=2d limit=1000': [IO-102] PYTHON VERSION: 3.7.11 (default, May 25 2022, 12:23:55) 07-29-2022 15:13:31.348 ERROR ScriptRunner [15057 phase_1] - stderr from '/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/misp42splunk/bin/mispgetioc.py EXECUTE misp_instance=CSO_misp_test2 last=2d limit=1000': [GCC 9.1.0] 07-29-2022 15:13:31.348 ERROR ScriptRunner [15057 phase_1] - stderr from '/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/misp42splunk/bin/mispgetioc.py EXECUTE misp_instance=CSO_misp_test2 last=2d limit=1000': GET request to https://127.0.0.1:8089/servicesNS/nobody/misp42splunk/misp42splunk_instances (body: {}) 07-29-2022 15:13:31.967 ERROR ScriptRunner [15057 phase_1] - stderr from '/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/misp42splunk/bin/mispgetioc.py EXECUTE misp_instance=CSO_misp_test2 last=2d limit=1000': Operation took 0:00:00.618513 07-29-2022 15:13:31.967 ERROR ScriptRunner [15057 phase_1] - stderr from '/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/misp42splunk/bin/mispgetioc.py EXECUTE misp_instance=CSO_misp_test2 last=2d limit=1000': [MC-PC-D01] response.status=200 07-29-2022 15:13:31.968 ERROR ScriptRunner [15057 phase_1] - stderr from '/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/misp42splunk/bin/mispgetioc.py EXECUTE misp_instance=CSO_misp_test2 last=2d limit=1000': [MC-PC-D02] instance_count=2 07-29-2022 15:13:31.968 ERROR ScriptRunner [15057 phase_1] - stderr from '/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/misp42splunk/bin/mispgetioc.py EXECUTE misp_instance=CSO_misp_test2 last=2d limit=1000': [MC-PC-D04] instance item={'title': 'CSO_misp_test', 'id': 'https://127.0.0.1/servicesNS/nobody/misp42splunk/misp42splunk_instances/CSO_misp_test', 'updated': '1969-12-31T16:00:00-08:00', 'link': [{'href': '/servicesNS/nobody/misp42splunk/misp42splunk_instances/CSO_misp_test', 'rel': 'alternate'}, {'href': '/servicesNS/nobody/misp42splunk/misp42splunk_instances/CSO_misp_test', 'rel': 'list'}, {'href': '/servicesNS/nobody/misp42splunk/misp42splunk_instances/CSO_misp_test', 'rel': 'edit'}, {'href': '/servicesNS/nobody/misp42splunk/misp42splunk_instances/CSO_misp_test', 'rel': 'remove'}], 'author': {'name': 'REDACTED@acme.com'}, 'content': {'disabled': '0', 'eai:acl': {'app': 'misp42splunk', 'can_change_perms': '1', 'can_list': '1', 'can_share_app': '1', 'can_share_global': '1', 'can_share_user': '1', 'can_write': '1', 'modifiable': '1', 'owner': 'REDACTED@acme.com', 'perms': {'read': ['*'], 'write': ['admin']}, 'removable': '1', 'sharing': 'global'}, 'eai:appName': 'misp42splunk', 'eai:userName': 'nobody', 'misp_key': '**', 'misp_url': 'https://OUR.SERVER.REDACTED.acme.com', 'misp_use_proxy': '1', 'misp_verifycert': '0', 'type': 'text/xml'}} 07-29-2022 15:13:31.968 ERROR ScriptRunner [15057 phase_1] - stderr from '/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/misp42splunk/bin/mispgetioc.py EXECUTE misp_instance=CSO_misp_test2 last=2d limit=1000': [MC-PC-D04] instance item={'title': 'CSO_misp_test2', 'id': 'https://127.0.0.1/servicesNS/nobody/misp42splunk/misp42splunk_instances/CSO_misp_test2', 'updated': '1969-12-31T16:00:00-08:00', 'link': [{'href': '/servicesNS/nobody/misp42splunk/misp42splunk_instances/CSO_misp_test2', 'rel': 'alternate'}, {'href': '/servicesNS/nobody/misp42splunk/misp42splunk_instances/CSO_misp_test2', 'rel': 'list'}, {'href': '/servicesNS/nobody/misp42splunk/misp42splunk_instances/CSO_misp_test2', 'rel': 'edit'}, {'href': '/servicesNS/nobody/misp42splunk/misp42splunk_instances/CSO_misp_test2', 'rel': 'remove'}], 'author': {'name': 'REDACTED@acme.com'}, 'content': {'disabled': '0', 'eai:acl': {'app': 'misp42splunk', 'can_change_perms': '1', 'can_list': '1', 'can_share_app': '1', 'can_share_global': '1', 'can_share_user': '1', 'can_write': '1', 'modifiable': '1', 'owner': 'REDACTED@acme.com', 'perms': {'read': ['*'], 'write': ['admin']}, 'removable': '1', 'sharing': 'global'}, 'eai:appName': 'misp42splunk', 'eai:userName': 'nobody', 'misp_key': '**', 'misp_url': 'https://OUR.SERVER.REDACTED.acme.com', 'misp_use_proxy': '1', 'misp_verifycert': '0', 'type': 'text/xml'}} 07-29-2022 15:13:31.968 ERROR ScriptRunner [15057 phase_1] - stderr from '/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/misp42splunk/bin/mispgetioc.py EXECUTE misp_instance=CSO_misp_test2 last=2d limit=1000': GET request to https://127.0.0.1:8089/servicesNS/nobody/misp42splunk/storage/passwords/ (body: {'count': -1, 'offset': 0}) 07-29-2022 15:13:31.976 ERROR ScriptRunner [15057 phase_1] - stderr from '/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/misp42splunk/bin/mispgetioc.py EXECUTE misp_instance=CSO_misp_test2 last=2d limit=1000': Operation took 0:00:00.008125 07-29-2022 15:13:31.978 ERROR ScriptRunner [15057 phase_1] - stderr from '/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/misp42splunk/bin/mispgetioc.py EXECUTE misp_instance=CSO_misp_test2 last=2d limit=1000': Option "last" set with 2d 07-29-2022 15:13:31.980 ERROR ScriptRunner [15057 phase_1] - stderr from '/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/misp42splunk/bin/mispgetioc.py EXECUTE misp_instance=CSO_misp_test2 last=2d limit=1000': Starting new HTTPS connection (1):OUR.SERVER.REDACTED.acme.com:443 07-29-2022 15:13:32.213 INFO ReducePhaseExecutor [15050 StatusEnforcerThread] - ReducePhaseExecutor=1 action=PREVIEW 07-29-2022 15:15:34.856 ERROR ScriptRunner [15057 phase_1] - stderr from '/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/misp42splunk/bin/mispgetioc.py EXECUTE misp_instance=CSO_misp_test2 last=2d limit=1000': ProxyError at "/opt/splunk/lib/python3.7/site-packages/requests/adapters.py", line 510 : HTTPSConnectionPool(host='OUR.SERVER.REDACTED.acme.com', port=443): Max retries exceeded with url: /attributes/restSearch (Caused by ProxyError('Cannot connect to proxy.', ConnectionResetError(104, 'Connection reset by peer'))) 07-29-2022 15:15:34.856 ERROR ScriptRunner [15057 phase_1] - stderr from '/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/misp42splunk/bin/mispgetioc.py EXECUTE misp_instance=CSO_misp_test2 last=2d limit=1000': Traceback: 07-29-2022 15:15:34.856 ERROR ScriptRunner [15057 phase_1] - stderr from '/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/misp42splunk/bin/mispgetioc.py EXECUTE misp_instance=CSO_misp_test2 last=2d limit=1000': File "/opt/splunk/etc/apps/misp42splunk/lib/splunklib/searchcommands/search_command.py", line 619, in _process_protocol_v1 07-29-2022 15:15:34.856 ERROR ScriptRunner [15057 phase_1] - stderr from '/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/misp42splunk/bin/mispgetioc.py EXECUTE misp_instance=CSO_misp_test2 last=2d limit=1000': self._execute(ifile, None) 07-29-2022 15:15:34.856 ERROR ScriptRunner [15057 phase_1] - stderr from '/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/misp42splunk/bin/mispgetioc.py EXECUTE misp_instance=CSO_misp_test2 last=2d limit=1000': File "/opt/splunk/etc/apps/misp42splunk/lib/splunklib/searchcommands/generating_command.py", line 211, in _execute 07-29-2022 15:15:34.856 ERROR ScriptRunner [15057 phase_1] - stderr from '/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/misp42splunk/bin/mispgetioc.py EXECUTE misp_instance=CSO_misp_test2 last=2d limit=1000': self._record_writer.write_records(self.generate()) 07-29-2022 15:15:34.856 ERROR ScriptRunner [15057 phase_1] - stderr from '/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/misp42splunk/bin/mispgetioc.py EXECUTE misp_instance=CSO_misp_test2 last=2d limit=1000': File "/opt/splunk/etc/apps/misp42splunk/lib/splunklib/searchcommands/internals.py", line 576, in write_records 07-29-2022 15:15:34.856 ERROR ScriptRunner [15057 phase_1] - stderr from '/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/misp42splunk/bin/mispgetioc.py EXECUTE misp_instance=CSO_misp_test2 last=2d limit=1000': records = list(records) 07-29-2022 15:15:34.856 ERROR ScriptRunner [15057 phase_1] - stderr from '/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/misp42splunk/bin/mispgetioc.py EXECUTE misp_instance=CSO_misp_test2 last=2d limit=1000': File "/opt/splunk/etc/apps/misp42splunk/bin/mispgetioc.py", line 450, in generate 07-29-2022 15:15:34.856 ERROR ScriptRunner [15057 phase_1] - stderr from '/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/misp42splunk/bin/mispgetioc.py EXECUTE misp_instance=CSO_misp_test2 last=2d limit=1000': proxies=my_args['proxies']) 07-29-2022 15:15:34.856 ERROR ScriptRunner [15057 phase_1] - stderr from '/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/misp42splunk/bin/mispgetioc.py EXECUTE misp_instance=CSO_misp_test2 last=2d limit=1000': File "/opt/splunk/lib/python3.7/site-packages/requests/api.py", line 119, in post 07-29-2022 15:15:34.856 ERROR ScriptRunner [15057 phase_1] - stderr from '/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/misp42splunk/bin/mispgetioc.py EXECUTE misp_instance=CSO_misp_test2 last=2d limit=1000': return request('post', url, data=data, json=json, kwargs) 07-29-2022 15:15:34.856 ERROR ScriptRunner [15057 phase_1] - stderr from '/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/misp42splunk/bin/mispgetioc.py EXECUTE misp_instance=CSO_misp_test2 last=2d limit=1000': File "/opt/splunk/lib/python3.7/site-packages/requests/api.py", line 61, in request 07-29-2022 15:15:34.856 ERROR ScriptRunner [15057 phase_1] - stderr from '/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/misp42splunk/bin/mispgetioc.py EXECUTE misp_instance=CSO_misp_test2 last=2d limit=1000': return session.request(method=method, url=url, kwargs) 07-29-2022 15:15:34.856 ERROR ScriptRunner [15057 phase_1] - stderr from '/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/misp42splunk/bin/mispgetioc.py EXECUTE misp_instance=CSO_misp_test2 last=2d limit=1000': File "/opt/splunk/lib/python3.7/site-packages/requests/sessions.py", line 542, in request 07-29-2022 15:15:34.856 ERROR ScriptRunner [15057 phase_1] - stderr from '/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/misp42splunk/bin/mispgetioc.py EXECUTE misp_instance=CSO_misp_test2 last=2d limit=1000': resp = self.send(prep, send_kwargs) 07-29-2022 15:15:34.856 ERROR ScriptRunner [15057 phase_1] - stderr from '/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/misp42splunk/bin/mispgetioc.py EXECUTE misp_instance=CSO_misp_test2 last=2d limit=1000': File "/opt/splunk/lib/python3.7/site-packages/requests/sessions.py", line 655, in send 07-29-2022 15:15:34.856 ERROR ScriptRunner [15057 phase_1] - stderr from '/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/misp42splunk/bin/mispgetioc.py EXECUTE misp_instance=CSO_misp_test2 last=2d limit=1000': r = adapter.send(request, kwargs) 07-29-2022 15:15:34.856 ERROR ScriptRunner [15057 phase_1] - stderr from '/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/misp42splunk/bin/mispgetioc.py EXECUTE misp_instance=CSO_misp_test2 last=2d limit=1000': File "/opt/splunk/lib/python3.7/site-packages/requests/adapters.py", line 510, in send 07-29-2022 15:15:34.856 ERROR ScriptRunner [15057 phase_1] - stderr from '/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/misp42splunk/bin/mispgetioc.py EXECUTE misp_instance=CSO_misp_test2 last=2d limit=1000': raise ProxyError(e, request=request) 07-29-2022 15:15:34.887 ERROR script [15057 phase_1] - SearchMessage orig_component=script sid=1659107610.2342_3AEC3D0E-E3C5-47A2-945F-AD3A6E99B633 message_key=EXTERN:SCRIPT_NONZERO_RETURN_%s%d_%s message=External search command 'mispgetioc' returned error code 1. Script output = "error_message=ProxyError at "/opt/splunk/lib/python3.7/site-packages/requests/adapters.py", line 510 : HTTPSConnectionPool(host='OUR.SERVER.REDACTED.acme.com', port=443): Max retries exceeded with url: /attributes/restSearch (Caused by ProxyError('Cannot connect to proxy.', ConnectionResetError(104, 'Connection reset by peer')))\r\n\r\n".

[brwnskndgirl]

I fixed this issue by adding the following line to mispgetioc.py. The error coming from adapters.py will confuse you. When editing the adapters.py I found the error to actually be coming from the mispgetioc.py script. This is the second python script where I had to hard-code the proxy for Splunk. It appears with the latest python urlib3 code disallows proxy defined in MISP Splunk not being acknowledged and passed downstream.

proxies = {http: 'proxy.com:xxxx', https: 'proxy.com:xxxx'}

[remg427]

I cannot fix that issue. @brwnskndgirl I tried your patch but then there is an error message and the request fails. researching on Internet links this bug to a version of lib request

[timo92700]

Hello, Well this does not appears to come from python request module. I developed this application that permit to create curl request using the search bar : https://splunkbase.splunk.com/app/5667/

Once installed ( on splunk9 ), by providing the correct parameters, for example, tried with this :

| curl url="https:///events/1" method=post headers="{'Authorization':'','Content-type': 'application/json','Accept':'application/json'}" proxies="http://,<YOUR_HTTPS_PROXY" timeout=10 output=json | spath input=Event

It works, this is only a workaround, but at least it is usable. I did not investigated the code of the misp application, but as it is using the requests module ( maybe the code uses requests module part that became deprecated / deleted after the splunk9/python upgrade ? ), it should work in some way.

Feel free to modify the endpoint / parameters of the request to match your needs, then add a | collect and schedule the search, until the official app is upgraded. Regards,

[remg427]

Hi, it is still related as switch to urllib3 helped to fix the requests using proxy and self-signed certificates. version 4.2.0 works on my side.

[timo92700]

Hi, Just tested 4.2.0, also works on my side now. Well done !

[remg427]

Hi Thank you so much for feedback I will publish soon on splunkbase Remi

Le 3 octobre 2022 11:40:43 GMT+02:00, timo92700 @.***> a écrit :

Hi, Just tested 4.2.0, also works on my side now. Well done !

-- Reply to this email directly or view it on GitHub: https://github.com/remg427/misp42splunk/issues/216#issuecomment-1265187754 You are receiving this because you were assigned.

Message ID: @.***>

-- Sent with K-9 Mail.