The sync from my MISP instance to Microsoft Sentinel is failing.
This is the output from Application logs
I've changed PHP max_memory in the php.ini file and rebooted the server. But the same error still comes up.
This is the output from running the script.py:
└─# python3 script.py
fetching & parsing data from misp...
/root/anaconda3/envs/python3-venv/lib/python3.6/site-packages/urllib3/connectionpool.py:857: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecureRequestWarning)
WARNING [api.py:82 - init() ] The version of PyMISP recommended by the MISP instance (2.4.169) is newer than the one you're using now (2.4.121.1). Please upgrade PyMISP.
/root/anaconda3/envs/python3-venv/lib/python3.6/site-packages/urllib3/connectionpool.py:857: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecureRequestWarning)
/root/anaconda3/envs/python3-venv/lib/python3.6/site-packages/urllib3/connectionpool.py:857: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecureRequestWarning)
/root/anaconda3/envs/python3-venv/lib/python3.6/site-packages/urllib3/connectionpool.py:857: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecureRequestWarning)
/root/anaconda3/envs/python3-venv/lib/python3.6/site-packages/urllib3/connectionpool.py:857: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecureRequestWarning)
CRITICAL [api.py:2168 - _check_response() ] Unknown error: the response is not in JSON.
Something is broken server-side, please send us everything that follows (careful with the auth key):
Request headers:
{'User-Agent': 'PyMISP 2.4.121.1 - Python 3.6', 'Accept-Encoding': 'gzip, deflate', 'Accept': 'application/json', 'Connection': 'keep-alive', 'Content-Length': '292', 'Authorization': 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXX', 'content-type': 'application/json'}
Request body:
{"returnFormat": "json", "category": "", "org": "", "withAttachments": 0, "metadata": 0, "enforceWarninglist": 0, "includeEventUuid": 0, "includeEventTags": 0, "sgReferenceOnly": 0, "includeContext": 0, "headerless": 0, "includeSightings": 0, "includeDecayScore": 0, "includeCorrelations": 0}
Response (if any):
{"name":"An Internal Error Has Occurred.","message":"An Internal Error Has Occurred.","url":"\/events\/restSearch"}
Traceback (most recent call last):
File "script.py", line 100, in
main()
File "script.py", line 69, in main
events = _get_events()
File "script.py", line 18, in _get_events
[event['Event'] for event in misp.search(controller='events', return_format='json', **config.misp_event_filters)]
File "/root/anaconda3/envs/python3-venv/lib/python3.6/site-packages/pymisp/api.py", line 1524, in search
normalized_response = self._check_json_response(response)
File "/root/anaconda3/envs/python3-venv/lib/python3.6/site-packages/pymisp/api.py", line 2160, in _check_json_response
r = self._check_response(response, expect_json=True)
File "/root/anaconda3/envs/python3-venv/lib/python3.6/site-packages/pymisp/api.py", line 2169, in _check_response
raise MISPServerError(f'Error code 500:\n{response.text}')
pymisp.exceptions.MISPServerError: Error code 500:
{"name":"An Internal Error Has Occurred.","message":"An Internal Error Has Occurred.","url":"\/events\/restSearch"}
The strange things is MISP was able to do the initial sync. Not sure if it fully finished or not but within the day sync stopped and I don't know enough of Python to figure out what needs changing to get it back up and running.
Hello there,
The sync from my MISP instance to Microsoft Sentinel is failing. This is the output from Application logs
I've changed PHP max_memory in the php.ini file and rebooted the server. But the same error still comes up.
This is the output from running the script.py: └─# python3 script.py fetching & parsing data from misp... /root/anaconda3/envs/python3-venv/lib/python3.6/site-packages/urllib3/connectionpool.py:857: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) WARNING [api.py:82 - init() ] The version of PyMISP recommended by the MISP instance (2.4.169) is newer than the one you're using now (2.4.121.1). Please upgrade PyMISP. /root/anaconda3/envs/python3-venv/lib/python3.6/site-packages/urllib3/connectionpool.py:857: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) /root/anaconda3/envs/python3-venv/lib/python3.6/site-packages/urllib3/connectionpool.py:857: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) /root/anaconda3/envs/python3-venv/lib/python3.6/site-packages/urllib3/connectionpool.py:857: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) /root/anaconda3/envs/python3-venv/lib/python3.6/site-packages/urllib3/connectionpool.py:857: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) CRITICAL [api.py:2168 - _check_response() ] Unknown error: the response is not in JSON. Something is broken server-side, please send us everything that follows (careful with the auth key): Request headers: {'User-Agent': 'PyMISP 2.4.121.1 - Python 3.6', 'Accept-Encoding': 'gzip, deflate', 'Accept': 'application/json', 'Connection': 'keep-alive', 'Content-Length': '292', 'Authorization': 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXX', 'content-type': 'application/json'} Request body: {"returnFormat": "json", "category": "", "org": "", "withAttachments": 0, "metadata": 0, "enforceWarninglist": 0, "includeEventUuid": 0, "includeEventTags": 0, "sgReferenceOnly": 0, "includeContext": 0, "headerless": 0, "includeSightings": 0, "includeDecayScore": 0, "includeCorrelations": 0} Response (if any): {"name":"An Internal Error Has Occurred.","message":"An Internal Error Has Occurred.","url":"\/events\/restSearch"} Traceback (most recent call last): File "script.py", line 100, in
main()
File "script.py", line 69, in main
events = _get_events()
File "script.py", line 18, in _get_events
[event['Event'] for event in misp.search(controller='events', return_format='json', **config.misp_event_filters)]
File "/root/anaconda3/envs/python3-venv/lib/python3.6/site-packages/pymisp/api.py", line 1524, in search
normalized_response = self._check_json_response(response)
File "/root/anaconda3/envs/python3-venv/lib/python3.6/site-packages/pymisp/api.py", line 2160, in _check_json_response
r = self._check_response(response, expect_json=True)
File "/root/anaconda3/envs/python3-venv/lib/python3.6/site-packages/pymisp/api.py", line 2169, in _check_response
raise MISPServerError(f'Error code 500:\n{response.text}')
pymisp.exceptions.MISPServerError: Error code 500:
{"name":"An Internal Error Has Occurred.","message":"An Internal Error Has Occurred.","url":"\/events\/restSearch"}
The strange things is MISP was able to do the initial sync. Not sure if it fully finished or not but within the day sync stopped and I don't know enough of Python to figure out what needs changing to get it back up and running.
Any help would be appreciated.