cudeso / misp2sentinel

MISP to Sentinel integration
MIT License
53 stars 17 forks source link

Upload Indicators: Unknown error: the response is not in JSON. #45

Closed NickS-2022 closed 11 months ago

NickS-2022 commented 12 months ago

I've recently built a new Ubuntu VM (22.02) and trying to test the new Upload Indicators option. I'm getting the following error:

Unknown error: the response is not in JSON. Something is broken server-side, please send us everything that follows (careful with the auth key): Request headers: {'User-Agent': 'PyMISP 2.4.173 - Python 3.10', 'Accept-Encoding': 'gzip, deflate', 'Accept': 'application/json', 'Connection': 'keep-alive', 'Cookie': 'xxxxxxxxxxxxxx', 'Content-Length': '367', 'content-type': 'applic ation/json'} Request body: {"returnFormat": "stix2", "page": 5, "limit": 10, "withAttachments": 0, "metadata": 0, "publish_timestamp": "3d", "published": 1, "enforceWarninglist": 0, "to_ids": 1, "includeEventUuid": 0, "includeEventTags": 0, "sgReferenceOnly": 0, "includ eContext": 0, "headerless": 0, "includeSightings": 0, "includeDecayScore": 0, "includeCorrelations": 0, "excludeDecayed": 0} Response (if any): {"name":"An Internal Error Has Occurred.","message":"An Internal Error Has Occurred.","url":"\/events\/restSearch"} ----------------CLEAR existing_indicators_hash--------------------------- Traceback (most recent call last): File "/home/azureuser/misp-nda-cc/script.py", line 264, in main() File "/home/azureuser/misp-nda-cc/script.py", line 247, in main with RequestManager(total_indicators, logger) as request_manager: File "/home/azureuser/misp-nda-cc/RequestManager.py", line 51, in enter self.headers = {"Authorization": f"Bearer {access_token}", "user-agent": config.ms_useragent, "content-type": "application/json"} AttributeError: module 'config' has no attribute 'ms_useragent'

This is the log file showing that it is successfully pulling the indicators from MiSP (throttled to 10 per page), but not getting any further.

2023-07-25 13:02:49,707 - misp2sentinel - DEBUG - Received MISP events page 1 2023-07-25 13:02:49,707 - misp2sentinel - INFO - Processed 10 indicators. 2023-07-25 13:02:51,808 - misp2sentinel - DEBUG - Received MISP events page 2 2023-07-25 13:02:51,809 - misp2sentinel - INFO - Processed 20 indicators. 2023-07-25 13:02:53,889 - misp2sentinel - DEBUG - Received MISP events page 3 2023-07-25 13:02:53,889 - misp2sentinel - INFO - Processed 30 indicators. 2023-07-25 13:02:56,281 - misp2sentinel - DEBUG - Received MISP events page 4 2023-07-25 13:02:56,281 - misp2sentinel - INFO - Processed 35 indicators. 2023-07-25 13:02:57,434 - misp2sentinel - INFO - Finished receiving MISP events. 2023-07-25 13:02:57,434 - misp2sentinel - INFO - Found 35 indicators in MISP

Checking Azure AD I can see that the sign-in to the Azure AD application is successful.

It looks like RequestManager.py is expecting ms_useragent to be defined in config.py, but it isn't. I removed "user-agent": config.ms_useragent..and this fixed this issue, but I've not got the same SSLError - EOF as I had with 22.02.

cudeso commented 11 months ago

Hello. The line of code ----------------CLEAR existing_indicators_hash--------------------------- does not exist in the Upload Indicators branch > https://github.com/cudeso/misp2sentinel/blob/upload_indicators_api/RequestManager.py It does exist in the main branch though. I think you're branch is not set to "upload_indicators_api".

Check with git branch. Switch branches with git checkout upload_indicators_api Maybe same cause as for https://github.com/cudeso/misp2sentinel/issues/41 ?

NickS-2022 commented 11 months ago

Well spotted! That's what happens when you're trying to re-build a system in a hurry whilst doing ten other jobs.

I've switch to correct branch now and that's solved that issue. I've closed #41 as well, which was a different issue - albeit still at our side!