cudeso / misp2sentinel

MISP to Sentinel integration
MIT License
52 stars 17 forks source link

Error when processing data in event ... from MISP Invalid value for Note 'object_refs': referen ce to custom object type: x-misp-attribute #54

Closed lsuto closed 10 months ago

lsuto commented 11 months ago

I Received 0 indicators in MISP but I am getting the following errors when processing MISP events:

2023-08-12 10:05:28,311 - misp2sentinel - INFO - Received MISP events page 1 with 100 events 2023-08-12 10:05:28,459 - misp2sentinel - ERROR - Error when processing data in event 494 from MISP Invalid value for Note 'object_refs': referen ce to custom object type: x-misp-attribute 2023-08-12 10:05:28,466 - misp2sentinel - ERROR - Error when processing data in event 495 from MISP Invalid value for Note 'object_refs': referen ce to custom object type: x-misp-attribute 2023-08-12 10:05:28,484 - misp2sentinel - ERROR - Error when processing data in event 496 from MISP Invalid value for Note 'object_refs': referen ce to custom object type: x-misp-attribute 2023-08-12 10:05:28,603 - misp2sentinel - ERROR - Ignoring non STIX pattern type yara 2023-08-12 10:05:28,603 - misp2sentinel - ERROR - Unable to process indicator 2023-08-12 10:05:28,610 - misp2sentinel - ERROR - Ignoring non STIX pattern type yara 2023-08-12 10:05:28,610 - misp2sentinel - ERROR - Unable to process indicator 2023-08-12 10:05:28,610 - misp2sentinel - ERROR - Ignoring non STIX pattern type yara 2023-08-12 10:05:28,610 - misp2sentinel - ERROR - Unable to process indicator 2023-08-12 10:05:28,686 - misp2sentinel - ERROR - Error when processing data in event 503 from MISP Invalid value for Note 'object_refs': referen ce to custom object type: x-misp-attribute

misp filters in config.py:

misp_event_filters = { "published": 1,

"tags": [ "sentinel-threattype=\"Malware\""],

"enforceWarninglist": True,
"includeEventTags": True,
"publish_timestamp": "7d",

}

cudeso commented 11 months ago

Can you check with the latest commit?

The Ignoring non STIX pattern type yara is normal. Microsoft Sentinel does not accept yara indicators for uploading via Upload Indicators API, these attributes are ignored, this causes the ERROR - Unable to process indicator.

The Invalid value for Note 'object_refs': reference to custom object type: x-misp-attribute is caused by the conversion by MISP to STIX (this is done with the misp-stix package). I don't know why, but some EventReports seem to be causing problems for this conversion, and it's more a bug that needs to be raised in misp-stix. I added an option that removes the EventReports from the event, before conversion takes place. Currently Microsoft Sentinel does not support STIX Notes (to which reports get translated) so you don't lose any functionality anyway. The change is in https://github.com/cudeso/misp2sentinel/blob/main/RequestObject.py#L172 and the value "misp_remove_eventreports" is set to True by default when it's missing from the config file (https://github.com/cudeso/misp2sentinel/blob/main/script.py#L181)

lsuto commented 10 months ago

I will try shortly thanks!

On Sunday, August 13, 2023, Koen Van Impe @.***> wrote:

Can you check with the latest commit?

The Ignoring non STIX pattern type yara is normal. Microsoft Sentinel does not accept yara indicators for uploading via Upload Indicators API, these attributes are ignored, this causes the ERROR - Unable to process indicator.

The Invalid value for Note 'object_refs': reference to custom object type: x-misp-attribute is caused by the conversion by MISP to STIX (this is done with the misp-stix package). I don't know why, but some EventReports seem to be causing problems for this conversion, and it's more a bug that needs to be raised in misp-stix. I added an option that removes the EventReports from the event, before conversion takes place. Currently Microsoft Sentinel does not support STIX Notes (to which reports get translated) so you don't lose any functionality anyway. The change is in https://github.com/cudeso/misp2sentinel/blob/main/RequestObject.py#L172 and the value "misp_remove_eventreports" is set to True by default when it's missing from the config file (https://github.com/cudeso/ misp2sentinel/blob/main/script.py#L181)

— Reply to this email directly, view it on GitHub https://github.com/cudeso/misp2sentinel/issues/54#issuecomment-1676325639, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAA6TXBFD6MA2YR44AQFSF3XVCZ6VANCNFSM6AAAAAA3OLOPUQ . You are receiving this because you authored the thread.Message ID: @.***>

lsuto commented 10 months ago

Thanks works great!

lsuto commented 10 months ago

What would be the best way to print the contents of the event with the yara rule that is rejected? Thanks

On Monday, August 14, 2023, Koen Van Impe @.***> wrote:

Closed #54 https://github.com/cudeso/misp2sentinel/issues/54 as completed.

— Reply to this email directly, view it on GitHub https://github.com/cudeso/misp2sentinel/issues/54#event-10094140786, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAA6TXBROMUZ4O6B3CDRKUTXVMI75ANCNFSM6AAAAAA3OLOPUQ . You are receiving this because you authored the thread.Message ID: @.***>

cudeso commented 10 months ago

It's not printing the content of the event; but this change logs the event ID. This should already point you in the right direction. https://github.com/cudeso/misp2sentinel/commit/6f625de61ace076b4d8fafcfdd49dd74fcfc8d6a