remg427 / misp42splunk

A Splunk app to use MISP in background
GNU Lesser General Public License v3.0
109 stars 30 forks source link

Create events in MISP with splunk query #38

Closed melamedbn closed 5 years ago

melamedbn commented 5 years ago

Hey,

We're trying to create a MISP event based on Splunk Alert. When the alert is triggered multiple new empty events are created, with only the event tag and info. There is no documentation for Splunk query best practice/must have fields/fields naming etc'.

Can someone provide a working Splunk query that creates event, adds the attributes and publish the event? *The alert must apply the output to the same event_id.

Thank you, Ben

remg427 commented 5 years ago

index=sandbox_emails | eval eventkey=md5(src_user) | dedup eventkey | rename src_user AS eo_from | eval eo_subject=...message_subject... | eval eo_attachment=...malicious_attachment_name...l)) | eval misp_url=...malicious_url... | eval fo_filename=...original_name... | where isnotnull(fo_filename) OR isnotnull(misp_url) | eval fo_md5=a_md5sum | eval fo_sha1=a_sha1 | eval fo_sha256=a_sha256 | eval fo_sha512=a_sha512 | eval misp_domain=a_domain | eval misp_address=spath(_raw,"alert.explanation.cnc-services.cnc-service{}.address") | eval misp_hostname=a_hostname | eval misp_info=if(isnotnull(fo_filename),"malspam with attachment","malspam") | eval misp_tag=if(isnotnull(fo_filename),"misp-galaxy:mitre-enterprise-attack-attack-pattern=\"Spearphishing Attachment - T1193\"","misp-galaxy:mitre-enterprise-attack-attack-pattern=\"Spearphishing Link - T1192\"") | eval misp_time=round(_time,0) | table eventkey misp_time misp_info misp_tag eo_from eo_subject eo_attachment misp_url misp_domain misp_hostname fo_filename fo_md5 fo_sha1 fo_sha256 fo_sha512

melamedbn commented 5 years ago

Hey, Thank you for replying so quickly. I saw the query you provided. still Splunk Alert creates event for each trigger. I want to add attributes to existing event, is it possible?

remg427 commented 5 years ago

Yes alert is designed to create new events not to add to existing events from previous triggers. Eventkey is meant to group several rows of one trigger into one event still a new one by trigger. What I usually do is to manually review the new events and merge them as appropriate before publishing. What would be the criterion to select the existing event to add to?

melamedbn commented 5 years ago

Any option that will provide in order to achieve that will be appreciated. By field name/event id/Alert name etc.

We want to replace our ESB with your splunk-misp correlation, to do so we will need alerts running in real time that will add attributes to block list. Each alert designed to look after different assets of our network, so each alert will need to have linked event to push the results to.

remg427 commented 5 years ago

Hi, then providing event_id will be the easiest as this param is required by the endpoint /attributes/add/[event_id]. I am going to make an update of the alert to either add attributes to existing event (event_id provided) or to create event(s) if no event_id provided.

remg427 commented 5 years ago

in v2.0.14, misp_alert_create_event.py has a new parameter eventid (it accepts eventid, uuid). If provided, the corresponding existing event is updated instead of creating a new one. So you can update an event across several trigger. Hope it meets your expectation. Let's try and report any issue/improvement.

melamedbn commented 5 years ago

remg427, Thank you for the quick response and solution. We'll check and report in the next few hours regarding any issues/improvements.

melamedbn commented 5 years ago

issues - got a solution by reviewing the code :) improvements - option to add comment within the raw output of the alert ( misp_comment ).

Thank you my friend.

melamedbn commented 5 years ago

issues - got a solution by reviewing the code :) improvements - option to add comment within the raw output of the alert ( misp_comment ).

Thank you my friend.

fixed the comment issue. Thank you!

remg427 commented 5 years ago

could you share how you solved the issue/comment. Maybe worth adapting the code

ShimiCohen commented 5 years ago

def store_attribute(t, v, to_ids=None, category=None, comment=None): Attribute = {} Attribute['type'] = t Attribute['value'] = v if to_ids is not None: Attribute['to_ids'] = to_ids if category is not None: Attribute['category'] = category if comment is not None: Attribute['comment'] = comment return Attribute

def prepare_misp_events(config, results, event_list):

for row in results:

    if 'misp_comment' in row:
        comment = row.pop('misp_comment')
else:
    comment = None

    # collect attribute value and build type=value entry
    if 'misp_to_ids' in row:
        if str(row.pop('misp_to_ids')) == 'True':
            to_ids = True
        else:
            to_ids = False
    else:
        to_ids = None

    if 'misp_category' in row:
        category = str(row.pop('misp_category'))
    else:
        category = None

    # now we take KV pairs starting by misp_ to add to event as single attribute(s)
    for key, value in row.items():
        if key.startswith("misp_") and value != "":
            misp_key = str(key).replace('misp_', '').replace('_', '-')
            attributes.append(store_attribute(misp_key, str(value), to_ids, category, str(comment)))
remg427 commented 5 years ago

Hi, I have implemented your code for misp_comment

remg427 commented 5 years ago

Thanks for it