search

LetMeR00t / TA-thehive-cortex

Technical add-on for Splunk related to TheHive/Cortex from TheHive project
GNU Lesser General Public License v3.0
47 stars 11 forks source link

[BUG] Unexpected Error: 'Unknown' #73

Closed kvaratop closed 9 months ago

kvaratop commented 10 months ago

Request Type

Bug

Work Environment

Question Answer
OS version (server) Ubuntu 22
TheHive version / git hash 5.2.5-1

Problem Description

I don't see new cases created in TheHive platform due to unexpected errors. Please help to debug deeper.

Steps to Reproduce

  1. Create Notable
  2. Run Adaptive Response Action automatically or manually
  3. Get an error

Possible solutions

Logs (issued from the search.log with logging mode set to DEBUG under Settings/Configuration)

image

Also interesting why raw event is clipped in the log event.

LetMeR00t commented 10 months ago

Hi @AlexeyGlu would it be possible to get an example of the raw event you are using ? If you can provide a CSV sanitized to help me reproduce your error as this error is showing that a part of my code isn’t handled as expected ;)

LetMeR00t commented 10 months ago

Oh sorry I didn’t get that it was related to the severity field Why don’t you provide a valid value for this field ? If this field is provided in the notable itself, it’s probably the original rule that is generating the notable that doesn’t specify the severity of your detection rule

kvaratop commented 10 months ago

Oh sorry I didn’t get that it was related to the severity field Why don’t you provide a valid value for this field ? If this field is provided in the notable itself, it’s probably the original rule that is generating the notable that doesn’t specify the severity of your detection rule

it's a bug in the system I can not solve for now with eval field inside the DM.

Hi @AlexeyGlu would it be possible to get an example of the raw event you are using ? If you can provide a CSV sanitized to help me reproduce your error as this error is showing that a part of my code isn’t handled as expected ;)

I've created a notable with NOT unknown severity and yeah - case has been created! But it's strange behaviour, what do you think? I'll provide raw notable and you can check it

LetMeR00t commented 10 months ago

Hi I’m not surprised because the severity has a list of available values expected so not having one of them is an issue

When you say system you mean Splunk itself ?

I can provide a fix for a next release to set Medium by default if the value isn’t provided or incorrect

kvaratop commented 10 months ago

Hi @LetMeR00t After testing I see If severity equals something from predefined values like "high", "medium", the case is created normally. But I am facing with another errors - I don't know if should I create another issue or continue writing here. Sorry if I was wrong.

I want to limit group of fields in notable which will be transferred to case. So I've described them in thehive_datatypes.csv and choose an appropriate option whithin the adaptive response action. Case is created but

the problems are:

Tested data:

thehive_datatypes.csv: datatype description field_name field_type
autonomous-system default hive field autonomous-system observable
domain default hive field domain observable
fadn default hive field fadn observable
file default hive field file observable
filename default hive field filename observable
hash default hive field hash observable
hostname default hive field hostname observable
ip default hive field ip observable
mail default hive field mail observable
mail-subject default hive field mail-subject observable
other default hive field other observable
regexp default hive field regexp observable
registry default hive field registry observable
uri_path default hive field uri_path observable
url default hive field url observable
other EDR correlation search field category observable
other EDR correlation search field cim_entity_zone observable
other EDR correlation search field dest observable
other EDR correlation search field dest_bunit observable
other EDR correlation search field dest_category observable
other EDR correlation search field dest_owner observable
other EDR correlation search field external_local_ip observable
other EDR correlation search field file_disposition observable
other EDR correlation search field file_name observable
other EDR correlation search field file_parent_disposition observable
other EDR correlation search field file_path observable
other EDR correlation search field host_link observable
other EDR correlation search field local_ip observable
other EDR correlation search field mac observable
other EDR correlation search field original_file_name observable
other EDR correlation search field parent_file_hash observable
other EDR correlation search field parent_process_hash observable
other EDR correlation search field parent_process_id observable
other EDR correlation search field parent_process_name observable
other EDR correlation search field process_exec observable
other EDR correlation search field process_hash observable
other EDR correlation search field severity observable
other EDR correlation search field signature observable
other EDR correlation search field user observable
other EDR correlation search field action observable
other EDR correlation search field file_hash observable
other EDR correlation search field user observable
other EDR correlation search field count observable
other EDR correlation search field src_bunit observable
other EDR correlation search field src_category observable
other EDR correlation search field urgency observable
other EDR correlation search field vendor_product observable
other EDR correlation search field grand_parent_file_path observable
other EDR correlation search field grand_parent_file_name observable
other EDR correlation search field first_time observable
other EDR correlation search field ioc_triggered_type observable
other EDR correlation search field ioc_triggered_value observable
other EDR correlation search field last_time observable
other EDR correlation search field local_port observable
other EDR correlation search field parent_process observable
other EDR correlation search field parent_process_exec observable
other EDR correlation search field parent_process_path observable
other EDR correlation search field process observable
other EDR correlation search field process_id observable
other EDR correlation search field process_name observable
other EDR correlation search field process_path observable
other EDR correlation search field protocol observable
other EDR correlation search field remote_ip observable
other EDR correlation search field remote_port observable

raw notable event: 1699858800, search_name="Threat - CrowdStrike Malware Detected test - Rule", orig_time="1699858800", action="blocked", category="NGAV", cim_entity_zone="XXXX", count="1", dest="XXXX", file_hash="XXXX", file_name="XXXX", file_path="unknown", info_max_time="1699912800.000000000", info_min_time="1699826400.000000000", info_search_time="1700030101.606154000", severity="high", signature="This file meets the machine learning-based on-sensor AV protection's high confidence threshold for malicious files.", urgency="Informational", user="XXXX", vendor_product="CrowdStrike Falcon"

Expected results:

LetMeR00t commented 10 months ago

Hi @AlexeyGlu , Which version of the app are you running currently ? You have to know that in the latest versions, the thehive_datatypes.csv file is filled automatically thanks to an API call directly to TheHive to get all the list of fields available.

kvaratop commented 10 months ago

Hi @AlexeyGlu , Which version of the app are you running currently ? You have to know that in the latest versions, the thehive_datatypes.csv file is filled automatically thanks to an API call directly to TheHive to get all the list of fields available.

hm I missed it. Today I have updated the TA to the latest (3.0.1) version. I created the lookup file manually

LetMeR00t commented 10 months ago

Hi @AlexeyGlu , Which version of the app are you running currently ? You have to know that in the latest versions, the thehive_datatypes.csv file is filled automatically thanks to an API call directly to TheHive to get all the list of fields available.

hm I missed it. Today I have updated the TA to the latest (3.0.1) version. I created the lookup file manually

Do you mean v3.1?

Normally, the v3.1 is filling automatically the lookup, do you still see the issue ?

kvaratop commented 10 months ago

Hi @AlexeyGlu , Which version of the app are you running currently ? You have to know that in the latest versions, the thehive_datatypes.csv file is filled automatically thanks to an API call directly to TheHive to get all the list of fields available.

hm I missed it. Today I have updated the TA to the latest (3.0.1) version. I created the lookup file manually

Do you mean v3.1?

Normally, the v3.1 is filling automatically the lookup, do you still see the issue? yeah, sorry for my mistake.

so, have I to remove my manually created lookup. Because I didn't see that it was created automatically after TA upgrade

LetMeR00t commented 10 months ago

Yes, you can delete the lookup and then rerun the custom alert action, it should work

kvaratop commented 10 months ago

Yes, you can delete the lookup and then rerun the custom alert action, it should work

hi @LetMeR00t ! Yeah, you are right, I've deleted csv file, execute ES Repsonse action an lookup has created automatically, but I want to transfer only chosen fields as observables. So, should I add them manually to lookup as I tried before?

After the renewal of the datatype file

image

I've tried to test case creating via ES adaptive response action:

Debug info point me out to field _serial but I do not have such field in my raw notable event.

image

An example of the notable source you can check in the comments above

raw notable event: 1699858800, search_name="Threat - CrowdStrike Malware Detected test - Rule", orig_time="1699858800", action="blocked", category="NGAV", cim_entity_zone="XXXX", count="1", dest="XXXX", file_hash="XXXX", file_name="XXXX", file_path="unknown", info_max_time="1699912800.000000000", info_min_time="1699826400.000000000", info_search_time="1700030101.606154000", severity="high", signature="This file meets the machine learning-based on-sensor AV protection's high confidence threshold for malicious files.", urgency="Informational", user="XXXX", vendor_product="CrowdStrike Falcon"

kvaratop commented 10 months ago

telegram-cloud-photo-size-2-5386563376815133990-y

LetMeR00t commented 10 months ago

Hi @kvaratop,

For your first case with "fields only included in datatypes csv", I wasn't able to reproduce your error, on my side everything is created as expected. My only comment would be that within your screenshot, you have a cim_entity_zone custom fields which is wrongly named as within TheHive, custom fields aren't using underscores but dashes so the accurate name of your custom field should be cim-entity-zone instead. Anyway, even with this, I didn't have your error. Could you provide more information about the execution of this ? (What might help are two debug events happening named [CAA-THC-61] Row before pre-processing[...] and [CAA-THC-62] Row after pre-processing[..]

For your second case with "all fields as other", I successfully reproduced the error. It seems that my script wasn't handling well when we try to create an observable that is already existing (which is the case as we might have for instance the fields "sourcetype" and "_sourcetype" with the same value named "stash" so the script is raising an error. I'll deliver a fix for the next release to manage that better in the script and allow the rest of the observables to be created. In the meantime, it means that using directly the adaptive response from the "Incident Review" dashboard isn't working well.

LetMeR00t commented 10 months ago

FYI, and if you want to update the code directly on your side, you can change this part of the code situated here: https://github.com/LetMeR00t/TA-thehive-cortex/blob/521d5b168e95b25a35d8504329e988dfb62e1f80/TA-thehive-cortex/bin/ta_thehive_cortex/modalert_thehive_create_a_new_case_helper.py#L209-L226

with:

        # Processing Observables if any
        if "observables" in cases[srcRef]:
            for observable in cases[srcRef]['observables']:
                response = thehive.observable.create_in_case(case_id=new_case["_id"],observable=observable)

                if "failure" in response:
                    # somehow we got a bad response code from thehive
                    helper.log_error(
                        "[CAA-THCC-135-ERROR] TheHive observable update on recent case creation has failed. "
                        "url={}, data={}, content={}, observable={}, error={}"
                        .format(thehive.session.hive_url, str(case), str(response), str(observable), str(response["failure"]))
                    ) 
                else:
                    response = response[0]
                    # log response status
                    helper.log_info(
                        "[CAA-THCC-130] TheHive case {} was successfully updated with the observable {} on url={}".format(new_case["_id"],response["data"].replace(".","[.]"),thehive.session.hive_url)
                    )
kvaratop commented 10 months ago

Hi @LetMeR00t I have been trying to send alerts directly via | sendalert command with case_mode="alert" with a different set of the fields and their values. Also, I have chosen different scopes: 0,1.

General conclusion: I always get an error Unexpected error 'X', where 'X' can be fieldname or fieldvalue (typically = 0) from the row. For example, I have got the error: Unexpected error: 'severity' while the field severity had a value "high" in the row. I do not have any '0' values within my fields in the event I am using for test purposes. There are only Splunk metadata fields with '0' values like _serial,rid,...

I have started by tabling only 1 field - the case has been created successfully. Nice to point out that I have not got any errors reffered to '0' values

After 3 more fields were tabled before | sendalert- Unexpected error was raised again.

I also can not see the whole event of the logs such as:[CAA-THC-62] Row after pre-processing[..], [CAA-THC-61] Row before pre-processing[..]

LetMeR00t commented 10 months ago

Hello, Without any data nor screenshots, I can't help you with your issues. CAA-THC-61/62 are events happening in the Audit Logs dashboard when you have the logging set to DEBUG in your configuration, it's mandatory that it's happening otherwise it means that you aren't using the python scripts. If you can send me a step by step with data you are doing by email, it could be useful to help you.

kvaratop commented 10 months ago

Hello, Without any data nor screenshots, I can't help you with your issues. CAA-THC-61/62 are events happening in the Audit Logs dashboard when you have the logging set to DEBUG in your configuration, it's mandatory that it's happening otherwise it means that you aren't using the python scripts. If you can send me a step by step with data you are doing by email, it could be useful to help you.

I've sent you an email with detailed info

kvaratop commented 10 months ago

@LetMeR00t excuse me, have you seen the e-mail with details?

LetMeR00t commented 10 months ago

Hi @kvaratop Yes sorry, I’m working on my personal time on this and right now isn’t a good time for me I planned to work on it around Christmas I’ll keep you posted

LetMeR00t commented 9 months ago

Hi @kvaratop , Sorry for the delay. I've checked your provided notes (thank you for the details). Your first issue regarding the severity might be solved by a refinement of my code following this part, see https://github.com/LetMeR00t/TA-thehive-cortex/issues/74 and the commit provided as a fix (code+doc). For your second issue, can you try this search and tell me if you have also the same issue with the same context as you provided to send the alert ? (I can't reproduce the error when I though that test1 and test2 will raise the issue in my example

| makeresults
| eval ip="5.6.7.8", ttps="reconnaissance::T1595.001::"+tostring(strftime(now(),"%Y-%m-%d")), unique=tostring(now())+".1"
| table ip, ttp, unique
| append
    [| makeresults
| eval ip="1.2.3.4", ttps="reconnaissance::T1593.002::"+tostring(strftime(now(),"%Y-%m-%d")), unique=tostring(now())+".1"
| table ip, ttp, unique]
| append
    [| makeresults
| eval ip="1.3.5.8", ttps="reconnaissance::T1593.002::"+tostring(strftime(now(),"%Y-%m-%d")), unique=tostring(now())+".2"
| table ip, ttp, unique]
| eventstats values(ip) as all_ip by unique
| eval all_ip = mvjoin(all_ip,", ") 
| eval name = "Massive scan on website company.corp from "+all_ip, cert-alerted-on=now(), risk="Medium", info="A custom info added to the event", severity="high", th_severity=4, th_tlp="AMBER+STRICT", th_pap="2", test1="A", test2="A"
| fields - all_ip
kvaratop commented 9 months ago

Hi! @LetMeR00t Here is my flow:

  1. Just copied your search and add | sendalertcommand| makeresults | eval ip="5.6.7.8", ttps="reconnaissance::T1595.001::"+tostring(strftime(now(),"%Y-%m-%d")), unique=tostring(now())+".1" | table ip, ttp, unique | append [| makeresults | eval ip="1.2.3.4", ttps="reconnaissance::T1593.002::"+tostring(strftime(now(),"%Y-%m-%d")), unique=tostring(now())+".1" | table ip, ttp, unique] | append [| makeresults | eval ip="1.3.5.8", ttps="reconnaissance::T1593.002::"+tostring(strftime(now(),"%Y-%m-%d")), unique=tostring(now())+".2" | table ip, ttp, unique] | eventstats values(ip) as all_ip by unique | eval all_ip = mvjoin(all_ip,", ") | eval name = "Massive scan on website company.corp from "+all_ip, cert-alerted-on=now(), risk="Medium", info="A custom info added to the event", severity="high", th_severity=4, th_tlp="AMBER+STRICT", th_pap="2", test1="A", test2="A" | fields - all_ip | sendalert thehive_create_a_new_case param.thehive_instance_id=3ce5b522 param.case_mode="alert" param.source="splunk" param.title="$result.name$" param.description="$result.info$" param.tags="test;manual;bug" param.scope=1 param.severity="$result.th_severity$" param.pap="$result.th_pap$" Case hasn't been created

    image

    By the way, what amount of cases should be created in such a scenario?

  2. delete severity field from table | fields - all_ip severity case has been created without any errors

    image

  3. updated TA to the latest version 3.2. And repeatd the | sendmodalert command (the same result with\without severity field)

    image
LetMeR00t commented 9 months ago

Hello @kvaratop ,

I tried your search on my side. It doesn’t work neither as you didn’t provide a mandatory field named “unique”. After a check, it seems that this is coming from the usage of the “scope” field set to 1. I need to review the usage of the "Scope" parameter but right now could you make a change directly in the Python code by removing this line:

https://github.com/LetMeR00t/TA-thehive-cortex/blob/55d60ea63abd4e82dff3e3e1cbc8d64ad0816691/TA-thehive-cortex/bin/ta_thehive_cortex/modalert_thehive_common.py#L417-L417

And rerun your test ? It shall work.

LetMeR00t commented 9 months ago

Okay so I dig into your search.

| makeresults | eval ip="5.6.7.8", ttps="reconnaissance::T1595.001::"+tostring(strftime(now(),"%Y-%m-%d")), unique=tostring(now())+".1" | table ip, ttp, unique | append [| makeresults | eval ip="1.2.3.4", ttps="reconnaissance::T1593.002::"+tostring(strftime(now(),"%Y-%m-%d")), unique=tostring(now())+".1" | table ip, ttp, unique] | append [| makeresults | eval ip="1.3.5.8", ttps="reconnaissance::T1593.002::"+tostring(strftime(now(),"%Y-%m-%d")), unique=tostring(now())+".2" | table ip, ttp, unique] | eventstats values(ip) as all_ip by unique | eval all_ip = mvjoin(all_ip,", ") | eval name = "Massive scan on website company.corp from "+all_ip, cert-alerted-on=now(), risk="Medium", info="A custom info added to the event", severity="high", th_severity=4, th_tlp="AMBER+STRICT", th_pap="2", test1="A", test2="A" | fields - all_ip

wasn't touched but:

| sendalert thehive_create_a_new_case param.thehive_instance_id=xxxxxx param.case_mode="regular_mode" **param.source="splunk" param.title="name" param.description="$result.info$" param.tags="test;manual;bug" param.scope=1 param.severity="$result.th_severity$" param.pap="$result.th_pap$" param.unique_id_field="unique"

Parameter "case_mode" wasn't right, you were missing the "unique_id_field" too information. Moreover, you specify "$result.name$" for the name of your alert but this is taking only the first row value and not for each one. By giving the name of the field you want to use, the script is doing it better.

Can you tell me if this is better for you? Complete query:

| makeresults
| eval ip="5.6.7.8", ttps="reconnaissance::T1595.001::"+tostring(strftime(now(),"%Y-%m-%d")), unique=tostring(now())+".1"
| table ip, ttp, unique
| append
    [| makeresults
| eval ip="1.2.3.4", ttps="reconnaissance::T1593.002::"+tostring(strftime(now(),"%Y-%m-%d")), unique=tostring(now())+".1"
| table ip, ttp, unique]
| append
    [| makeresults
| eval ip="1.3.5.8", ttps="reconnaissance::T1593.002::"+tostring(strftime(now(),"%Y-%m-%d")), unique=tostring(now())+".2"
| table ip, ttp, unique]
| eventstats values(ip) as all_ip by unique
| eval all_ip = mvjoin(all_ip,", ") 
| eval name = "Massive scan on website company.corp from "+all_ip, cert-alerted-on=now(), risk="Medium", info="A custom info added to the event", severity="high", th_severity=4, th_tlp="AMBER+STRICT", th_pap="2", test1="A", test2="A"
| fields - all_ip
| sendalert thehive_create_a_new_alert param.thehive_instance_id=3ce5b522 param.alert_mode="regular_mode" param.source="splunk" param.title="name" param.description="$result.info$" param.tags="test;manual;bug" param.scope=1 param.severity="$result.th_severity$" param.pap="$result.th_pap$" param.unique_id_field="unique"

I've made a few changes at the same time, a patch will be provided soon.

kvaratop commented 9 months ago

Parameter "case_mode" wasn't right, you were missing the "unique_id_field" too information.

Hi @LetMeR00t and sorry for being long with response. the same error there!

image

p.s: specific raw deletion from source code and changing of scope field wasn't help

LetMeR00t commented 9 months ago

Hi @kvaratop , where does this parameter come from ? Do you have the associated search ? Again, which version are you running, still the v3.2 ? (not the develop version one)

kvaratop commented 9 months ago

Hi @kvaratop , where does this parameter come from ? Do you have the associated search ? Again, which version are you running, still the v3.2 ? (not the develop version one)

excuse me, which param? 'analyst' ? I don't know honestly...) Only mention about "analyst" string in searchjob.log pertains to "ess_analyst" role

I have used your last search provided. | makeresults | eval ip="5.6.7.8", ttps="reconnaissance::T1595.001::"+tostring(strftime(now(),"%Y-%m-%d")), unique=tostring(now())+".1" | table ip, ttp, unique | append [| makeresults | eval ip="1.2.3.4", ttps="reconnaissance::T1593.002::"+tostring(strftime(now(),"%Y-%m-%d")), unique=tostring(now())+".1" | table ip, ttp, unique] | append [| makeresults | eval ip="1.3.5.8", ttps="reconnaissance::T1593.002::"+tostring(strftime(now(),"%Y-%m-%d")), unique=tostring(now())+".2" | table ip, ttp, unique] | eventstats values(ip) as all_ip by unique | eval all_ip = mvjoin(all_ip,", ") | eval name = "Massive scan on website company.corp from "+all_ip, cert-alerted-on=now(), risk="Medium", info="A custom info added to the event", severity="high", th_severity=4, th_tlp="AMBER+STRICT", th_pap="2", test1="A", test2="A" | fields - all_ip | sendalert thehive_create_a_new_alert param.thehive_instance_id=3ce5b522 param.alert_mode="regular_mode" param.source="splunk" param.title="name" param.description="$result.info$" param.tags="test;manual;bug" param.scope=1 param.severity="$result.th_severity$" param.pap="$result.th_pap$" param.unique_id_field="unique" The TA version is that one from the last release - 3.2

kvaratop commented 9 months ago

Hi @kvaratop , where does this parameter come from ? Do you have the associated search ? Again, which version are you running, still the v3.2 ? (not the develop version one)

@LetMeR00t Merry Christmas there! Kindly asking about updates ;)

LetMeR00t commented 9 months ago

Hello @kvaratop ,

Sorry for the delay. Are you using a savedsearch or a correlation search ?... Having "analyst" as an error is weird as you shall not have it at all from the search if you are running a classic saved search. Can you elaborate how you execute this search to help me ? There is no field with "analyst" in the original search so I'm looking to know where this is coming from, probably ES I guess...

kvaratop commented 9 months ago

Hello @kvaratop ,

Sorry for the delay. Are you using a savedsearch or a correlation search ?... Having "analyst" as an error is weird as you shall not have it at all from the search if you are running a classic saved search. Can you elaborate how you execute this search to help me ? There is no field with "analyst" in the original search so I'm looking to know where this is coming from, probably ES I guess...

I use the usual search. no alert, no ES action. Just search bar...

LetMeR00t commented 9 months ago

Okay Can you enable the DEBUG mode in the app configuration, rerun the search and check the audit logs please ?

kvaratop commented 9 months ago

@LetMeR00t let me describe what went wrong:

Now I see Alerts created in the Hive as expected after your example search used

image

Let me check how cases will be created with our data

kvaratop commented 9 months ago

@LetMeR00t I've used such example search: index=notable search_name="Threat - CrowdStrike Malware Detected test - Rule" category="Malicious Document" | eval th_severity=3, th_pap="2" | table action category cim_entity_zone dest file_hash file_name file_path link_host original_file_name parent_process parent_process_exec parent_process_name parent_process_path process process_exec process_hash process_name process_path search_name signature source sourcetype urgency user vendor_product th_severity th_pap | sendalert thehive_create_a_new_case param.thehive_instance_id=3ce5b522 param.case_mode="regular_mode" param.source="splunk" param.title="name" param.description="$result.signature$" param.tags="test;manual;" param.scope=1 param.severity="$result.th_severity$" param.pap="$result.th_pap$" param.unique_id_field="link_host"

The case has been created without error in the search job. Checking audit logs I found: telegram-cloud-document-2-5233352839119520060

LetMeR00t commented 9 months ago

Hi @kvaratop Thank you for letting me know this The issue raised here is fixed in the v3.3 recently published. If you can download and test the fix, you’ll be able to tell me if it’s going better for you If not, let me know

Thank you

kvaratop commented 9 months ago

Hi @kvaratop Thank you for letting me know this The issue raised here is fixed in the v3.3 recently published. If you can download and test the fix, you’ll be able to tell me if it’s going better for you If not, let me know

Thank you

Hi! Now I think all is working as expected! Thank you!