search

LetMeR00t / TA-thehive-cortex

Technical add-on for Splunk related to TheHive/Cortex from TheHive project
GNU Lesser General Public License v3.0
47 stars 11 forks source link

[BUG] I have a trouble when creating a new test alert #58

Closed dimasix9 closed 1 year ago

dimasix9 commented 1 year ago

Request Type

Bug

Work Environment

Question Answer
OS version (server) Ubuntu 22.04
TheHive version / git hash 3.0.2

Problem Description

I have a curious trouble when creating a new test alert which I have no idea how to solve. It is somehow related with rf variable in alert_actions_base.py file. Please give an explanation if possible. image image

Possible Solutions

-

Logs (issued from the search.log with logging mode set to DEBUG under Settings/Configuration)

05-26-2023 18:16:48.610 ERROR sendmodalert [80654 phase_1] - action=thehive_create_a_new_alert STDERR - Traceback (most recent call last): 05-26-2023 18:16:48.610 ERROR sendmodalert [80654 phase_1] - action=thehive_create_a_new_alert STDERR - File "/opt/splunk/etc/apps/TA-thehive-cortex/bin/ta_thehive_cortex/alert_actions_base.py", line 195, in prepare_meta_for_cam 05-26-2023 18:16:48.610 ERROR sendmodalert [80654 phase_1] - action=thehive_create_a_new_alert STDERR - rf = gzip.open(self.results_file, 'rt') 05-26-2023 18:16:48.610 ERROR sendmodalert [80654 phase_1] - action=thehive_create_a_new_alert STDERR - File "/opt/splunk/lib/python3.7/gzip.py", line 58, in open 05-26-2023 18:16:48.610 ERROR sendmodalert [80654 phase_1] - action=thehive_create_a_new_alert STDERR - binary_file = GzipFile(filename, gz_mode, compresslevel) 05-26-2023 18:16:48.610 ERROR sendmodalert [80654 phase_1] - action=thehive_create_a_new_alert STDERR - File "/opt/splunk/lib/python3.7/gzip.py", line 168, in init 05-26-2023 18:16:48.610 ERROR sendmodalert [80654 phase_1] - action=thehive_create_a_new_alert STDERR - fileobj = self.myfileobj = builtins.open(filename, mode or 'rb') 05-26-2023 18:16:48.610 ERROR sendmodalert [80654 phase_1] - action=thehive_create_a_new_alert STDERR - FileNotFoundError: [Errno 2] No such file or directory: '/opt/splunk/var/run/splunk/dispatch/1685114207.263/sendalert_temp_results.csv.gz' 05-26-2023 18:16:48.610 ERROR sendmodalert [80654 phase_1] - action=thehive_create_a_new_alert STDERR - During handling of the above exception, another exception occurred: 05-26-2023 18:16:48.610 ERROR sendmodalert [80654 phase_1] - action=thehive_create_a_new_alert STDERR - Traceback (most recent call last): 05-26-2023 18:16:48.610 ERROR sendmodalert [80654 phase_1] - action=thehive_create_a_new_alert STDERR - File "/opt/splunk/etc/apps/TA-thehive-cortex/bin/thehive_create_a_new_alert.py", line 82, in 05-26-2023 18:16:48.610 ERROR sendmodalert [80654 phase_1] - action=thehive_create_a_new_alert STDERR - exitcode = AlertActionWorkerthehive_create_a_new_alert("TA-thehive-cortex", "thehive_create_a_new_alert").run(sys.argv) 05-26-2023 18:16:48.610 ERROR sendmodalert [80654 phase_1] - action=thehive_create_a_new_alert STDERR - File "/opt/splunk/etc/apps/TA-thehive-cortex/bin/ta_thehive_cortex/alert_actions_base.py", line 215, in run 05-26-2023 18:16:48.610 ERROR sendmodalert [80654 phase_1] - action=thehive_create_a_new_alert STDERR - self.prepare_meta_for_cam() 05-26-2023 18:16:48.610 ERROR sendmodalert [80654 phase_1] - action=thehive_create_a_new_alert STDERR - File "/opt/splunk/etc/apps/TA-thehive-cortex/bin/ta_thehive_cortex/alert_actions_base.py", line 204, in prepare_meta_for_cam 05-26-2023 18:16:48.610 ERROR sendmodalert [80654 phase_1] - action=thehive_create_a_new_alert STDERR - if rf: 05-26-2023 18:16:48.610 ERROR sendmodalert [80654 phase_1] - action=thehive_create_a_new_alert STDERR - UnboundLocalError: local variable 'rf' referenced before assignment 05-26-2023 18:16:48.667 INFO sendmodalert [80654 phase_1] - action=thehive_create_a_new_alert - Alert action script completed in duration=524 ms with exit code=1 05-26-2023 18:16:48.667 WARN sendmodalert [80654 phase_1] - action=thehive_create_a_new_alert - Alert action script returned error code=1 05-26-2023 18:16:48.668 ERROR sendmodalert [80654 phase_1] - Error in 'sendalert' command: Alert script returned error code 1. 05-26-2023 18:16:48.677 INFO ReducePhaseExecutor [80654 phase_1] - Ending phase_1 05-26-2023 18:16:48.677 INFO UserManager [80654 phase_1] - Unwound user context: dmytro -> NULL 05-26-2023 18:16:48.677 ERROR SearchOrchestrator [80645 searchOrchestrator] - Phase_1 failed due to : Error in 'sendalert' command: Alert script returned error code 1. 05-26-2023 18:16:48.677 INFO ReducePhaseExecutor [80647 StatusEnforcerThread] - ReducePhaseExecutor=1 action=QUIT 05-26-2023 18:16:48.677 INFO DispatchExecutor [80647 StatusEnforcerThread] - Search applied action=QUIT while status=GROUND 05-26-2023 18:16:48.677 INFO SearchStatusEnforcer [80647 StatusEnforcerThread] - sid=1685114207.263, newState=QUIT, message=Error in 'sendalert' command: Alert script returned error code 1. 05-26-2023 18:16:48.677 ERROR SearchStatusEnforcer [80647 StatusEnforcerThread] - SearchMessage orig_component=SearchStatusEnforcer sid=1685114207.263 message_key= message=Error in 'sendalert' command: Alert script returned error code 1. 05-26-2023 18:16:48.677 INFO SearchStatusEnforcer [80647 StatusEnforcerThread] - State changed to QUIT: Error in 'sendalert' command: Alert script returned error code 1. 05-26-2023 18:16:48.681 INFO UserManager [80647 StatusEnforcerThread] - Unwound user context: dmytro -> NULL 05-26-2023 18:16:48.682 INFO DispatchStorageManager [80645 searchOrchestrator] - Remote storage disabled for search artifacts. 05-26-2023 18:16:48.682 INFO DispatchManager [80645 searchOrchestrator] - DispatchManager::dispatchHasFinished(id='1685114207.263', username='dmytro') 05-26-2023 18:16:48.682 INFO UserManager [80645 searchOrchestrator] - Unwound user context: dmytro -> NULL 05-26-2023 18:16:48.682 INFO SearchStatusEnforcer [80638 RunDispatch] - SearchStatusEnforcer is already terminated 05-26-2023 18:16:48.682 INFO UserManager [80638 RunDispatch] - Unwound user context: dmytro -> NULL 05-26-2023 18:16:48.682 INFO LookupDataProvider [80638 RunDispatch] - Clearing out lookup shared provider map 05-26-2023 18:16:48.685 ERROR dispatchRunner [25584 MainThread] - RunDispatch has failed: sid=1685114207.263, exit=-1, error=Error in 'sendalert' command: Alert script returned error code 1.

LetMeR00t commented 1 year ago

Hi @dimasix9 , I agree with you that there is a strange error with this rf non assigned variable. This part of the code is coming from an auto-generated code from the Add-on Builder from Splunk. However, even this is error is occuring, I think I know why it's not working in fact. This error is coming because it can't open the results file from your search because... you didn't provide any search as an input ... The "Job SID (data)" field must be filled, in this dashboard, with the SID of a search from which you want to reuse the results to be processed by the application and then create an alert accordingly (this implies to run the search before, get the SID from it and wait for it to finish before doing so). If you want to avoid that, I invite you to do whatever you want directly within the custom alert action existing in the savedsearches actions that you can setup to create an alert in TheHive. Doing this will reuse automatically the output of your savedsearch as the input for the script.

I hope it's more clear now :)

dimasix9 commented 1 year ago

Hi @LetMeR00t , TYSM for such an immediate reply.

But another issue occurs even when I fill the Job SID field. I took an event from my job activity and made a search with makeresults and eval as an example. Sorry, I'm obligated to hide most of field values due to privacy :) But I hope you get what I mean. image Then I fill all the required fields and options in theHive app. Somewhy sendalert command appends to the _time field that's taken from the Job SID search. I don't clearly understand if that's how this action is supposed to work. image And when I click Run, I get another error image I go to the search.log once more and see these strings image I saw you answered in a different thread that the error 5 is related to your custom code. But I'm still wondering if there's any chance to get to the root of the problem.

P.S. Decided to attach a full search.log file. Hope it'll be helpful. search.log

LetMeR00t commented 1 year ago

Hi @dimasix9 Thank you very much for such amount of details from your issue, it’s helping a lot.

so the output you see on the dashboard is just a fake view to show you the result (if you check the dashboard code, you’ll see hidden panel that are effectively doing the action correctly.

I was not able to find anything in the search.log…

However, did you checked on the Audit logs dashboard to see if any error is raised ? If the error is managed by my script (most of the time it’s the case) then the audit logs dashboard will help you to find out the issue (you can enable the DEBUG mode on your loggings too to activate more logs especially on the audit dashboard)

If you don’t find anything , try to rerun your search as sometimes the error isn’t show every time in the search log…

Keep me posted, if I need to help you more, we can discuss it further by email directly if you agree to send me more screenshots with details.

Thank you

dimasix9 commented 1 year ago

Thank you @LetMeR00t No, I haven’t checked audit logs yet. But I’ll write a comment when I check. Thank you again

dimasix9 commented 1 year ago

Hello @LetMeR00t It seems that the problem consists in CaseTemplate which is not found even though I filled the Case Template field with the MISP value Here's a screen of what I found in thehive_create_a_new_case_modalert.log image

Upd. The thing is I can create an alert now but creating case still returns error code 5( I suppose that an alert trouble was somehow related to the absense of a lookup that matches fields but it magically appeared a little later.

Upd2. The ploblem now seems to be solved. It was my fault I didn't get to know the proper name of the template which is actually MISPEvent. And please, if you don't mind, may this thread remain open as long as I need to do this task fully. I'm quite a newcomer in this topic so sometimes tend to make stupid mistakes. :)

LetMeR00t commented 1 year ago

Hi @dimasix9 , I suggest you to reopen any new issue if you encounter any new trouble but this one could be closed.