LetMeR00t / TA-thehive-cortex

Technical add-on for Splunk related to TheHive/Cortex from TheHive project
GNU Lesser General Public License v3.0
47 stars 11 forks source link

Unable to load input #5

Closed danblae closed 3 years ago

danblae commented 3 years ago

Hey, I am unable to load the input page, just like for the Cortex app. I then tried to use different Splunk version (8.1.0, 8.0.0, 7.3.0) but every time it does not load. The "good" thing is, that I get an error message from splunk:

Unable to initialize modular input "thehive_supervisor" defined in the app "TA-thehive-cortex": Introspecting scheme=thehive_supervisor: script running failed (exited with code 1)

I have also followed all the steps in the Installation and quick start guide. Searching for this issue did not help either because the app is too new. Do you have any recommendations on how to proceed and what to do?

Thanks in advance!

LetMeR00t commented 3 years ago

Hi This issue is the same kind than #4 I can’t reproduce this thing on my side using different versions of Splunk . This is weird but something tells me that I probably missing some libraries used by the thehive4py library. I will do another test on my side with a Splunk not running on my Linux instance to see if something changed Could you try to run a TheHive custom command (with logging set to DEBUG in the configuration page) and see if in the search.log you find something related to a missing library ? Thank you

LetMeR00t commented 3 years ago

I've reinstalled it on a different OS and now I have the issue. After investigation, it seems that there are some issues with a missing library used by the python library named "magic". It seems that it's missing the libraries cffi, pycparser ... (I ran the test I explained you on the previous comment) I don't understand why it's working on my side as they are not installed using pip and Splunk doesn't have it ............ I assume that Splunk is using an embedded python executable ? ... I will try to identify the missing libraries and adding them in the app.

danblae commented 3 years ago

Could you maybe give me a list of libraries used by your tool so that I could try installing them manually?

LetMeR00t commented 3 years ago

@danblae , I found out what was the issue. I'm using an incomplete python library "magic". I've just solved the issue on my new instance, I will push the changes right now to see if it's working better for you.

LetMeR00t commented 3 years ago

Hi @danblae,

Could you install the new version of the app (1.1.2) and see if the bug is resolved ?

Thank you

danblae commented 3 years ago

I send this from my phone. I will test it first thing tomorrow. I can not do it right now. Thanks a lot!

danblae commented 3 years ago

I have tried the new version and I get the same error. I forgot to mention, that I am running Splunk on a docker container with the WSL2 Linux Kernel on a Windows machine (I hope that helps). I am uncertain on how to actually get TheHive into debug under docker

Could you try to run a TheHive custom command (with logging set to DEBUG in the configuration page) and see if in the search.log

Maybe you have some advice. I would love to help resolve this issue.

LetMeR00t commented 3 years ago

It seems that on a Windows instance it worked for me x) Well, to do so : 1) Open the application in Splunk 2) Go to "Configuration" on the navigation bar then under "Logging", set the parameter to "DEBUG". 3) Click on "TheHive: Cases" on the navigation bar 4) Open the search once it ran that list all the cases issues. (https://github.com/LetMeR00t/TA-thehive-cortex/blob/master/images/cases_list.png) The search on the bottom can be opened by click on the right bottom of the panel, I hope you will understand what I've just said :) You should see something with :

| makeresults
| eval keyword = "*", status = "*", severity = "*", tags = "*", title = "*", assignee = "*", date = "* TO *"
| thehivecases

5) You can remove everything after the custom command "thehivecases" or just copy paste the above search and run it 6) Once it's done (I assume it will say that the script has an error), open the search.log (by clicking under "Job" -> "Inspect Job" image 7) Then click on the "search.log" image 8) Now search for issues related to something like "No module named" or some errors from the python script :)

danblae commented 3 years ago

Ok I have the search log. I hope this helps: search.log (I did update the app to the newest version)

LetMeR00t commented 3 years ago

The issue is here :

11-19-2020 10:40:35.781 ERROR ScriptRunner - stderr from '/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/TA-thehive-cortex/bin/thehive_search_cases.py': Traceback (most recent call last): 11-19-2020 10:40:35.781 ERROR ScriptRunner - stderr from '/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/TA-thehive-cortex/bin/thehive_search_cases.py': File "/opt/splunk/etc/apps/TA-thehive-cortex/bin/thehive_search_cases.py", line 6, in 11-19-2020 10:40:35.781 ERROR ScriptRunner - stderr from '/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/TA-thehive-cortex/bin/thehive_search_cases.py': from thehive import TheHive 11-19-2020 10:40:35.781 ERROR ScriptRunner - stderr from '/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/TA-thehive-cortex/bin/thehive_search_cases.py': File "/opt/splunk/etc/apps/TA-thehive-cortex/bin/thehive.py", line 3, in 11-19-2020 10:40:35.781 ERROR ScriptRunner - stderr from '/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/TA-thehive-cortex/bin/thehive_search_cases.py': from thehive4py.api import TheHiveApi 11-19-2020 10:40:35.781 ERROR ScriptRunner - stderr from '/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/TA-thehive-cortex/bin/thehive_search_cases.py': File "/opt/splunk/etc/apps/TA-thehive-cortex/bin/ta_thehive_cortex/aob_py3/thehive4py/api.py", line 7, in 11-19-2020 10:40:35.781 ERROR ScriptRunner - stderr from '/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/TA-thehive-cortex/bin/thehive_search_cases.py': import magic 11-19-2020 10:40:35.781 ERROR ScriptRunner - stderr from '/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/TA-thehive-cortex/bin/thehive_search_cases.py': File "/opt/splunk/etc/apps/TA-thehive-cortex/bin/ta_thehive_cortex/magic/init.py", line 1, in 11-19-2020 10:40:35.781 ERROR ScriptRunner - stderr from '/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/TA-thehive-cortex/bin/thehive_search_cases.py': from .magic import * 11-19-2020 10:40:35.781 ERROR ScriptRunner - stderr from '/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/TA-thehive-cortex/bin/thehive_search_cases.py': File "/opt/splunk/etc/apps/TA-thehive-cortex/bin/ta_thehive_cortex/magic/magic.py", line 189, in 11-19-2020 10:40:35.781 ERROR ScriptRunner - stderr from '/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/TA-thehive-cortex/bin/thehive_search_cases.py': raise ImportError('failed to find libmagic. Check your installation') 11-19-2020 10:40:35.781 ERROR ScriptRunner - stderr from '/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/TA-thehive-cortex/bin/thehive_search_cases.py': ImportError: failed to find libmagic. Check your installation

I will try to run Splunk under a docker environment to check the result ...

LetMeR00t commented 3 years ago

First results of my investigation is that my last fix is not working for you and the reason is because I have the libmagic library inside a DLL file and when python detects that it's running on a Windows instance, it's using the DDL. However, in our case, we are running under Linux :) I will keep searching a solution.

LetMeR00t commented 3 years ago

As this library is not used and will not used by the Splunk app, I will make some changes to remove the usage of the magic library. It's not a lot of lines so I will do it to let you have a working application. I'm changing the things and keep you in touch once you can test it.

LetMeR00t commented 3 years ago

@danblae , @remg427, A new version is available. It's working on my docker 👍 Could you check again please ?

danblae commented 3 years ago

the new version (1.1.3) resolved the issue for me and even runs on the latest Splunk version. Thanks!