search

olafhartong / ThreatHunting

A Splunk app mapped to MITRE ATT&CK to guide your threat hunts
MIT License
1.13k stars 178 forks source link

threathunting_file_summary_index is not populated #87

Closed Mark-Law closed 1 year ago

Mark-Law commented 2 years ago

I've searched through all of the XML and CONF files in the ThreatHunting application and cannot find how the summary index is being populated. Is there additional configuration to populate this index that is not in the base documentation? Because the index is not being populated, several dashboards are not returning information that I believe were intended to show additional information.

hassj commented 2 years ago

the same issue, i've already get many event from threathunting index, but have not anything index is populated on "Threat Hunting trigger Overview" dashboard. Anybody kindly help

dstaulcu commented 2 years ago

The threathunting index is populated via collect commands which are part of scheduled searches defined in savedsearches.conf.

As you browse through those searches note the use of macros such as sysmon , windows-security , threathuting_index and so on. Take a look at macros.conf to see what those macros resolve to by default.

Readme.md encourages you to update field values for index, source, and sourcetype in macro definitions as needed within your environment. You may also need to make sure whitelist csv files are properly imported.

If things still aren't working after that please follow up.

hassj commented 2 years ago

Thank for your reply @dstaulcu. everything you said is fine, only "whitelist csv file Created/Installed" is missing although upload empty file in lookup folder to Splunk. or i missing something else.

Whitelist-missing

dstaulcu commented 2 years ago

The output of savedsearch titled "[T1036] Masquerading - renamedbin" should be common when no whitelist entries are defined. Copy the search string associated with that and paste it into search input in threathunting namespace (app). Remove last command in search redirecting output to threathunting index via collect command. If you execute what remains over all time do you get results? If not, remove one command at a time and try again. Do you eventually get results?

hassj commented 2 years ago

The output of savedsearch titled "[T1036] Masquerading - renamedbin" should be common when no whitelist entries are defined. Copy the search string associated with that and paste it into search input in threathunting namespace (app). Remove last command in search redirecting output to threathunting index via collect command. If you execute what remains over all time do you get results? If not, remove one command at a time and try again. Do you eventually get results?

yes, it have result.

hassj commented 2 years ago

i also missing whitelist CVS on board as bellow: image show me some keypoint for resolving it @dstaulcu , thank in advance

dstaulcu commented 2 years ago

You can review the source of the about dashboard to examine the underlying search query for the "Whitelist files created/installed.." panel throwing errors for you. To debug the problem, break down the query into its primary elements to see where things go wrong:

Run this query: "| rest /servicesNS/-/-/data/lookup-table-files" -- Do you get results? Run this query: "| rest /servicesNS/-/ThreatHunting/configs/conf-macros" -- Do you get results? I'm guessing you won't and that your macros.conf file has issues either relating to corruption or permissions. Reinstall app. Run this query: "| rest /servicesNS/-/search/configs/conf-macros" -- Do you get results? I'm guessing you will. If you do get results this affirms theory above that your ThreatHunting app install does not have integrity. If you do not get results, then that would suggest some underling access issue to the specific REST endpoint indicating integrity issues with your Splunk server installation itself.

dstaulcu commented 2 years ago

The output of savedsearch titled "[T1036] Masquerading - renamedbin" should be common when no whitelist entries are defined. Copy the search string associated with that and paste it into search input in threathunting namespace (app). Remove last command in search redirecting output to threathunting index via collect command. If you execute what remains over all time do you get results? If not, remove one command at a time and try again. Do you eventually get results?

yes, it have result.

Great - if you are getting results now add the "| collect threathunting_index" command to the end of the search and run it again. Do you get an error? I'm guessing this relates back to some sort of problem with Splunk interacting with your macros.conf file as indicated by errors in Whitelist files created/installed error messages.

hassj commented 2 years ago

i also missing whitelist CVS on board as bellow: image show me some keypoint for resolving it @dstaulcu , thank in advance

i've already fixed it by install ThreatHunting version 1.4.

hassj commented 2 years ago

The output of savedsearch titled "[T1036] Masquerading - renamedbin" should be common when no whitelist entries are defined. Copy the search string associated with that and paste it into search input in threathunting namespace (app). Remove last command in search redirecting output to threathunting index via collect command. If you execute what remains over all time do you get results? If not, remove one command at a time and try again. Do you eventually get results?

yes, it have result.

Great - if you are getting results now add the "| collect threathunting_index" command to the end of the search and run it again. Do you get an error? I'm guessing this relates back to some sort of problem with Splunk interacting with your macros.conf file as indicated by errors in Whitelist files created/installed error messages.

I got metric on "Activity by time per day" board but "Top triggered host_fqdns in the selected timeframe" and the rest boards still empty. image

image trying search with index=threathunting just get only one host that is splunk enterprise server, the other server which installed Universal Forwader have not.

dstaulcu commented 2 years ago

See discussion in issue #102.

The GitHub version of threathunting app is far ahead of the splunkbase versions and corrects for many issues which I fear you will encounter next.

hassj commented 2 years ago

Thank you for your reply. but everything's not going as well, I think search command on that boards have something wrong. My topo:

image

Any help would be appreciated.

dstaulcu commented 1 year ago

I just installed the latest version of the app from GitHub (not Splunkbase) on a new search head.
After creating the expected indexes and updating index names in macros as appropriate I started seeing events in the trigger overview dashboard as expected within an hour.
There was one bug in the "required app status" panel of the "about this app" dashboard. That bug was a problem in requirements.csv which I created two months ago. I just submitted pull request #104 to correct for that and some other small things which should not affect your situation. I don't think there is anything further I can/will do to help your situation.