search

aplura / Tango

Honeypot Intelligence with Splunk
GNU General Public License v2.0
255 stars 43 forks source link

Universal Forwarder Configuration failed #29

Open redheadontherun opened 8 years ago

redheadontherun commented 8 years ago

When installing through the .sensor.sh or .uf_only.sh script I get the error: Universal Forwarder Configuration failed. Please check /var/log/tango_install.log for more details. I have attached the log file UF Error Log.txt

This has happened when trying to use the .sensor.sh script on a brand new Ubuntu 14.04.4 LTS server as well.

mackwage commented 8 years ago

Did you stop the splunk process before running this script? It indicated splunkd was already running:

"-- Migration information is being logged to '/opt/splunkforwarder/var/log/splunk/migration.log.201 ERROR: In order to migrate, Splunkd must not be running. ERROR while running renew-certs migration."

redheadontherun commented 8 years ago

the splunkd on the actual splunk server or on the honeypot?

redheadontherun commented 8 years ago

that did it - install completed successfully. However, now i'm not seeing any logs from my sensors. My sensors show up but the Tango app isnt pulling logs in. I even tried uploading the .json file directly into splunk to see if that would populate the charts but that didnt work either.

mackwage commented 8 years ago

When you go to Search, are you able to manually see the logs?

redheadontherun commented 8 years ago

If I seach for index=* yes

Sent from my T-Mobile 4G LTE Device

-------- Original message -------- From: mackwage notifications@github.com Date: 2/29/2016 12:42 PM (GMT-08:00) To: aplura/Tango Tango@noreply.github.com Cc: redheadontherun redheadontherun@gmail.com Subject: Re: [Tango] Universal Forwarder Configuration failed (#29)

When you go to Search, are you able to manually see the logs?

— Reply to this email directly or view it on GitHub.

mackwage commented 8 years ago

What if you search specifically in the tango index?

Also, could you paste the contents of inputs.conf from the Tango directory on your honeypot?

redheadontherun commented 8 years ago

i get what 1 event. Looks like the event that is generated after the UF installation.

inputs.conf =

!/bin/sh

unset LD_LIBRARY_PATH python /opt/splunkforwarder/etc/apps/tango_input/bin/input.py

mackwage commented 8 years ago

Please post the contents of the inputs.conf file from the honeypot

redheadontherun commented 8 years ago

[monitor:///opt/cowrie/log/cowrie.json*] disabled = false index = honeypot sourcetype = cowrie host = test

[script://./bin/input.sh] disabled=false sourcetype=sensor index=honeypot interval=86400 host = test

redheadontherun commented 8 years ago

Posted

Sent from my T-Mobile 4G LTE Device

-------- Original message -------- From: mackwage notifications@github.com Date: 2/29/2016 1:54 PM (GMT-08:00) To: aplura/Tango Tango@noreply.github.com Cc: redheadontherun redheadontherun@gmail.com Subject: Re: [Tango] Universal Forwarder Configuration failed (#29)

Please post the contents of the inputs.conf file from the honeypot

— Reply to this email directly or view it on GitHub.

brianwarehime commented 8 years ago

If you didn't enable the hoenypot index to be able to be searched by default, you'll need to enable that.

You'll need to allow users to search the 'honeypot' index by default. To do this, go into “Settings”, then “Access Controls”, then “Roles”, “Admin”, then scroll all the way down to “Indexes Searched by Default”, then add honeypot to the right-hand column.

Let me know if that works out for you!

redheadontherun commented 8 years ago

I did that, still nothing. But I did notice the splunklogger index is disabled.

Sent from my T-Mobile 4G LTE Device

-------- Original message -------- From: Brian Warehime notifications@github.com Date: 3/2/2016 7:37 AM (GMT-08:00) To: aplura/Tango Tango@noreply.github.com Cc: redheadontherun redheadontherun@gmail.com Subject: Re: [Tango] Universal Forwarder Configuration failed (#29)

If you didn't enable the hoenypot index to be able to be searched by default, you'll need to enable that.

You'll need to allow users to search the 'honeypot' index by default. To do this, go into “Settings”, then “Access Controls”, then “Roles”, “Admin”, then scroll all the way down to “Indexes Searched by Default”, then add honeypot to the right-hand column.

Let me know if that works out for you!

— Reply to this email directly or view it on GitHub.

redheadontherun commented 8 years ago

UPDATE -

I reinstalled splunk, however, the tango app still does not load any values. I have ensured the Admin role can search the honeypot index. I can confirm that if i manually load a cowrie.json file to the honeypot index NOTHING shows up in the tango app. However, you can search specifically for the index and i get results. I can confirm that the honeypots are sending traffic through tcpdump and that the splunk server is receiving that traffic. The issue still is that the tango app does not process the data.

redheadontherun commented 8 years ago

Turns out the user being used to read the logs did not have permission to do so. Updated permissions and everything is working fine...except the virustotal hash submit. But that's another issue...

mlalibs commented 8 years ago

Specifically, the Tango sensor.sh script sets up users 'cowrie' and 'splunk' on the system. When cowrie is launched using it's own 'start.sh' script with the 'cowrie' user, it sets a umask of 0077 which prevents the 'splunk' user from reading the generated log files. This can be resolved by modifying cowrie's 'start.sh' script and changing the umask to 0027 then adding the 'splunk' user to the 'cowrie' group.