search

sneakymonk3y / foxhound-nsm

Foxhound: Blackbox - A Raspberry Pi NSM
37 stars 14 forks source link

Update to Bro 2.5.2 and raspbian 201711-29 #3

Open ramirezversion opened 6 years ago

ramirezversion commented 6 years ago

I am having some errors installing Bro 2.5.2. I am working to try to update to the latest version of tools. Could you help me?

gebhard73 commented 6 years ago

you may try my pull request (which hasn't been accepted yet): https://github.com/sneakymonk3y/foxhound-nsm/pull/2 and make sure to use my repo for download of the code: https://github.com/gebhard73/foxhound-nsm git clone https://github.com/gebhard73/foxhound-nsm.git pls also note that an update will have side effects (because the installation file isn't yet suitable for updates but only for clean installs); I'd suggest investing in a new SD card... Any feedback is appreciated.

ramirezversion commented 6 years ago

Thank you gebhard73. I found your pull yesterday and I am trying to install following it better than original. If I found some error or mistake I will feedback you.

I have started again from an empty SD card.

Thanks for your kindly help. Regards,

2017-12-28 16:20 GMT+01:00 gebhard73 notifications@github.com:

you may try my pull request (which hasn't been accepted yet):

2 https://github.com/sneakymonk3y/foxhound-nsm/pull/2

and make sure to use my repo for download of the code: https://github.com/gebhard73/foxhound-nsm git clone https://github.com/gebhard73/foxhound-nsm.git pls also note that an update will have side effects (because the installation file isn't yet suitable for updates but only for clean installs); I'd suggest investing in a new SD card... Any feedback is appreciated.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/sneakymonk3y/foxhound-nsm/issues/3#issuecomment-354304378, or mute the thread https://github.com/notifications/unsubscribe-auth/AhOCr4s-4lLnNLRu7CpIvbPc1r7JcOMPks5tE7GsgaJpZM4ROQ8l .

-- Antonio Ramírez

ramirezversion commented 6 years ago

Hi @gebhard73 , I've found an error during the criticalstack configuration. Is it normal?

image

gebhard73 commented 6 years ago

Hi, the error should only be present during installation because afterwards the mentioned configuration should be automatically written with broctl by the install script. Please verify that these errors vanish after installation by manually executing the cron jobs. Thanks for helping :-)

gebhard73 commented 6 years ago

Hi, any update, has it worked for you? Thanks.

ramirezversion commented 6 years ago

Hello,

I have been so busy in job and with exams. Work and study is a little bit hard. This weekend I will test to execute de cron jobs manually and see what happend.

For my master final job I have choose to use the foxhound deployment as a base for an IDS domestic black box and deploy a web user interface. If you feel confortable I will have for sure a lot of configurations questions for bro and the critical stack integration so If you feel confortable maybe you can help me.

I have the first question, ¿how can I disable the mail notifications?

Thanks and regards,

2017-12-30 20:00 GMT+01:00 gebhard73 notifications@github.com:

Hi, any update, has it worked for you? Thanks.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/sneakymonk3y/foxhound-nsm/issues/3#issuecomment-354562581, or mute the thread https://github.com/notifications/unsubscribe-auth/AhOCr-j7I2p0gC69Ug7szJKNWObeio3-ks5tFohMgaJpZM4ROQ8l .

-- Antonio Ramírez

gebhard73 commented 6 years ago

I may suggest that you read the bro documentation to get familiar with it, e.g. https://www.bro.org/documentation/index.html

Hints:

broctl config

grep ^Mail /etc/bro/broctl.cfg | grep -v @
MailConnectionSummary = 1
MailHostUpDown = 1

broctl deploy

If foxhound works in general, please close the issue. If not, have a look at my pull requests and the issue I've opened.

ramirezversion commented 6 years ago

thanks for your kindly help. Now i have things already to work but the geoip is not working. have you done something special for it?

Thanks

gebhard73 commented 6 years ago

How do you access geoip / how do you recognize the error?

ramirezversion commented 6 years ago

I was wrong. I found the country information in the resp_cc field of connections.log. The only point is that critical stack log is not generated and i do not know how try to fix. critical stack is already istalled and in master i have the signatures downloaded. in broctl scripts loaded is already these ones... Do you find this issue too?

Thanks

gebhard73 commented 6 years ago

Glad you found it. Regarding critical stack log: can you please provide more details where you have loooked? Where have you missed the logs? Thanks!

ramirezversion commented 6 years ago

yes, I looked the following files to check if everything is ok and it seems to be.

pi@raspberrypi$ more /opt/critical-stack/frameworks/intel/master-public.bro.dat
#fields indicator       indicator_type  meta.source     meta.do_notice
165505c954ef9c182b5dbaeb98834b3d5025bb31        Intel::FILE_HASH        from https://sslbl.abuse.ch/sslbl.rss via intel.criticalstack.com    F
d0351b59fdd6e82b260780f2b60c156e25303fa4        Intel::FILE_HASH        from https://sslbl.abuse.ch/sslbl.rss via intel.criticalstack.com    F
9b585b4014ef6cc5eabc235f63b81a01b6a7d091        Intel::FILE_HASH        from https://sslbl.abuse.ch/sslbl.rss via intel.criticalstack.com    F
c12ec7ea7046337031ee6a1777f79dba2198def6        Intel::FILE_HASH        from https://sslbl.abuse.ch/sslbl.rss via intel.criticalstack.com    F
...
pi@raspberrypi$ more /opt/critical-stack/frameworks/intel/feeds.bro
@load base/frameworks/intel
@load frameworks/intel/seen
@load frameworks/intel/do_notice

redef Intel::read_files += {
        "/opt/critical-stack/frameworks/intel/master-public.bro.dat"
};
pi@raspberrypi$ more /opt/critical-stack/frameworks/intel/__load__.bro
@load ./feeds.bro

Also with the broctl I check for the loaded scripts

pi@raspberrypi$ sudo broctl scripts | grep critical
  {"name":"  /opt/critical-stack/frameworks/intel/__load__.bro"}
  {"name":"    /opt/critical-stack/frameworks/intel/feeds.bro"}

but in the bro var log folder doesn't appear the intel.log

pi@raspberrypi:/var/log/bro/current $ ls -al
total 408
drwxr-xr-x 3 root root   4096 May 25 13:30 .
drwxr-xr-x 5 root root   4096 May 25 13:42 ..
-rw-r--r-- 1 root root    152 May 25 13:30 .cmdline
-rw-r--r-- 1 root root    999 May 25 13:30 communication.log
-rw-r--r-- 1 root root 223362 May 25 13:42 conn.log
-rw-r--r-- 1 root root  49628 May 25 13:42 dns.log
-rw-r--r-- 1 root root    283 May 25 13:30 .env_vars
-rw-r--r-- 1 root root   9569 May 25 13:32 files.log
-rw-r--r-- 1 root root   4065 May 25 13:42 http.log
-rw-r--r-- 1 root root     50 May 25 13:30 known_hosts.log
-rw-r--r-- 1 root root  28616 May 25 13:30 loaded_scripts.log
-rw-r--r-- 1 root root     89 May 25 13:30 packet_filter.log
-rw-r--r-- 1 root root      5 May 25 13:30 .pid
-rw-r--r-- 1 root root   8293 May 25 13:41 ssl.log
-rw-r--r-- 1 root root     59 May 25 13:30 .startup
drwx------ 3 root root   4096 May 25 13:30 .state
-rw-r--r-- 1 root root   1437 May 25 13:40 stats.log
-rwx------ 1 root root     18 May 25 13:30 .status
-rw-r--r-- 1 root root     43 May 25 13:30 stderr.log
-rw-r--r-- 1 root root    188 May 25 13:30 stdout.log
-rw-r--r-- 1 root root   1310 May 25 13:41 weird.log
-rw-r--r-- 1 root root   8755 May 25 13:32 x509.log

This is the output of the broctl deploy too

pi@raspberrypi $ sudo broctl deploy
checking configurations ...
installing ...
removing old policies in /var/spool/bro/installed-scripts-do-not-touch/site ...
removing old policies in /var/spool/bro/installed-scripts-do-not-touch/auto ...
creating policy directories ...
installing site policies ...
generating standalone-layout.bro ...
generating local-networks.bro ...
generating broctl-config.bro ...
generating broctl-config.sh ...
updating nodes ...
stopping ...
stopping bro ...
starting ...
starting bro ...

and the update script.

pi@raspberrypi:/nsm/scripts $ sudo ./update
#### Pulling feed update ####
critical-stack 13:47:08 [INFO] Pulling feed list from the Intel Marketplace.
critical-stack 13:47:09 [INFO] Downloading feed information. Run with the `--debug` flag for more information.
4 / 4 [==================================================================================================================================] 100.00 % 3s
critical-stack 13:47:12 [INFO] Creating master file: master-public.bro.dat. Please wait.
critical-stack 13:47:12 [INFO] Master file created successfully.
critical-stack 13:47:12 [INFO] Checking bro configuration files.
critical-stack 13:47:12 [INFO] Intel include exists in: /usr/share/bro/site/local.bro
critical-stack 13:47:12 [WARN] --- RESTART NOTICE ---
critical-stack 13:47:12 [WARN] You need to restart bro for changes to take effect.
critical-stack 13:47:12 [INFO]  * sudo broctl check
critical-stack 13:47:12 [INFO]  * sudo broctl install
critical-stack 13:47:12 [INFO]  * sudo broctl restart
critical-stack 13:47:12 [INFO] For automatic restarts run: `critical-stack-intel config --set bro.restart=true`
critical-stack 13:47:12 [INFO] Intel files located at: /opt/critical-stack/frameworks/intel
critical-stack 13:47:12 [INFO] API Requests Remaining: 997 of 1000/minute
#### Applying the updates to the bro config ####
bro scripts are ok.
removing old policies in /var/spool/bro/installed-scripts-do-not-touch/site ...
removing old policies in /var/spool/bro/installed-scripts-do-not-touch/auto ...
creating policy directories ...
installing site policies ...
generating standalone-layout.bro ...
generating local-networks.bro ...
generating broctl-config.bro ...
generating broctl-config.sh ...
updating nodes ...
#### Restarting bro ####
stopping ...
stopping bro ...
starting ...
starting bro ...

      __   ____  __ ______
     / /  / __ \/ //_/  _/
    / /__/ /_/ / ,< _/ /
   /____/\____/_/|_/___/
      ________  _____  ____
     /  _/ __ \/ ___/ / __/______ ____  ___  ___ ____
    _/ // /_/ / /__  _\ \/ __/ _ `/ _ \/ _ \/ -_) __/
   /___/\____/\___/ /___/\__/\_,_/_//_/_//_/\__/_/

   Copyright by Florian Roth, Released under the GNU General Public License
   Version 0.28.0

   DISCLAIMER - USE AT YOUR OWN RISK
   Please report false positives via https://github.com/Neo23x0/Loki/issues

                                                                                                                                                    [INFO] Starting separate updater process ...
pi@raspberrypi:/nsm/scripts $

  LOKI UPGRADER

NFO] Updating LOKI ...                                                                                                                              [INFO] Checking location of latest release https://api.github.com/repos/Neo23x0/Loki/releases/latest ...                                              [INFO] Downloading latest release https://github.com/Neo23x0/Loki/releases/download/v0.28.0/loki_0.28.0.zip ...                                       [INFO] Extracting docs/LICENSE-doublepulsarcheck ...                                                                                                  [INFO] Extracting docs/LICENSE-PE-Sieve ...                                                                                                           [INFO] Extracting LICENSE ...                                                                                                                         [INFO] Extracting loki.exe ...                                                                                                                        [INFO] Extracting README.md ...                                                                                                                       [INFO] Extracting requirements.txt ...                                                                                                                [INFO] Extracting tools/pe-sieve32.exe ...                                                                                                            [INFO] Extracting tools/pe-sieve64.exe ...                                                                                                            [INFO] Updating Signatures ...                                                                                                                        [INFO] Downloading https://github.com/Neo23x0/signature-base/archive/master.zip ...                                                                   [INFO] Update complete                                                                                                                                [INFO] Press any key to return ...

Am I doing something wrong related to the critical stack?

Thanks for your kindly help!

gebhard73 commented 6 years ago

thanks for the details, I'll have a look over the weekend

gebhard73 commented 6 years ago

OK, it seems that critical stack package for Pi isn't up to date anymore ... I've analyzed severtal things, but it may come down to the following:

The string policy is missing in the load path for intel frameworks in /opt/critical-stack/frameworks/intel/feeds.bro ... and this file is re-generated wrong every time the feeds from critical-stack are updated.

Workaround: Edit /etc/bro/site/local.bro and add these two lines, e.g. at the bottom below the critical stack entry:

@load policy/frameworks/intel/seen
@load policy/frameworks/intel/do_notice

Then run a /nsm/scripts/update . Then trigger "malicious" traffic, e.g. get some domains (Intel::DOMAIN) from the file /opt/critical-stack/frameworks/intel/master-public.bro.dat and try to access theses on a throw-away VM. The intel.log should finally show up.

Double-check: the seen and do_notice stuff is loaded, see loaded_scripts.log

Please let me know if this helped.

ramirezversion commented 6 years ago

yeah!! it works!!! I will mention you (if you want) in my academic report so please, write me a private message or email.

ramirezversion commented 6 years ago

just last question, is there any way to add geoip location in all logs not only in the connections?

gebhard73 commented 6 years ago

hmmm - I don't think so, but you may add it using scripts: https://www.bro.org/sphinx/frameworks/geoip.html

Perhaps use geo information one step later, e.g. in ELK? / Splunk?

ramirezversion commented 6 years ago

image

now i have the geolocation. i will configure in a cleaner way and maybe we can create a fork with all the integrations and config for ELK.