search

wzod / wzod_mastiff_installer

Installs MASTIFF v0.6.0 including all dependencies and default plugins for Ubuntu with one command.
Apache License 2.0
3 stars 1 forks source link

Mastiff Docker image logfile location #1

Open wayward710 opened 8 years ago

wayward710 commented 8 years ago

I tested the Mastiff Docker image on a Fedora 23 64-bit VM and ran into some errors. It looks like the script is setting the log file to the work directory (/home/nonroot/workdir in my Docker container). This caused conflicts because it was overwriting malware files to be analyzed with directories of the same name. I experimented with changing the location of the log file in the mastiff.conf file to "/home/nonroot/workdir/logs" and that executed successfully.

wzod commented 8 years ago

Hi Wendy,

Can you provide an example of how the malware files were overwritten? I'm unable to replicate the conflict as the workdir directory contains all the files processed/stored and logs are appended to the mastiff.log file.

Please list the steps to reproduce the conflicts/errors as I am currently not seeing the issue.

Also, to be clear, are you using the Dockerfile from the Docker repo ( https://github.com/wzod/Dockerfiles/blob/master/MASTIFF-0.7.1/Dockerfile )? Asking since this issue was entered in the repo for the standalone script, which is for installing MASTIFF on a dedicated system (i.e. tested for Ubuntu Linux host or virtual machine). Based on the path of the workdir directory, it looks like you're using the Dockerfile from the Docker repo, but figured to ask.

Thanks!

wayward710 commented 8 years ago

Yes, I was using the Docker repo ("sudo docker pull remnux/mastiff"). Then I started up the Docker container per instructions ("sudo docker run --rm -it -v ~/mastiff-workdir:/home/nonroot/workdir remnux/mastiff"). To run mastiff within the Docker container, I'd type something like "./mas.py -c ./mastiff.conf /home/nonroot/workdir/e67cd39ab0a3a49cf1c4fdda89687b03".

This is what happens when log_dir is set to the work directory in mastiff.con e.g. log_dir = /home/nonroot/workdir

[2016-02-29 04:38:34,834] [INFO] [Mastiff.analyze] : Starting analysis on /home/nonroot/workdir/91530bde7d5d48021cda6843314bb02d Traceback (most recent call last): File "./mas.py", line 248, in main() File "./mas.py", line 243, in main analyze_file(fname, opts, loglevel)
File "./mas.py", line 68, in analyze_file my_analysis.analyze(fname, opts.plugin_name) File "/home/nonroot/mastiff-0.7.1/mastiff/core.py", line 471, in analyze if self.init_file(fname) is None: File "/home/nonroot/mastiff-0.7.1/mastiff/core.py", line 271, in init_file fh = logging.FileHandler(log_dir + os.sep + 'mastiff.log' ) File "/usr/lib/python2.7/logging/init.py", line 903, in init StreamHandler.init(self, self._open()) File "/usr/lib/python2.7/logging/init.py", line 928, in _open stream = open(self.baseFilename, self.mode) IOError: [Errno 20] Not a directory: '/home/nonroot/workdir/91530bde7d5d48021cda6843314bb02d/mastiff.log'

When I change the log directory, the problem goes away: log_dir = /home/nonroot/workdir/log

[2016-02-29 04:46:30,157] [INFO] [Mastiff.analyze] : Starting analysis on /home/nonroot/workdir/e67cd39ab0a3a49cf1c4fdda89687b03 [2016-02-29 04:46:30,295] [INFO] [Mastiff.Init_File] : Analyzing /home/nonroot/workdir/e67cd39ab0a3a49cf1c4fdda89687b03. [2016-02-29 04:46:30,296] [INFO] [Mastiff.Init_File] : Log Directory: /home/nonroot/workdir/log/e67cd39ab0a3a49cf1c4fdda89687b03 [2016-02-29 04:46:30,652] [INFO] [Mastiff.DB.Insert] : Adding ['EXE', 'Generic'] [2016-02-29 04:46:30,699] [INFO] [Mastiff.Analysis] : File categories are ['EXE', 'Generic']. [2016-02-29 04:46:30,702] [INFO] [Mastiff.Plugins.PE Info] : Starting execution. [2016-02-29 04:46:30,861] [INFO] [Mastiff.Plugins.Resources] : Starting execution. [2016-02-29 04:46:30,936] [INFO] [Mastiff.Plugins.Digital Signatures] : Starting execution. [2016-02-29 04:46:31,029] [INFO] [Mastiff.Plugins.Digital Signatures] : No signature on the file. [2016-02-29 04:46:31,035] [INFO] [Mastiff.Plugins.Single-Byte Strings] : Starting execution. [2016-02-29 04:46:31,899] [INFO] [Mastiff.Plugins.File Information] : Starting execution. [2016-02-29 04:46:32,011] [INFO] [Mastiff.Plugins.Fuzzy Hashing] : Starting execution. [2016-02-29 04:46:32,011] [INFO] [Mastiff.Plugins.Fuzzy Hashing] : Generating fuzzy hash. [2016-02-29 04:46:32,164] [INFO] [Mastiff.Plugins.Fuzzy Hashing.compare] : Comparing fuzzy hashes. [2016-02-29 04:46:32,165] [INFO] [Mastiff.Plugins.Hex Dump] : Starting execution. [2016-02-29 04:46:32,165] [INFO] [Mastiff.Plugins.Hex Dump] : Disabled. Exiting. [2016-02-29 04:46:32,165] [INFO] [Mastiff.Plugins.Metascan Online] : Starting execution. [2016-02-29 04:46:32,166] [ERROR] [Mastiff.Plugins.Metascan Online] : No Metascan Online API Key - exiting. [2016-02-29 04:46:32,166] [INFO] [Mastiff.Plugins.Embedded Strings Plugin] : Starting execution. [2016-02-29 04:46:32,607] [INFO] [Mastiff.Plugins.VirusTotal] : Starting execution. [2016-02-29 04:46:32,607] [ERROR] [Mastiff.Plugins.VirusTotal] : No VirusTotal API Key - exiting. [2016-02-29 04:46:32,608] [INFO] [Mastiff.Plugins.yara] : Starting execution. [2016-02-29 04:46:32,993] [INFO] [Mastiff.Analysis] : Finished analysis for /home/nonroot/workdir/e67cd39ab0a3a49cf1c4fdda89687b03. [2016-02-29 04:46:33,004] [INFO] [Mastiff] : There are 0 jobs in the queue.

The log files look right: $ ls ~/workdir/logs e67cd39ab0a3a49cf1c4fdda89687b03 mastiff.db mastiff.log

wzod commented 8 years ago

Thanks for the info! I just tried following your steps and did not run into any issues; MASTIFF was able to process the file and append the output to mastiff.log. I can certainly look into modifying the Dockerfile to move the logs directory, but want to make sure I can replicate the issue.

One other question: before running MASTIFF, did you make the ~/mastiff-workdir directory world accessible (such as running 'chmod a=rwx ~/mastiff-workdir') prior to running the Docker container?

I plan on running some more tests on my end and taking a closer look at the python errors as I am curious as to why I'm not able to replicate the issue.

wayward710 commented 8 years ago

Just tried it again and made sure that I had made the directory world accessible and got the same results. If it's helpful, here's the information about my VM:

$ cat /etc/os-release NAME=Fedora VERSION="23 (Workstation Edition)" ID=fedora VERSION_ID=23 PRETTY_NAME="Fedora 23 (Workstation Edition)" ANSI_COLOR="0;34" CPE_NAME="cpe:/o:fedoraproject:fedora:23" HOME_URL="https://fedoraproject.org/" BUG_REPORT_URL="https://bugzilla.redhat.com/" REDHAT_BUGZILLA_PRODUCT="Fedora" REDHAT_BUGZILLA_PRODUCT_VERSION=23 REDHAT_SUPPORT_PRODUCT="Fedora" REDHAT_SUPPORT_PRODUCT_VERSION=23 PRIVACY_POLICY_URL=https://fedoraproject.org/wiki/Legal:PrivacyPolicy VARIANT="Workstation Edition" VARIANT_ID=workstation

Here's the information about the Python I'm running: $ python --version Python 2.7.10

Here's my Docker version: $ docker --version Docker version 1.10.1, build 9e83765