orlikoski / Skadi

Collect, Process, and Hunt with host based data from MacOS, Windows, and Linux
https://www.skadivm.com
GNU General Public License v3.0
487 stars 69 forks source link

cdqr docker helper doesn't #122

Open boingomw opened 4 years ago

boingomw commented 4 years ago

cdqr --max_cpu --nohash in:/tmp/cfreds_2015_data_leakage_pc.E01 out:/tmp/results-mormanual

Fails with a "pyewf_handle_read_buffer" error, but running the same command from inside the docker container works. I narrowed it down to this part of the translated docker command:

docker run --network host -v /tmp/cfreds_2015_data_leakage_pc.E01:/tmp/cfreds_2015_data_leakage_pc.E01 -v /tmp/output:/tmp/output --add-host=elasticsearch:127.0.0.1 --add-host=postgres:127.0.0.1 -v /opt/Skadi/Docker/timesketch/timesketch_default.conf:/etc/timesketch.conf aorlikoski/cdqr:5.1.0 -y /tmp/cfreds_2015_data_leakage_pc.E01 /tmp/output

If that is changed to /tmp:/tmp then the command runs.

I haven't looked at the helper file, but the change should be reasonably simple, right?

orlikoski commented 4 years ago

Hi @boingomw, This is a fantastic question! TL;DR you've already found the recommended solution

Why does it work? It works because the -v volume mount creates a link between the container and the host machine. If it's a folder then everything in the folder can be seen by the container while if it's a file then only that file can be seen. So the CDQR helper script does -v /tmp/cfreds_2015_data_leakage_pc.E01:/tmp/cfreds_2015_data_leakage_pc.E01 which is just the file but -v /tmp/:/tmp: shares everything in the /tmp folder.

I don't have all of the details but I can take a guess that there is more than one file in that disk image (E01, E02, E03, E0x, etc). The CDQR helper script makes the assumption that what is passed with in:<file or folder> is a file or folder. Now this creates a problem for multiple file disk images as Plaso must be pointed at the first file in the chain, such as filename.E01, for it to process it as a disk image but it is actually comprised of multiple files. If it is pointed at the folder then Plaso assumes it's not a disk image and does basic filestat information on each file in the folder (not really what is wanted in your case by any means but exactly how Plaso should behave). This results in the helper script not being able to handle the request in the way the user is would like.

ALL HOPE IS NOT LOST! The CDQR helper scripts cannot account for every situation and the reason it prints the docker commands to stdout is to enable everyone to learn how to use the aorlikoski/cdqr docker image in more advanced ways by using the native docker commands when there is a situation it cannot handle.

I'm really happy to see that has helped get you to this point as learning how to use the Docker commands opens amazing and new ways to use aorlikoski/cdqr. You've already found the solution I would recommend and it is something that can be turned into a custom script for personal use (or create a script for the community or add it into the helper script and file a PR to this repo to share with the community!). I'm all about supporting those who want to learn so please keep in touch and happy to help.

orlikoski commented 4 years ago

I also see that this was based off an example I have in the wiki. I'll take an action to go update that too. In the mean time try this wiki https://github.com/orlikoski/Skadi/wiki/Analyzing-Data-in-Three-Easy-Steps

epicsilence99 commented 4 years ago

@boingomw based off the info @orlikoski provided above did it answer your question? We will still work on getting that wiki updated.

Thanks

boingomw commented 4 years ago

Yup. it helped, thanks.