biggiesmallsAG / nightHawkResponse

Incident Response Forensic Framework
599 stars 139 forks source link

mans file too big #13

Closed xeen3d closed 7 years ago

xeen3d commented 8 years ago

Hi i make a sample Invest with a enhanced script from redline . The .mans file is round about 1,2 GB Size in my eyes not so big but i can't upload at maybe 40 % bit more or less i get a message "413 Request Entity Too Large" on a white page is there a size in php.ini ? not set right ? can i change that ? like give a bigger valve in php ini ?

it seems i get nothing work here a zip of Folder does not have any affect and a small mans file too nothing happens i can open same file without problems in Redline

a grep output shows me that nothing happens after i try a upload in VM

nightha+ 1365 1262 0 00:10 ? 00:00:00 [nightHawk] defunct nightha+ 1370 1258 0 00:10 ? 00:00:00 [nightHawk] defunct

after a clean reboot same situation i have installed and only try upload a mans or a zip both of them have no reaction in nightHawk best

Andre

biggiesmallsAG commented 8 years ago

Hi Andre,

There is a hard limit set by nginx config which can be changed, if you go to /opt/nighthawk/etc/nginx/conf.d/nighthawk.conf and change "client_max_body_size 500M;" to a size larger than the mans file then restart the services (sysctl restart nighthawk) it should work fine. We are looking at putting in a Lua parser to access environment variables for setting nginx limits like the one above.

Regards,

Daniel

xeen3d commented 8 years ago

Hi thanks for answer but i can't get nightHawk running here i test a mans File nothing happens (i check that in 3 Browsers) i try zip the Session Folder nothing happens i try zip the Files of the Session Folder nothing happens I try a different zip app nothing happens

In Redline i can open all that Files without any Problem but nightHawk do absolutely nothing here, i try a other Hypervisor first vmware now Parallels both in newest Version a grep give me always same output:

nightha+ 1365 1262 0 00:10 ? 00:00:00 [nightHawk] defunct

For me here the Installation is without function and that is the third installation now, look nice but nothing more, can someone give me a test File that work in a other installation ?

best

Andre

biggiesmallsAG commented 8 years ago

Hi Andre,

Im not sure what you mean with the grep output, are you running that on a ps -ef output ?

Can you zip up the logs in /opt/nighthawk/var/logs and send them over so i can take a look?

Regards,

Dan

biggiesmallsAG commented 8 years ago

Also, in the nighthawk_utils folder on github, there is a zip file in there, can you test that it works?

When you zip the sessions folder, what exactly are you zipping up, are you following our guide in the readme ?

xeen3d commented 8 years ago

Hi this Zip File works my own ones not is it possible that i must use windows to create that zip file ? i use OSx and my hole Investigations are on a OSx Disk not a Windows Disk. I use Windows in VM´s and put output of Redline on Mac OSx Shared Folder not on the Disk from where i investigate (not a good idea to use that Disk normally such Disk is writhe protected)

The content of your Zip looks like my Content i try now mount a external NTFS Disk to my test Windows and put output of redline there and use windows to create the zip File i told you tomorrow if that was the Problem.

Thanks for your help

best

Andre

xeen3d commented 8 years ago

Hi i found the Problem, if i put output of Redline to a windows Disk and use a Windows zip Tool for create the zip for NightHawk i get proper Results.

For now i check that with 6 different VM´s and different Redline Scripts all works.

It looks that Night Hawk get Problems if the Redline Folder (like 20160707054834) was stored on other Format than NTFS or not zipped within Windows self. nightHawk is a cool tool very cool :-) and you are at the beginning develop it if i can give you closer Reports about that Problem let me exactly know what i should do, for now i have a workaround for future it will be perfect if nightHawk can get zip files from where ever they are.

Let me know step by step what i can do to give you closer Information, i can give you some samples of Zips from my OSx Disk (all Redline Win Investigations)

best

Andre

roshanmaskey commented 8 years ago

Hi xeen3d,

nighthawk parser extracts uploaded zip file and looks into the folder. It has been hard-coded to look for audits on the parent folder or its sub-folder. If the audit files are deeper than that, parser will not look inside it.

As long as audit files are on parent directory or its sub folder it should work.

Can you give me details on what tools and directory structure you used to create the zip file, and I will look into it. You can email the samples to roshanmaskey@gmail.com.

cheers, roshan

xeen3d commented 8 years ago

Hi sorry for my late Answer.

i use nightHawk in a VM (VMware Fusion 8.1) on a Mac osx 10.11 I use Redline in actual Windows Version (late 2015) With that Redline i create a script and let it run against a test Windows 7 VM

The Result will be started in a shared Folder but that Folder is on my Mac OSx the Folder Structure is same than in Windows but if i zip the Session Folder like all Files in 20160707054834 to a 20160707054834.zip and upload that Zip nothing happens i try that a couple of Times with different Redline Scripts and different Windows Versions and different archive tools overtime same result.

Than i save the Redline Script output to a NTFS Windows Disk and do same and bam all was ok.

It looks that nightHawk can't use zip from OSx formatted Disks i feel that the resource forks was the real problem here i can upload after my daily work a sample File

best Andre

roshanmaskey commented 8 years ago

Andre,

Can you email me the sample archive created in OSX. I will test it for next release.

xeen3d commented 7 years ago

Hi, i send you here a small (very small) redline example zip and a picture what i have put in that zip file let me know if you need a bigger sample ok

ok here what i see on my mac folder structure (do i same in pure windows same pipeline works well)

and my Zip File created on Mac OSx 10.11

Let me know if you need more

best

Andre

Am 28.08.2016 um 15:06 schrieb roshanmaskey notifications@github.com:

Andre,

Can you email me the sample archive created in OSX. I will test it for next release.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

MfG

Andre Lauzon lauzona@xeen3d.de

xeen3d commented 7 years ago
screenshot 2016-08-30 di 23 30 56

Archiv.zip

xsallowed commented 7 years ago

Hi I am also having an error while uploading the files to nighthawk server and the error says that I am exceeding limit. Tried to fix the issue by modifying nginx config file but cannot see the variable client_max_body_size 500M. Just to avoid any confusion i could not find the the config file at location mentioned in second comment rather i found one @ /etc/nginx/conf.d/nighthawk.conf. please suggest if i am fixing it correctly or not?

xeen3d commented 7 years ago

Hi xallowed if you use Redline try switch off some Script parameters in Redline Script i do same Mistake here i try get all out of my Test VM i can so i configured Redline to catch all like Files hole Registry deleted Files and so far than i get out a very big Redline Output after Zip that it was round about 1,2 GB size. A real clear word here Night Hawk is a fine top tool and will be if sometime be ready for Working with it a great enhancement for us but i do many many tests and i use a OSx for running Night hawk VM i try in osx many Browsers like Safari Crome and Firefox every one of that get issues in Display the NightHawk site if the uploaded Files become very big.

My Tip here and i try that it works, make smaller Uploads name every Upload same Case Name with a Number at end like Case01,02,03 than you can import all you want and get all Information.

I do my first Import with Memory Dump and User Information a second one (case02) with File History a third one with hole Registry.

I be sure that NH sometime can handle such big Uploads but at this time NH is in my Eyes a cool testing Suite of Tools but not a Tool for daily Forensic work. NH is on a cool way i like it very much but is in development and i hope we all can see sometime a working ready Solution.

best Andre

xeen3d commented 7 years ago

Hi roshanmaskey i try now 103 if all was ok we can close that issue Andre

xeen3d commented 7 years ago

Hi all here, we can close this Ticket now, 103 works well i can import Files from OSx Disks and have issues with the Interface in my Browser. I do now some more tests but this Issue here is full removed and i am very happy about that. I open a new Ticket for Browser Issues i can´t get all NightHawk Features working in a Browser

Very good work many thank´s to all developers for fixing that in so short time

best

Andre

espressobeanies commented 7 years ago

FYI, the "client_max_body_size 500M" setting in Nighthawk 1.0.3 is now located in /etc/nginx/nginx.conf.