OtisOat commented 3 years ago

About accounts on capesandbox.com

Issues isn't the way to ask for account acctivation. Ping capesandbox in Twitter with your username

This is opensource and you getting free support so be friendly!

Free support from doomedraven ended, no whiskey no support. For something he updated the documentation :)

Prerequisites

Please answer the following questions for yourself before submitting an issue.

[x] I am running the latest version
[x] I checked the documentation and found no answer
[x] I checked to make sure that this issue has not already been filed
[x] I'm reporting the issue to the correct repository (for multi-repository projects)
[x] I'm have read all configs with all optional parts

Expected Behavior

Cape2 to finish processing the scanned file and generate a report or some event happening in the analysis machine.

cape2@ubuntu:~/Desktop$ file ChromeSetup.exe 
ChromeSetup.exe: PE32 executable (GUI) Intel 80386, for MS Windows

Current Behavior

Stuck in the pending stage from Web UI.

Failure Information (for bugs)

It looks like the process has exited with error code 1. However, the UI is still up and stuck in the pending stage.

cape2@ubuntu:~$ systemctl status cape
● cape.service - CAPE
     Loaded: loaded (/lib/systemd/system/cape.service; enabled; vendor preset: >
     Active: activating (auto-restart) (Result: exit-code) since Sat 2021-08-14>
       Docs: https://github.com/kevoreilly/CAPEv2
    Process: 927 ExecStart=/usr/bin/python3 cuckoo.py (code=exited, status=1/FA>
   Main PID: 927 (code=exited, status=1/FAILURE)

Aug 14 09:27:13 ubuntu systemd[1]: cape.service: Main process exited, code=exit>
Aug 14 09:27:13 ubuntu systemd[1]: cape.service: Failed with result 'exit-code'.
lines 1-9/9 (END)

Steps to Reproduce

Follow the steps here to install:
Installation recommendations and scripts for optimal performance
For best compability we strongly suggest installing on Ubuntu 20.04 LTS
KVM is recommended as hypervisor, replace <W00T> to real pattern
sudo ./kvm-qemu.sh all <username> | tee kvm-qemu.log
To install CAPE itself, cape2.sh with all optimizations
sudo ./cape2.sh base cape | tee cape.log
Reboot and enjoy

However, instead of using qemu, I have set up several VMs in the same custom VMNet.

Have installed the deps for Guest and changed the network configuration from: [https://capev2.readthedocs.io/en/latest/installation/guest_physical/requirements.html]
Have ensured that the VMs are able to ping each other (Firewall off)
Have ensured that the agent are listening to port 8000 by running netstat -ano

Changed cuckoo.conf


# Specify the name of the machinery module to use, this module will
# define the interaction between Cuckoo and your virtualization software
# of choice.
machinery = physical
...
[resultserver]
# The Result Server is used to receive in real time the behavioral logs
# produced by the analyzer.
# Specify the IP address of the host. The analysis machines should be able
# to contact the host through such address, so make sure it's valid.
# NOTE: if you set resultserver IP to 0.0.0.0 you have to set the option
# `resultserver_ip` for all your virtual machines in machinery configuration.
ip = 0.0.0.0

5. Changed physical.conf

[physical01]

Specify the label name of the current machine as specified in your

physical machine configuration.

label = physical01

Specify the operating system platform used by current machine

[windows/darwin/linux].

platform = windows

Specify the IP address of the current machine. Make sure that the IP address

is valid and that the host machine is able to reach it. If not, the analysis

will fail.

ip = 192.168.1.2

If you have not provided a resultserver in cuckoo.conf you can do it here

If you are using machinery "multi" you may have multiple networks connected to your CAPE host

and thus multiple possible ways for your analysis machines to connect so you can specify the

IP and port here

resultserver_ip = 192.168.1.1 resultserver_port = 2042


## Context

Currently, using the machines with VMware Workstation in an offline environment.
Guest machines DNS and Gateway are set to Cape Machine

| Question         | Answer
|------------------|--------------------
| Git commit       | <code> commit 5399fc958cf728cda1229d6795f1a3326f0609fb (HEAD -> master, origin/master, origin/HEAD) Author: doomedraven <doommedraven@gmail.com> Date:   Tue Aug 3 08:18:50 2021 +0200 Update views.py </code>
| OS version       | Ubuntu 20.04

## Failure Logs

Please include any relevant log snippets or files here.

doomedraven commented 3 years ago

once you open issue in one repo stay there with that issue. your steps to reproduce are wrong. why you just not run the command that cape.service runs? aka python3 cuckoo.py -d

doomedraven commented 3 years ago

and another question if you says vms but you set machinery = physical you have wrong configuration so far, you should set it to kvm

OtisOat commented 3 years ago

once you open issue in one repo stay there with that issue. your steps to reproduce are wrong. why you just not run the command that cape.service runs? aka python3 cuckoo.py -d

Thank you for helping to close the issue at another repo. As you have suggested, I decided to move the query here instead. After running with sudo python3 cuckoo.py -d, I am encountering with the following error: 2021-08-14 11:21:45,075 [root] CRITICAL: CuckooCriticalError: Physical machine credentials are missing, please add it to the config file

May I ask where I can insert the credentials?

Thank you @doomedraven for assisting :)

OtisOat commented 3 years ago

and another question if you says vms but you set machinery = physical you have wrong configuration so far, you should set it to kvm

I think I didnt explain properly above. What I am trying to achieve is on :https://capev2.readthedocs.io/en/latest/installation/guest_physical/saving.html#setup-using-vmware-bonus

I am not sure if have read the instructions properly and not much documentation are there on setting up using vmware. From what I understand, it should be physical as currently I have two VMs in the same custom vmnet. (1 is cape and 1 is host)

doomedraven commented 3 years ago

permissions looks ok for /opt/CAPEv2/data/yara/index_memory.yarc so im not really sure why it fails for you, i guess try to remove that file and see again.

you are so confused i guess, you installing KVM to run VMWARE/FOG, i also don't have the full picture what you trying to do %) you should use this if you using vms -> https://capev2.readthedocs.io/en/latest/installation/guest/index.html if you using FOG you should use https://capev2.readthedocs.io/en/latest/installation/guest_physical/saving.html

vmnet is vmware? if yes then you should go with vmware and configure it properly, if you want to use baremetal you should use FOG, if you going with vms and KVM use kvm,

about credentials, in config

OtisOat commented 3 years ago

From the link: https://capev2.readthedocs.io/en/latest/installation/guest_physical/saving.html#setup-using-vmware-bonus

Traditionally CAPE requires to be running some sort of virtualization software (e.g. KVM, VMware, Virtualbox, etc). The physical machine manager will also work with other virtual machines, so long as they are configured to revert to a snapshot on shutdown/reboot, and running the agent.py script. A use case for this functionality would be to run the CAPE server and the guest sandboxes each in their own virtual machine on a single host, allowing for development/testing of CAPE without requiring a dedicated Linux host.

I am trying to achieve the bolded part

OtisOat commented 3 years ago

For credentials, I am digging in the conf folder, but I am not sure which file I am supposed to edit to include the credentials for the physical setup or should it be kvm setup?

doomedraven commented 3 years ago

you are totally wrong, physical is baremetal boxes see how to setup, link from docs https://mariohenkel.medium.com/using-cape-sandbox-and-fog-to-analyze-malware-on-physical-machines-4dda328d4e2c

if you gonna use kvm you need to set kvm as that isn't physical, that is virtual machines inside of the physical box called server

OtisOat commented 3 years ago

A use case for this functionality would be to run the CAPE server and the guest sandboxes each in their own virtual machine on a  
 single host, allowing for development/testing of CAPE without requiring a dedicated Linux host.

Means for the above, and I am using VMWare, if I have two VMs (one is Cape and one is the guest machine). Note that: the guest machine is NOT in cape VM. I should be putting the configuration as vmware for my case?

doomedraven commented 3 years ago

i guess so, im not using vmware at all, you should be more familiar with your setup. there is some different tastes of vmware configs under conf folder. im clossing this issue as it not really cape issue is your setup, you might see which flavour of vmware you using and use proper config to manage that, but how that gonna work im not imagine, as networking routing using rooter won't be working for sure. so i can't really help here

OtisOat commented 3 years ago

I set physical because the page/guide I was trying to follow is park under https://capev2.readthedocs.io/en/latest/installation/**guest_physical**/saving.html#setup-using-vmware-bonus

I thought that my setup is the non-traditional way of setting up where the guest machines are not hosted in Cape. But outside of Cape so that each machine is separated as a different machine as stated in the installation guide.

If what I am trying to do is impossible (The non-traditional way): A use case for this functionality would be to run the CAPE server and the guest sandboxes each in their own virtual machine on a single host, allowing for development/testing of CAPE without requiring a dedicated Linux host.

I think should shift the link (https://capev2.readthedocs.io/en/latest/installation/guest_physical/saving.html#setup-using-vmware-bonus) to be out of guest_physical in the installation guide or at least not park it together with fog because it is a little misleading that it can be achieved with physical set up.

Will appreciate if you can re-open the issue and let @kevoreilly or the collaborator who wrote that part to answer my query. Thank you!

doomedraven commented 3 years ago

about docs could be, i didn't wrote that part so i can't mod what i didn't write. what you trying to do i guess is possible but you need to use proper version of vmware like vsphere something like that im not really vmware expect and im not using it at all, but as i told you there is diff configs for that so check them, but i can't help you here, you will need to figurate better which vmware are you using and how, but as i told having them in separate vms, if you fine to run all samples directly to internet or write custom firewall rules to route the traffic, if you looking toward testing enviroment, do not complicate your life, get one server that manages everything else and thats all

doomedraven commented 3 years ago

they can answer it anyway, but this isn't really cape issues, is issue that at least sound to me like you not sure how to setup it, and as i told you have few vmware flavours under conf folder so use proper one

OtisOat commented 3 years ago

about docs could be, i didn't wrote that part so i can't mod what i didn't write. what you trying to do i guess is possible but you need to use proper version of vmware like vsphere something like that im not really vmware expect and im not using it at all, but as i told you there is diff configs for that so check them, but i can't help you here, you will need to figurate better which vmware are you using and how, but as i told having them in separate vms, if you fine to run all samples directly to internet or write custom firewall rules to route the traffic, if you looking toward testing enviroment, do not complicate your life, get one server that manages everything else and thats all

@doomedraven Thank you for your help so far. I understand that you didnt write that part of the documentation. With it being closed, I am not sure they will look at this issue. Will appreciate if you can re-open the issue and let @kevoreilly or the collaborator who wrote that part to answer my query or close this issue.

As you have stated I have set up firewall rules to ensure that VMs are talking/listening to each other. However, I still cannot find the part where I need to enter the credential for CuckooCriticalError: Physical machine credentials are missing, please add it to the config file There isnt a part in physical.conf for me to fill in the credentials needed or the credentials are stored in a different location?

doomedraven commented 3 years ago

i have removed the part of vmware. i will open that for one week but if there no success i will close it

physical is FOG see https://github.com/kevoreilly/CAPEv2/issues/476

# TODO This should be moved to a per-machine thing.
        if not self.options.physical.user or not self.options.physical.password:
            raise CuckooCriticalError(
                "Physical machine credentials are missing, please add it to "
                "the Physical machinery configuration file."
            )

doomedraven commented 3 years ago

about physical creds, there is no code that throws that error, so that doesn't come from CAPE %) now im even more confused about your setup

OtisOat commented 3 years ago

@doomedraven Thank you re-opening this issue and helping out. I really appreciate it. I am trying to use VMs to stimulate physical machines in the same subnet. One of them will be Cape Server and the rest will be guest machines. I am not using FOG to control the VMs too.

The ideal will be from the Cape Web UI, choose the machinery (for e.g. machinery01) and upload the file for scanning. (The VM (machinery01) will be online and agent will be listening before uploading)

I am not sure if my setup is possible but I think it will be something new we can explore together

OtisOat commented 3 years ago

If it is possible, we can document it down and include it in the documentation and making Cape2 less resource intensive. For e.g. Supporting new VMs (separated) from CapeServer, Writing script to turn on the guest VM before using API to trigger Cape Server etc

doomedraven commented 3 years ago

if you not using FOG, forget about physical. and use proper flavour of vmware from conf folder. well we are not interested in vmware, due to malware detection, kvm is the best to beat the detection.

How having 2 vms will be less resource intensive? having cape on server and vms on it that is the light setup, when you put everything in vms, you wasting resources my personal point of view

doomedraven commented 3 years ago

please stop edit old msgs, i was writing response and i got aware of that but is easy to mess with what you writing

OtisOat commented 3 years ago

Oh I didnt know that you guys are not interested in vmware. Hmm, I thought it will be. For e.g. running Cape as the host with several guest machines. It will be limited by the resource that you allocated to the Cape VM. With testing on both the older and newer OS, the resource will just be expanding on 1 VM which will make it harder to move it in the end. (Like a 1TB VM compare to 1TB of several VMs)

doomedraven commented 3 years ago

cape runs on vmware just fine, but vmware is not the best hypervisor for malware analysis, and @kevoreilly can confirm that. you have kvm remote for that, but you have vmware remote, as i told you have many flavours of vmware supported, but as i told any network routing won't be handled by cape at all, so you will need to cover that part by yourself, im running cape cluster with many kvm servers(see distributed part) is much easier to scale horizontally than vertically and much cheaper. capesandbox.com is also in cluster mode

OtisOat commented 3 years ago

please stop edit old msgs, i was writing response and i got aware of that but is easy to mess with what you writing

Sorry, I was trying to add in more context with examples. Didnt edit the original content. I also noticed that you deleted the page "https://capev2.readthedocs.io/en/latest/installation/guest_physical/saving.html#setup-using-vmware-bonus" instead of moving it somewhere. I thought that it will be good if we can enhance on that page instead of removing it entirely.

doomedraven commented 3 years ago

yes i have removed that, as that just confusing people as you, and we don't want that, if you want to bring documentation for something you are more than welcome, but there is 4-5 flavours of vmware so that can confuse even more

OtisOat commented 3 years ago

cape runs on vmware just fine, but vmware is not the best hypervisor for malware analysis, and @kevoreilly can confirm that. you have kvm remote for that, but you have vmware remote, as i told you have many flavours of vmware supported, but as i told any network routing won't be handled by cape at all, so you will need to cover that part by yourself, im running cape cluster with many kvm servers(see distributed part) is much easier to scale horizontally than vertically and much cheaper. capesandbox.com is also in cluster mode

@doomedraven Thank you for assisting and answering my query. Oh didnt know that, vmware is not the best hypervisor for malware analysis. I guess I will try out with KVM cluster method. :) Thank you so much

doomedraven commented 3 years ago

you are welcome, i would suggest you, first get the normal setup, small, for playing arround if that isn't for your company, so you can try it and see if that fine for your needs(you can use https://capesandbox.com too to try it), but when you are not familiar how this works and start with networking complications, if you are not sysadmin that will be hard

OtisOat commented 3 years ago

Okay. I guess you are right. I will heed your advice to try out the normal setup and play around first. @doomedraven I would like to sincerely apologize to you if I have angered you in the process of my query. Really appreciate your help again in this matter.

doomedraven commented 3 years ago

hehe no im not anger at all :) i just feel you super confused what you trying to do, as i told if you see something confusing in docs, feel free to ping us or push improvements, but ya KVM is the best, and if you using my script don't forget to replace

kevoreilly commented 3 years ago

Hi folks looks like I am slow here but I do know a lot about running cape in a VMware vm and all that stuff. Problem is VMware is so bad from a malware detection perspective that I stopped using it completely a couple of years ago.

KVM on Linux is much more painful for a Windoze guy like me, but in the end the results are what matter and doomedraven's custom KVM is without parallel in the malware virtualization world. In the end the pain is worth it.

OtisOat commented 3 years ago

@doomedraven hahaha okay, thank you for your help!

I am trying out with KVM method as suggested but I cant seems to make the Host (Cape Server VM) to ping the guest machines (via KVM inside Cape Server VM) and setting up shared folder to transfer files between the host and guest machines. The IP issued in the guest machine seems to be incorrect from the range provided in KVM too.

GuestIP GuestVM-NIC-Settings

I also know there is not a right channel to ask about this kind of matters. May I ask if it is possible for me to contact you elsewhere? Like Slack or Glitter?

OtisOat commented 3 years ago

@kevoreilly Thank you for joining in. :) Thank you for your advice too.

doomedraven commented 3 years ago

@OtisOat IP inside of the vm should be set by you, as DHCP doesn't work for host only, and it useless here as you need to have static map of vm to ip

OtisOat commented 3 years ago

@doomedraven Oh didnt know need to set it ourselves. Thanks for all your help. :) Managed to run it with the recommended setup. Going to continue to play with it :)

doomedraven commented 3 years ago

glad taht you got it working, feel free to add clarification to the documentation that you think could help to all new users to start. so can we close this? you can keep posting here or open new issues if you want, i will get them in both cases

OtisOat commented 3 years ago

Here are some comments:

The two scripts cape2.sh and kvm-qemu,sh definitely help to speed up a lot of troubles in setting up both Cape and KVM.
I think can add in some debug commands for new users to run to also help to provide some context of the problem. Like virt-manager --debug & sudo python3 cuckoo.py -d & sudo service cape restart/stop
.To be more dummy/me proof, I think can add a line at https://www.doomedraven.com/2020/04/how-to-create-virtual-machine-with-virt.html (At Point 0. - To set static IP in the guest VM later with the given IP range)
Setting up Guest machine with https://github.com/doomedraven/Tools/blob/master/Windows/choco.bat. I also have a similar problem as https://mariohenkel.medium.com/using-cape-sandbox-and-fog-to-analyze-malware-on-physical-machines-4dda328d4e2c. Have to run the commands manually after installing the dependencies.

OtisOat commented 3 years ago

@doomedraven I would like to ask if you know about the reporting feature. I have tried to enable the configurations below. But under storage/analyses/X/ there are no reports folder generated. Am I missing some dependencies? There are no error messages from sudo python3 cuckoo.py -d too.


[reporthtml]
# required for the WSGI interface
enabled = yes

[reporthtmlsummary]
# much smaller, faster report generation, omits API logs and is non-interactive
enabled = yes

[reportpdf]
# Note that this requires reporthtmlsummary to be enabled above as well
enabled = yes

doomedraven commented 3 years ago

hello thanks for feedback, by parts :)

about debug points, well everything has -h, so that is users part, my script is done for production and problems that happens is due to users bad configuration or some mess on their side, so they can stop services and run everything by hand. that what you should do anyway when trying to spot the problem
did you read the cape docs? it says about static ip https://capev2.readthedocs.io/en/latest/installation/guest/network.html#virtual-networking
im aware of that but can't do much more, at least you just need copy&paste not search by yourself, the same with cape2.sh and kvm-qemu.sh as before everyone was walking docs page by page and installing everything. so here till user not installs proper deps, choco.bat won't work fine
about reporting, if you change config, you need to restart processing, in same cases cape and web, depends what you changing

OtisOat commented 3 years ago

hello :) thanks for all your help so far. :) I have restarted with two different ways sudo service cape restart and sudo service cape stop & sudo python3 cuckoo.py -d. But no reports are generated :(

doomedraven commented 3 years ago

that isn't processing, that is cape, you need to restart cape-processor, please become familiar with all cape services and what they do that will help you a lot

NEVER DO THIS!

sudo python3 cuckoo.py -d

you will break permissions on logs, and other stuff that can later give you more headache, forget about sudo for cape, only rooter service should be run as root

OtisOat commented 3 years ago

oh my bad. How do i restart cape-processor?

kevoreilly commented 3 years ago

sudo systemctl restart cape-processor

kevoreilly commented 3 years ago

The 4 cape services:

cape cape-web cape-processor cape-rooter

OtisOat commented 3 years ago

@kevoreilly thank you! Let me try :)

kevoreilly commented 3 years ago

I chime in for the easy bits :-D

OtisOat commented 3 years ago

@kevoreilly Thank you. It works by restarting cape-processor. I think both @doomedraven and you had put in lots of effort in this. It is already a blessing for both of you helping out

doomedraven commented 3 years ago

@OtisOat see readme i have updated it a bit to include some of that info https://github.com/kevoreilly/CAPEv2/blob/master/README.md

closing this as issue is solved

OtisOat commented 3 years ago

@doomedraven Looks good. :) Thank you for all your help :)

OtisOat commented 3 years ago

Can i also ask sth about my setup? Currently Cape VM has over 50GB of free space. However, I am encountering this error:

2021-08-24 03:12:05,458 [lib.cuckoo.core.scheduler] ERROR: Not enough free disk space! (Only 40353 MB!). You can change  
limits it in cuckoo.conf -> freespace

doomedraven commented 3 years ago

As msg says you can change that in that conf

El mar., 24 ago. 2021 5:13, OtisOat @.***> escribió:

Can i also ask sth about my setup? Currently Cape VM has over 50GB of free space. However, I am encountering this error:

2021-08-24 03:12:05,458 [lib.cuckoo.core.scheduler] ERROR: Not enough free disk space! (Only 40353 MB!). You can change limits it in cuckoo.conf -> freespace

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/kevoreilly/CAPEv2/issues/556#issuecomment-904288571, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAOFH33ZROMM5POESUSPVA3T6ME4TANCNFSM5CE55QJQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email .

OtisOat commented 3 years ago

@doomedraven Yup, I know. I can set it to lower or higher or even 0 to disable the check. But I was wondering why it is reporting no free space when there is over 50 GB of free space

kevoreilly / CAPEv2

32bit executable stuck in pending stage #556

About accounts on capesandbox.com

This is opensource and you getting free support so be friendly!

Prerequisites

Expected Behavior

Current Behavior

Failure Information (for bugs)

Steps to Reproduce

Specify the label name of the current machine as specified in your

physical machine configuration.

Specify the operating system platform used by current machine

[windows/darwin/linux].

Specify the IP address of the current machine. Make sure that the IP address

is valid and that the host machine is able to reach it. If not, the analysis

will fail.

If you have not provided a resultserver in cuckoo.conf you can do it here

If you are using machinery "multi" you may have multiple networks connected to your CAPE host

and thus multiple possible ways for your analysis machines to connect so you can specify the

IP and port here