cuckoosandbox / cuckoo

Cuckoo Sandbox is an automated dynamic malware analysis system
http://www.cuckoosandbox.org
Other
5.56k stars 1.7k forks source link

Routing to VPN not working #1280

Open seanthegeek opened 7 years ago

seanthegeek commented 7 years ago

I'm trying to get VPN routing working. I installed OpenVPN and resolvconf, configured and started an OpenVPN client connection at /etc/openvpn/us.conf:


dev tun0

remote [REDACTED] 1300
cipher AES-256-CBC

# Tunnel IPv6 traffic as well as IPv4
tun-ipv6

# Ignore routes
route-nopull

# Keep trying indefinitely to resolve the
# host name of the OpenVPN server.  Very useful
# on machines which are not permanently connected
# to the internet such as laptops.
resolv-retry infinite

# Most clients don't need to bind to
# a specific local port number.
nobind

# Try to preserve some state across restarts.
persist-key
persist-tun

# Enable compression on the VPN link.
comp-lzo

# Set log file verbosity.
verb 3

remote-cert-tls server

ping-restart 60

# Allow calling of built-in executables and user-defined scripts.
script-security 2

# Parses DHCP options from openvpn to update resolv.conf
up /etc/openvpn/update-resolv-conf
down /etc/openvpn/update-resolv-conf

ping 10

[REDACTED KEYS]

I configured Cuckoo's kvm.conf:

# By default we disable VPN support as it requires running utils/rooter.py as
# root next to cuckoo.py (which should run as regular user).
enabled = yes

# Comma-separated list of the available VPNs.
vpns = us

[us]
# Name of this VPN. The name is represented by the filepath to the
# configuration file, e.g., cuckoo would represent /etc/openvpn/cuckoo.conf
# Note that you can't assign the names "none" and "internet" as those would
# conflict with the routing section in cuckoo.conf.
name = us

# The description of this VPN which will be displayed in the web interface.
# Can be used to for example describe the country where this VPN ends up.
description = United States

# The tun device hardcoded for this VPN. Each VPN *must* be configured to use
# a hardcoded/persistent tun device by explicitly adding the line "dev tunX"
# to its configuration (e.g., /etc/openvpn/vpn1.conf) where X in tunX is a
# unique number between 0 and your lucky number of choice.
interface = tun0

# Routing table name/id for this VPN. If table name is used it *must* be
# added to /etc/iproute2/rt_tables as "<id> <name>" line (e.g., "201 tun0"). 
# ID and name must be unique across the system (refer /etc/iproute2/rt_tables
# for existing names and IDs).
rt_table = tun0

I also added 42 tun0 to /etc/iproute2/rt_tables, and started rooter.py, but the actions by rooter.py do not do not redirect the VPN's traffic. What could be going wrong.

doomedraven commented 7 years ago

hm everything looks fine, it works with curl --interface tun0 ipinfo.io/json?

seanthegeek commented 7 years ago

Hmmm

curl --interface tun0 ipinfo.io/json

Gives my actual IP info, not the one for the VPN

I wonder if my route-nopull (ignore pushed gateways) VPN client config option is preventing tun0 from being routed properly? Without that option, all server traffic is routed through the VPN, which I do not want.

doomedraven commented 7 years ago

hm i have default options in .ovpns and they works fine

client
dev tun1
proto udp
remote REPOTE ADDRESS HERE 1194
resolv-retry infinite
nobind
persist-key
persist-tun
persist-remote-ip
;ca TUNNEL HERE

comp-lzo
verb 3

<ca>
ca here
</ca>
seanthegeek commented 7 years ago

Ok, after a bit more research, trial, and error, I discovered that I had to remove the route-nopull and add a route command to the config for my local network in order to maintain local connectivity, and be able to access the server from my local network.

route 10.0.0.0 255.0.0.0 10.205.2.1 # Local network
route 192.168.100.0 255.255.255.0 192.168.200.1 # KVM virbr0 bridge

Now curl --interface tun0 ipinfo.io/json and utils/vpncheck.py each return the proper VPN IP information.

However, the VM still does not have access to the internet at all. The DNS request from a URL analysis is captured, but to reply is received.

Are you using the KVM backend?

doomedraven commented 7 years ago

weird, add print to router to see if all commands applied correctly, yes i'm use kvm

jbremer commented 7 years ago

I'll check & unit test soon'ish if VPN is still working correctly etc :) Thanks for feedback.

seanthegeek commented 7 years ago

@doomedraven I finally got around to doing more testing. Here is the output you asked for:

You are running a development version! Current stable is 2.0-rc1. INFO:cuckoo-rooter:Processing command: forward_drop
INFO:cuckoo-rooter:Processing command: forward_drop
INFO:cuckoo-rooter:Processing command: forward_drop
INFO:cuckoo-rooter:Processing command: nic_available tun0 INFO:cuckoo-rooter:Processing command: rt_available us.mullvad.net INFO:cuckoo-rooter:Processing command: nic_available tun0 INFO:cuckoo-rooter:Processing command: nic_available tun0 INFO:cuckoo-rooter:Processing command: disable_nat tun0 INFO:cuckoo-rooter:Processing command: rt_available us.mullvad.net INFO:cuckoo-rooter:Processing command: enable_nat tun0 INFO:cuckoo-rooter:Processing command: rt_available us.mullvad.net INFO:cuckoo-rooter:Processing command: disable_nat tun0 INFO:cuckoo-rooter:Processing command: flush_rttable us.mullvad.net INFO:cuckoo-rooter:Processing command: disable_nat tun0 INFO:cuckoo-rooter:Processing command: enable_nat tun0 INFO:cuckoo-rooter:Processing command: init_rttable us.mullvad.net tun0 INFO:cuckoo-rooter:Processing command: enable_nat tun0 INFO:cuckoo-rooter:Processing command: flush_rttable us.mullvad.net INFO:cuckoo-rooter:Processing command: flush_rttable us.mullvad.net INFO:cuckoo-rooter:Processing command: init_rttable us.mullvad.net tun0 INFO:cuckoo-rooter:Processing command: init_rttable us.mullvad.net tun0 INFO:cuckoo-rooter:Processing command: forward_drop
INFO:cuckoo-rooter:Processing command: nic_available tun0 INFO:cuckoo-rooter:Processing command: rt_available us.mullvad.net INFO:cuckoo-rooter:Processing command: disable_nat tun0 INFO:cuckoo-rooter:Processing command: enable_nat tun0 INFO:cuckoo-rooter:Processing command: flush_rttable us.mullvad.net INFO:cuckoo-rooter:Processing command: init_rttable us.mullvad.net tun0 INFO:cuckoo-rooter:Processing command: forward_drop
INFO:cuckoo-rooter:Processing command: nic_available tun0 INFO:cuckoo-rooter:Processing command: rt_available us.mullvad.net INFO:cuckoo-rooter:Processing command: disable_nat tun0 INFO:cuckoo-rooter:Processing command: enable_nat tun0 INFO:cuckoo-rooter:Processing command: flush_rttable us.mullvad.net INFO:cuckoo-rooter:Processing command: init_rttable us.mullvad.net tun0

017-02-28 19:48:07,585 [lib.cuckoo.core.scheduler] INFO: Using "kvm" as machine manager 2017-02-28 19:48:07,720 [lib.cuckoo.core.scheduler] INFO: Loaded 1 machine/s INFO:cuckoo-rooter:Processing command: forward_disable virbr0 tun0 192.168.100.42 2017-02-28 19:48:07,736 [lib.cuckoo.core.scheduler] INFO: Waiting for analysis tasks. 2017-02-28 19:49:15,137 [lib.cuckoo.core.scheduler] INFO: Starting analysis of URL "http://whatismyipaddress.com/" (task #10, options "route=us,procmemdump=yes") 2017-02-28 19:49:15,170 [lib.cuckoo.core.scheduler] INFO: Task #10: acquired machine win7 (label=win7) 2017-02-28 19:49:15,179 [modules.auxiliary.sniffer] INFO: Started sniffer with PID 3889 (interface=virbr0, host=192.168.100.42, pcap=/opt/cuckoo/storage/analyses/10/dump.pcap) INFO:cuckoo-rooter:Processing command: nic_available tun0 INFO:cuckoo-rooter:Processing command: forward_enable virbr0 tun0 192.168.100.42 INFO:cuckoo-rooter:Processing command: srcroute_enable us.mullvad.net 192.168.100.42

Here is my vpn.conf:

[vpn] enabled = yes

vpns = us

[us] name = us

description = United States

interface = tun0

rt_table = us.mullvad.net

I wasn't quite sure what is needed for the rt_table. I've also tried setting that to tun0 with no change.

Anything jump out?

jbremer commented 7 years ago

I'm going to have to take a quick look soon :-) Will keep you posted.. I think srcroute_enable/_disable are missing, but that's about all I can say right now. Also might be useful to clear pending iptables rules after running that stuff for a while with interrupts etc (otherwise there might still be some rules left).

doomedraven commented 7 years ago

rt_table is correct in vpn.conf, rt_table should be the same as in /etc/iproute2/rt_tables number:rt_table_name

check also ip route you should see something like X/24 dev tunX proto kernel scope link src X