Open zackph168 opened 5 months ago
d3vilh
commented
5 months ago Hi @zackph168
could you share docker logs openvpn and docker logs openvpn-ui with your docker-compose.yml file.
Seems you have issue with firewall of openvpn container.
Thanks!
zackph168
commented
5 months ago Sure, this is docker-compose.yml configure
---
version: "3.5"
services:
openvpn:
container_name: openvpn
# If you want to build your own image with docker-compose, uncomment the next line, comment the "image:" line and run "docker-compose build" following by "docker-compose up -d"
# build: .
image: d3vilh/openvpn-server:latest
privileged: true
ports:
- "12894:1194/udp" # openvpn UDP port
# - "1194:1194/tcp" # openvpn TCP port
# - "2080:2080/tcp" # management port. uncomment if you would like to share it with the host
environment:
TRUST_SUB: "10.0.70.0/24"
GUEST_SUB: "10.0.71.0/24"
HOME_SUB: "192.168.88.0/24"
volumes:
- ./pki:/etc/openvpn/pki
- ./clients:/etc/openvpn/clients
- ./config:/etc/openvpn/config
- ./staticclients:/etc/openvpn/staticclients
- ./log:/var/log/openvpn
- ./fw-rules.sh:/opt/app/fw-rules.sh
- ./checkpsw.sh:/opt/app/checkpsw.sh
- ./server.conf:/etc/openvpn/server.conf
cap_add:
- NET_ADMIN
restart: always
depends_on:
- "openvpn-ui"
openvpn-ui:
container_name: openvpn-ui
image: d3vilh/openvpn-ui:latest
environment:
- OPENVPN_ADMIN_USERNAME=admin
- OPENVPN_ADMIN_PASSWORD=gagaZush
privileged: true
ports:
- "8080:8080/tcp"
volumes:
- ./:/etc/openvpn
- ./db:/opt/openvpn-ui/db
- ./pki:/usr/share/easy-rsa/pki
- /var/run/docker.sock:/var/run/docker.sock:ro
restart: alwaysdocker logs -f openvpn:
EasyRSA path: /usr/share/easy-rsa OVPN path: /etc/openvpn
PKI already set up.
Following EASYRSA variables were set during CA init:
EASYRSA_DN "org"
EASYRSA_REQ_COUNTRY "UA"
EASYRSA_REQ_PROVINCE "KY"
EASYRSA_REQ_CITY "Kyiv"
EASYRSA_REQ_ORG "SweetHome"
EASYRSA_REQ_EMAIL "sweet@home.net"
EASYRSA_REQ_OU "MyOrganizationalUnit"
EASYRSA_REQ_CN "OpenVPNServer"
EASYRSA_KEY_SIZE 2048
EASYRSA_CA_EXPIRE 3650
EASYRSA_CERT_EXPIRE 825
EASYRSA_CERT_RENEW 30
EASYRSA_CRL_DAYS 180
Configuring networking rules...
net.ipv4.ip_forward = 1
net.ipv4.ip_forward = 1
net.ipv4.ip_forward = 1
net.ipv4.ip_forward = 1
net.ipv4.ip_forward = 1
Configuring iptables...
NAT for OpenVPN clients
Blocking ICMP for external clients
Blocking internal home subnet to access from external openvpn clients (Internet still available)
Applying firewall rules
Additional firewall rules applied.
IPT MASQ Chains:
# Warning: iptables-legacy tables present, use iptables-legacy to see them
MASQUERADE all -- 10.0.70.0/24 anywhere
MASQUERADE all -- 10.0.71.0/24 anywhere
IPT FWD Chains:
# Warning: iptables-legacy tables present, use iptables-legacy to see them
0 0 DROP 1 -- * * 10.0.71.0/24 0.0.0.0/0 icmptype 8
0 0 DROP 1 -- * * 10.0.71.0/24 0.0.0.0/0 icmptype 0
0 0 DROP 0 -- * * 10.0.71.0/24 192.168.88.0/24
Start openvpn process...docker logs -f openvpn-ui:
2024/05/04 04:34:53.811 [I] [certificates.go:139] No password and no ip
[ORM]2024/05/04 04:34:54 -[Queries/default] - [ OK / db.QueryRow / 0.1ms] - [SELECT `id`, `profile`, `easy_r_s_a_d_n`, `easy_r_s_a_req_country`, `easy_r_s_a_req_province`, `easy_r_s_a_req_city`, `easy_r_s_a_req_org`, `easy_r_s_a_req_email`, `easy_r_s_a_req_ou`, `easy_r_s_a_req_cn`, `easy_r_s_a_key_size`, `easy_r_s_a_ca_expire`, `easy_r_s_a_cert_expire`, `easy_r_s_a_cert_renew`, `easy_r_s_a_crl_days` FROM `easy_r_s_a_config` WHERE `profile` = ? ] - `default`
2024/05/04 04:34:54.197 [D] [utils.go:51] [
{
"EntryType": "V",
"Expiration": "260807041715Z",
"ExpirationT": "2026-08-07T04:17:15Z",
"IsExpiring": false,
"Revocation": "",
"RevocationT": "0001-01-01T00:00:00Z",
"Serial": "E889881BD43EB21684FB4A377ED6BC14",
"FileName": "unknown",
"Details": {
"Name": "",
"CN": "OpenVPNServer",
"Country": "UA",
"State": "KY",
"City": "Kyiv",
"Organisation": "SweetHome",
"OrganisationUnit": "MyOrganizationalUnit",
"Email": "sweet@home.net",
"LocalIP": "",
"TFAName": ""
}
},
{
"EntryType": "V",
"Expiration": "260807043454Z",
"ExpirationT": "2026-08-07T04:34:54Z",
"IsExpiring": false,
"Revocation": "",
"RevocationT": "0001-01-01T00:00:00Z",
"Serial": "382B865979B6C33171E6BCCB10CBC1F6",
"FileName": "unknown",
"Details": {
"Name": "ceshi1",
"CN": "ceshi1",
"Country": "UA",
"State": "KY",
"City": "Kyiv",
"Organisation": "SweetHome",
"OrganisationUnit": "MyOrganizationalUnit",
"Email": "sweet@home.net",
"LocalIP": "dynamic.pool",
"TFAName": "ceshi1"
}
}
]
[ORM]2024/05/04 04:34:54 -[Queries/default] - [ OK / db.QueryRow / 0.0ms] - [SELECT `id`, `profile`, `easy_r_s_a_d_n`, `easy_r_s_a_req_country`, `easy_r_s_a_req_province`, `easy_r_s_a_req_city`, `easy_r_s_a_req_org`, `easy_r_s_a_req_email`, `easy_r_s_a_req_ou`, `easy_r_s_a_req_cn`, `easy_r_s_a_key_size`, `easy_r_s_a_ca_expire`, `easy_r_s_a_cert_expire`, `easy_r_s_a_cert_renew`, `easy_r_s_a_crl_days` FROM `easy_r_s_a_config` WHERE `profile` = ? ] - `default`
[ORM]2024/05/04 04:34:54 -[Queries/default] - [ OK / db.QueryRow / 0.1ms] - [SELECT `id`, `profile`, `func_mode`, `device`, `server_address`, `port`, `resolve_retry`, `o_v_client_user`, `o_v_client_group`, `persist_tun`, `persist_key`, `remote_cert_t_l_s`, `open_vpn_server_port`, `proto`, `ca`, `cert`, `key`, `ta`, `cipher`, `redirect_gateway`, `auth`, `auth_no_cache`, `tls_client`, `verbose`, `auth_user_pass`, `t_f_a_issuer`, `custom_conf_one`, `custom_conf_two`, `custom_conf_three` FROM `o_v_client_config` WHERE `profile` = ? ] - `default`
[ORM]2024/05/04 04:34:54 -[Queries/default] - [ OK / db.QueryRow / 0.1ms] - [SELECT `id`, `login`, `is_admin`, `name`, `email`, `password`, `lastlogintime`, `created`, `updated` FROM `user` WHERE `id` = ? ] - `1`
2024/05/04 04:34:54.278 [I] [certificates.go:91] Image name: ceshi1
2024/05/04 04:34:54.278 [I] [certificates.go:95] Image path: /etc/openvpn/clients/ceshi1.png
[ORM]2024/05/04 04:35:17 -[Queries/default] - [ OK / db.QueryRow / 0.1ms] - [SELECT `id`, `login`, `is_admin`, `name`, `email`, `password`, `lastlogintime`, `created`, `updated` FROM `user` WHERE `id` = ? ] - `1`
[ORM]2024/05/04 04:35:17 -[Queries/default] - [ OK / db.QueryRow / 0.2ms] - [SELECT `id`, `profile`, `func_mode`, `device`, `server_address`, `port`, `resolve_retry`, `o_v_client_user`, `o_v_client_group`, `persist_tun`, `persist_key`, `remote_cert_t_l_s`, `open_vpn_server_port`, `proto`, `ca`, `cert`, `key`, `ta`, `cipher`, `redirect_gateway`, `auth`, `auth_no_cache`, `tls_client`, `verbose`, `auth_user_pass`, `t_f_a_issuer`, `custom_conf_one`, `custom_conf_two`, `custom_conf_three` FROM `o_v_client_config` WHERE `profile` = ? ] - `default`
[ORM]2024/05/04 04:35:17 -[Queries/default] - [ OK / db.QueryRow / 0.1ms] - [SELECT `id`, `profile`, `func_mode`, `management`, `script_security`, `user_pass_verify`, `device`, `port`, `proto`, `o_v_config_topology`, `keepalive`, `max_clients`, `o_v_config_user`, `o_v_config_group`, `o_v_config_client_config_dir`, `ifconfig_pool_persist`, `ca`, `cert`, `key`, `crl`, `dh`, `t_l_s_control_channel`, `t_l_s_min_version`, `t_l_s_remote_cert`, `cipher`, `o_v_config_ncp_ciphers`, `auth`, `server`, `route`, `push_route`, `d_n_s_server1`, `d_n_s_server2`, `redirect_g_w`, `o_v_config_logfile`, `o_v_config_log_verbose`, `o_v_config_status_log`, `o_v_config_status_log_version`, `custom_opt_one`, `custom_opt_two`, `custom_opt_three` FROM `o_v_config` WHERE `profile` = ? ] - `default`Hostserver iptables rules：
# Generated by iptables-save v1.4.21 on Sat May 4 04:51:30 2024
*nat
:PREROUTING ACCEPT [15169:908902]
:INPUT ACCEPT [15163:908491]
:OUTPUT ACCEPT [4:290]
:POSTROUTING ACCEPT [6:491]
:DOCKER - [0:0]
-A PREROUTING -m addrtype --dst-type LOCAL -j DOCKER
-A OUTPUT ! -d 127.0.0.0/8 -m addrtype --dst-type LOCAL -j DOCKER
-A POSTROUTING -s 172.19.0.0/16 ! -o br-3a4a8bc68322 -j MASQUERADE
-A POSTROUTING -s 172.17.0.0/16 ! -o docker0 -j MASQUERADE
-A POSTROUTING -s 172.19.0.2/32 -d 172.19.0.2/32 -p tcp -m tcp --dport 8080 -j MASQUERADE
-A POSTROUTING -s 172.19.0.3/32 -d 172.19.0.3/32 -p udp -m udp --dport 1194 -j MASQUERADE
-A DOCKER -i br-3a4a8bc68322 -j RETURN
-A DOCKER -i docker0 -j RETURN
-A DOCKER ! -i br-3a4a8bc68322 -p tcp -m tcp --dport 8080 -j DNAT --to-destination 172.19.0.2:8080
-A DOCKER ! -i br-3a4a8bc68322 -p udp -m udp --dport 12894 -j DNAT --to-destination 172.19.0.3:1194
COMMIT
# Completed on Sat May 4 04:51:30 2024
# Generated by iptables-save v1.4.21 on Sat May 4 04:51:30 2024
*filter
:INPUT ACCEPT [15292:918627]
:FORWARD DROP [0:0]
:OUTPUT ACCEPT [15253:616641]
:DOCKER - [0:0]
:DOCKER-ISOLATION-STAGE-1 - [0:0]
:DOCKER-ISOLATION-STAGE-2 - [0:0]
:DOCKER-USER - [0:0]
-A INPUT -p udp -m udp --dport 1194 -j ACCEPT
-A FORWARD -j DOCKER-USER
-A FORWARD -j DOCKER-ISOLATION-STAGE-1
-A FORWARD -o br-3a4a8bc68322 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -o br-3a4a8bc68322 -j DOCKER
-A FORWARD -i br-3a4a8bc68322 ! -o br-3a4a8bc68322 -j ACCEPT
-A FORWARD -i br-3a4a8bc68322 -o br-3a4a8bc68322 -j ACCEPT
-A FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -o docker0 -j DOCKER
-A FORWARD -i docker0 ! -o docker0 -j ACCEPT
-A FORWARD -i docker0 -o docker0 -j ACCEPT
-A DOCKER -d 172.19.0.2/32 ! -i br-3a4a8bc68322 -o br-3a4a8bc68322 -p tcp -m tcp --dport 8080 -j ACCEPT
-A DOCKER -d 172.19.0.3/32 ! -i br-3a4a8bc68322 -o br-3a4a8bc68322 -p udp -m udp --dport 1194 -j ACCEPT
-A DOCKER-ISOLATION-STAGE-1 -i br-3a4a8bc68322 ! -o br-3a4a8bc68322 -j DOCKER-ISOLATION-STAGE-2
-A DOCKER-ISOLATION-STAGE-1 -i docker0 ! -o docker0 -j DOCKER-ISOLATION-STAGE-2
-A DOCKER-ISOLATION-STAGE-1 -j RETURN
-A DOCKER-ISOLATION-STAGE-2 -o br-3a4a8bc68322 -j DROP
-A DOCKER-ISOLATION-STAGE-2 -o docker0 -j DROP
-A DOCKER-ISOLATION-STAGE-2 -j RETURN
-A DOCKER-USER -j RETURN
COMMIT
# Completed on Sat May 4 04:51:30 2024This is a fresh installation on a Centos 7 server, and it has the same issue. Even though I have enabled ipv4.forward, it still doesn't work...
Thank you!
d3vilh
commented
5 months ago By default clients lands to TRUST_SUB subnet, which is default in your configuration:
environment:
TRUST_SUB: "10.0.70.0/24"
GUEST_SUB: "10.0.71.0/24"
HOME_SUB: "192.168.88.0/24"When container start it apply SNAT(MASQUERADE) towards default internal container interface (eth0) for TRUST_SUB and GUEST_SUBs and this works fine, as per the logs:
IPT MASQ Chains:
# Warning: iptables-legacy tables present, use iptables-legacy to see them
MASQUERADE all -- 10.0.70.0/24 anywhere
MASQUERADE all -- 10.0.71.0/24 anywhere
IPT FWD Chains:
# Warning: iptables-legacy tables present, use iptables-legacy to see them
0 0 DROP 1 -- * * 10.0.71.0/24 0.0.0.0/0 icmptype 8
0 0 DROP 1 -- * * 10.0.71.0/24 0.0.0.0/0 icmptype 0
0 0 DROP 0 -- * * 10.0.71.0/24 192.168.88.0/24As you check sysctl net.ipv4.ip_forward and it returns 1 on both sides (server and container which should have it by default), then we should be good.
The Host server FW chains looks also legit and you don't have any additional rules there or containers except UI and openvpn-server.
Seems containervise we good and have to dig the session level.
Please share the openvpn.log from the server and client sides.
We also need to be sure that all redirect and push options from the client and server side matches your network:
Client side (client.ovpn):
redirect-gateway def1 - must be present in .ovpn file
Server side (server.conf):
push "route 10.0.60.0 255.255.255.0" - Route to Home VPN subnet
push "redirect-gateway def1 bypass-dhcp" - Redirect gateway for all VPN clients
Check version of openvpn-client you use to connect to the server (there were limitations for legacy openvpn-clients and new servers)?
Once client is connected do you see it in sessions list on OpenVPN-UI main page?
I would recommend to change HOME_SUB docker option to your local subnet, so clients with specific GUEST_SUB IPs will not be allowed to access it. But this is not related to our issue at all.
zackph168
commented
5 months ago In fact, after deploying and starting the containers, both my computer and phone can correctly connect to the services inside the containers from the host machine. I can also see my connection on the OpenVPN-UI dashboard you provided. The only issue is that the traffic requests from the TRUST_SUB network segment 10.0.70.0/24 cannot successfully forward from tun0 to the container's eth0. This involves IPtables NAT forwarding, which I don't fully understand. Maybe it's because both the host and the containers are nested with two layers of NAT forwarding?
I've tried to resolve this forwarding issue multiple times recently. When I don't use containers to deploy OpenVPN, the same forwarding rules work fine. However, the rules do not take effect when I use containers.
d3vilh
commented
4 months ago HI @zackph168
tun0 to eth0 SNAT should be part of OpenVPN container, you don't need to manually add any additional SNAT on the host side (Docker will take care).
For the host OpenVPN server is just application running on the desired port, not more.
TRUST_SUB and GUEST_SUB both are internal OpenVPN Server subnets, just to segregate clients to two groups, host does not know about this subnets.
On the container side we doing MASQUERADE (SNAT) from tun0 to eth0 container interface, all tun0 traffic will be forwarded to your host (for both subnets).
Means both subnes will have access to your local subnet as well. To prevent this for GUEST_SUB we have additional rules to drop any ICMP and TCP packets towards your HOME_SUB (that is why it is essential to define it).
If your clients have access to local subnet, then SNAT/MASQUERADE works as expected.
The internet access problem maybe related to DNS or routing announced to the client with push, route and redirect options.
In the session log we see following client options:
⏎[May 5, 2024, 19:21:21] OPTIONS:
0 [redirect-gateway] [def1]
1 [route] [10.0.60.0] [255.255.255.0]
2 [dhcp-option] [DNS] [8.8.8.8]
3 [dhcp-option] [DNS] [1.0.0.1]
4 [redirect-gateway] [def1] [bypass-dhcp]
5 [route-gateway] [10.0.70.1]
6 [topology] [subnet]
7 [ping] [10]
8 [ping-restart] [120]
9 [ifconfig] [10.0.70.3] [255.255.255.0]
10 [peer-id] [0]
11 [cipher] [AES-256-GCM]
12 [protocol-flags] [cc-exit] [tls-ekm] [dyn-tls-crypt]
13 [tun-mtu] [1500]Please check that:
10.0.60.0 is your local subnet, where host server lives.8.8.8.8 and 1.0.0.1 DNS are accessible for your client.You could change these parameters with UI or in server.conf directly.
After deploying the docker containers for openvpn-server and openvpn-ui, everything works normally up to the point of creating certificates and connecting to the openvpn server. However, after connecting to the server, I cannot access the external network, only internal network communication is possible.
Here is my connection log, which looks normal. Below are the container's routing table and iptables rules.
How can I view detailed client connection reports and observe the entire connection process?