Container not starting correctly since last update (?)

[alex171]

Since this morning, the container isn't reachable in my home network anymore. I tried some of your other containers (dyonr/sabnzbdvpn and dyonr/nzbgetvpn), all with minimal environment variables. I have VPN_ENABLED=yes, VPN_TYPE=openvpn, LAN_NETWORK=192.168.2.0/24 set. The dyonr/sabnzbdvpn container works as expected, i can reach the webinterface and ping domains/IPs from it. I cannot however ping domains from the dyonr/jackettvpn and the dyonr/nzbgetvpn containers. I can ping IPs. I know my VPN works, i do use all the same openvpn config files for the aforementioned containers. I think this is a DNS issue, could this be the recent update? I am running TrueNAS Scale Beta1. The Problems in dyonr/nzbgetvpn and dyonr/jackettvpn seem to be the same The container seems to be in a bootloop. If I set HEALTH_CHECK_HOST=1.1.1.1 the container keeps running, but isn't working (cannot resolve domainnames).

[DyonR]

Could you post some logs of the container?

[alex171]

here is the full log of an attempted start:

2021-08-17 16:08:46.934775+00:002021-08-17 18:08:46.934601 [INFO] VPN_ENABLED defined as 'yes' 2021-08-17 16:08:46.949111+00:002021-08-17 18:08:46.949033 [INFO] VPN_TYPE defined as 'openvpn' 2021-08-17 16:08:46.968717+00:002021-08-17 18:08:46.968626 [INFO] OpenVPN config file is found at /config/openvpn/cz2-ovpn-udp.ovpn 2021-08-17 16:08:46.973492+00:00dos2unix: converting file /config/openvpn/cz2-ovpn-udp.ovpn to Unix format... 2021-08-17 16:08:46.988891+00:002021-08-17 18:08:46.988828 [INFO] VPN remote line defined as 'cz2-ovpn-udp.pointtoserver.com 53' 2021-08-17 16:08:47.003644+00:002021-08-17 18:08:47.003582 [INFO] VPN_REMOTE defined as 'cz2-ovpn-udp.pointtoserver.com' 2021-08-17 16:08:47.018416+00:002021-08-17 18:08:47.018353 [INFO] VPN_PORT defined as '53' 2021-08-17 16:08:47.033137+00:002021-08-17 18:08:47.033075 [INFO] VPN_PROTOCOL defined as 'udp' 2021-08-17 16:08:47.048069+00:002021-08-17 18:08:47.048007 [INFO] VPN_DEVICE_TYPE defined as 'tun0' 2021-08-17 16:08:47.062854+00:002021-08-17 18:08:47.062782 [INFO] LAN_NETWORK defined as '192.168.2.0/24' 2021-08-17 16:08:47.077892+00:002021-08-17 18:08:47.077821 [WARNING] NAME_SERVERS not defined (via -e NAME_SERVERS), defaulting to CloudFlare and Google name servers 2021-08-17 16:08:47.092850+00:002021-08-17 18:08:47.092770 [INFO] VPN_OPTIONS not defined (via -e VPN_OPTIONS) 2021-08-17 16:08:47.107949+00:002021-08-17 18:08:47.107882 [INFO] Adding 1.1.1.1 to resolv.conf 2021-08-17 16:08:47.123142+00:002021-08-17 18:08:47.123069 [INFO] Adding 8.8.8.8 to resolv.conf 2021-08-17 16:08:47.151368+00:002021-08-17 18:08:47.151260 [INFO] Adding 1.0.0.1 to resolv.conf 2021-08-17 16:08:47.167745+00:002021-08-17 18:08:47.167650 [INFO] Adding 8.8.4.4 to resolv.conf 2021-08-17 16:08:47.188536+00:002021-08-17 18:08:47.188382 [INFO] Starting OpenVPN... 2021-08-17 16:08:47.214184+00:002021-08-17 18:08:47 DEPRECATED OPTION: --cipher set to 'AES-256-CBC' but missing in --data-ciphers (AES-256-GCM:AES-128-GCM). Future OpenVPN version will ignore --cipher for cipher negotiations. Add 'AES-256-CBC' to --data-ciphers or change --cipher 'AES-256-CBC' to --data-ciphers-fallback 'AES-256-CBC' to silence this warning. 2021-08-17 16:08:47.214333+00:002021-08-17 18:08:47 WARNING: file 'credentials.conf' is group or others accessible 2021-08-17 16:08:47.214355+00:002021-08-17 18:08:47 OpenVPN 2.5.1 x86_64-pc-linux-gnu [SSL (OpenSSL)] [LZO] [LZ4] [EPOLL] [PKCS11] [MH/PKTINFO] [AEAD] built on May 14 2021 2021-08-17 16:08:47.214376+00:002021-08-17 18:08:47 library versions: OpenSSL 1.1.1k 25 Mar 2021, LZO 2.10 2021-08-17 16:08:47.283195+00:002021-08-17 18:08:47 TCP/UDP: Preserving recently used remote address: [AF_INET]xxx.xxx.xxx.xxx:53 2021-08-17 16:08:47.283225+00:002021-08-17 18:08:47 UDP link local: (not bound) 2021-08-17 16:08:47.283250+00:002021-08-17 18:08:47 UDP link remote: [AF_INET]xxx.xxx.xxx.xxx:53 2021-08-17 16:08:47.456484+00:002021-08-17 18:08:47 WARNING: 'link-mtu' is used inconsistently, local='link-mtu 1557', remote='link-mtu 1550' 2021-08-17 16:08:47.456521+00:002021-08-17 18:08:47 WARNING: 'auth' is used inconsistently, local='auth SHA1', remote='auth [null-digest]' 2021-08-17 16:08:47.456546+00:002021-08-17 18:08:47 WARNING: 'comp-lzo' is present in remote config but missing in local config, remote='comp-lzo' 2021-08-17 16:08:47.456556+00:002021-08-17 18:08:47 [Secure-Server] Peer Connection Initiated with [AF_INET]xxx.xxx.xxx.xxx:53 2021-08-17 16:08:48.621494+00:002021-08-17 18:08:48 TUN/TAP device tun0 opened 2021-08-17 16:08:48.621570+00:002021-08-17 18:08:48 net_iface_mtu_set: mtu 1500 for tun0 2021-08-17 16:08:48.621584+00:002021-08-17 18:08:48 net_iface_up: set tun0 up 2021-08-17 16:08:48.621594+00:002021-08-17 18:08:48 net_addr_v4_add: xxx.xxx.xxx.xxx/28 dev tun0 2021-08-17 16:08:48.621694+00:002021-08-17 18:08:48 WARNING: this configuration may cache passwords in memory -- use the auth-nocache option to prevent this 2021-08-17 16:08:48.621767+00:002021-08-17 18:08:48 Initialization Sequence Completed 2021-08-17 16:08:49.362005+00:002021-08-17 18:08:49.361871 [INFO] Docker network defined as 172.16.0.0/16 2021-08-17 16:08:49.415364+00:002021-08-17 18:08:49.415238 [INFO] Adding 192.168.2.0/24 as route via docker eth0 2021-08-17 16:08:49.416982+00:00Error: Nexthop has invalid gateway. 2021-08-17 16:08:49.449901+00:002021-08-17 18:08:49.449783 [INFO] ip route defined as follows... 2021-08-17 16:08:49.450541+00:00-------------------- 2021-08-17 16:08:49.451653+00:000.0.0.0/1 via xxx.xxx.xxx.xxx dev tun0 2021-08-17 16:08:49.451678+00:00default via xxx.xxx.xxx.xxx dev tun0 2021-08-17 16:08:49.451707+00:00128.0.0.0/1 via xxx.xxx.xxx.xxx dev tun0 2021-08-17 16:08:49.451717+00:00xxx.xxx.xxx.xxx/16 dev eth0 proto kernel scope link src xxx.xxx.xxx.xxx 2021-08-17 16:08:49.451725+00:00xxx.xxx.xxx.xxx/28 dev tun0 proto kernel scope link src xxx.xxx.xxx.xxx 2021-08-17 16:08:49.451733+00:00xxx.xxx.xxx.xxx via 172.16.0.1 dev eth0 2021-08-17 16:08:49.451855+00:00-------------------- 2021-08-17 16:08:49.457858+00:00iptable_mangle 16384 1 2021-08-17 16:08:49.457880+00:00ip_tables 32768 8 iptable_filter,iptable_nat,iptable_mangle 2021-08-17 16:08:49.457889+00:00x_tables 53248 18 ip6table_filter,xt_conntrack,iptable_filter,nft_compat,xt_multiport,xt_tcpudp,xt_addrtype,xt_nat,xt_ipvs,xt_comment,xt_set,ip6_tables,ipt_REJECT,ip_tables,ip6table_mangle,xt_MASQUERADE,iptable_mangle,xt_mark 2021-08-17 16:08:49.478460+00:002021-08-17 18:08:49.478364 [INFO] iptable_mangle support detected, adding fwmark for tables 2021-08-17 16:08:49.666897+00:002021-08-17 18:08:49.666763 [INFO] iptables defined as follows... 2021-08-17 16:08:49.667539+00:00-------------------- 2021-08-17 16:08:49.669499+00:00-P INPUT DROP 2021-08-17 16:08:49.669522+00:00-P FORWARD ACCEPT 2021-08-17 16:08:49.669532+00:00-P OUTPUT DROP 2021-08-17 16:08:49.669540+00:00-A INPUT -i tun0 -j ACCEPT 2021-08-17 16:08:49.669548+00:00-A INPUT -s xxx.xxx.0.0/16 -d xxx.xxx.0.0/16 -j ACCEPT 2021-08-17 16:08:49.669559+00:00-A INPUT -i eth0 -p udp -m udp --sport 53 -j ACCEPT 2021-08-17 16:08:49.669592+00:00-A INPUT -i eth0 -p tcp -m tcp --dport 9117 -j ACCEPT 2021-08-17 16:08:49.669605+00:00-A INPUT -i eth0 -p tcp -m tcp --sport 9117 -j ACCEPT 2021-08-17 16:08:49.669616+00:00-A INPUT -p icmp -m icmp --icmp-type 0 -j ACCEPT 2021-08-17 16:08:49.669627+00:00-A INPUT -i lo -j ACCEPT 2021-08-17 16:08:49.669652+00:00-A OUTPUT -o tun0 -j ACCEPT 2021-08-17 16:08:49.669665+00:00-A OUTPUT -s xxx.xxx.0.0/16 -d xxx.xxx.0.0/16 -j ACCEPT 2021-08-17 16:08:49.669675+00:00-A OUTPUT -o eth0 -p udp -m udp --dport 53 -j ACCEPT 2021-08-17 16:08:49.669685+00:00-A OUTPUT -o eth0 -p tcp -m tcp --dport 9117 -j ACCEPT 2021-08-17 16:08:49.669703+00:00-A OUTPUT -o eth0 -p tcp -m tcp --sport 9117 -j ACCEPT 2021-08-17 16:08:49.669711+00:00-A OUTPUT -p icmp -m icmp --icmp-type 8 -j ACCEPT 2021-08-17 16:08:49.669719+00:00-A OUTPUT -o lo -j ACCEPT 2021-08-17 16:08:49.669730+00:00-------------------- 2021-08-17 16:08:49.706467+00:002021-08-17 18:08:49.706348 [INFO] A group with PGID 1000 does not exist, adding a group called 'jackett' with PGID 1000 2021-08-17 16:08:49.746480+00:002021-08-17 18:08:49.746379 [INFO] An user with PUID 1000 does not exist, adding an user called 'jackett user' with PUID 1000 2021-08-17 16:08:49.788662+00:002021-08-17 18:08:49.788542 [WARNING] UMASK not defined (via -e UMASK), defaulting to '002' 2021-08-17 16:08:49.862649+00:002021-08-17 18:08:49.862556 [INFO] Generating password hash 2021-08-17 16:08:49.909290+00:002021-08-17 18:08:49.909204 [INFO] Password hashes match, nothing to change. 2021-08-17 16:08:49.923335+00:002021-08-17 18:08:49.923256 [INFO] Starting Jackett daemon... 2021-08-17 16:08:49.927920+00:00Logging to /config/Jackett/Logs/log.txt. 2021-08-17 16:08:50.956863+00:002021-08-17 18:08:50.956720 [INFO] Jackett PID: 227 2021-08-17 16:08:50.957275+00:002021-08-17 18:08:50.957177 [INFO] Started Jackett daemon successfully... 2021-08-17 16:09:47.038785+00:002021-08-17 18:09:47.038621 [ERROR] Network is down, exiting this Docker

edit, after this, it gets stuck in an infinite loop of this:

2021-08-17 17:09:20.116399+00:002021-08-17 19:09:20.116252 [INFO] VPN_ENABLED defined as 'yes' 2021-08-17 17:09:20.130857+00:002021-08-17 19:09:20.130763 [INFO] VPN_TYPE defined as 'openvpn' 2021-08-17 17:09:20.148766+00:002021-08-17 19:09:20.148669 [INFO] OpenVPN config file is found at /config/openvpn/cz2-ovpn-udp.ovpn 2021-08-17 17:09:20.150222+00:00dos2unix: converting file /config/openvpn/cz2-ovpn-udp.ovpn to Unix format... 2021-08-17 17:09:20.165280+00:002021-08-17 19:09:20.165210 [INFO] VPN remote line defined as 'cz2-ovpn-udp.pointtoserver.com 53' 2021-08-17 17:09:20.180242+00:002021-08-17 19:09:20.180168 [INFO] VPN_REMOTE defined as 'cz2-ovpn-udp.pointtoserver.com' 2021-08-17 17:09:20.195259+00:002021-08-17 19:09:20.195191 [INFO] VPN_PORT defined as '53' 2021-08-17 17:09:20.210548+00:002021-08-17 19:09:20.210477 [INFO] VPN_PROTOCOL defined as 'udp' 2021-08-17 17:09:20.225315+00:002021-08-17 19:09:20.225244 [INFO] VPN_DEVICE_TYPE defined as 'tun0' 2021-08-17 17:09:20.240060+00:002021-08-17 19:09:20.239982 [INFO] LAN_NETWORK defined as '192.168.2.0/24' 2021-08-17 17:09:20.267969+00:002021-08-17 19:09:20.267861 [WARNING] NAME_SERVERS not defined (via -e NAME_SERVERS), defaulting to CloudFlare and Google name servers 2021-08-17 17:09:20.283656+00:002021-08-17 19:09:20.283579 [INFO] VPN_OPTIONS not defined (via -e VPN_OPTIONS) 2021-08-17 17:09:20.312149+00:002021-08-17 19:09:20.312039 [INFO] Adding 1.1.1.1 to resolv.conf 2021-08-17 17:09:20.340666+00:002021-08-17 19:09:20.340567 [INFO] Adding 8.8.8.8 to resolv.conf 2021-08-17 17:09:20.357673+00:002021-08-17 19:09:20.357571 [INFO] Adding 1.0.0.1 to resolv.conf 2021-08-17 17:09:20.390786+00:002021-08-17 19:09:20.390648 [INFO] Adding 8.8.4.4 to resolv.conf 2021-08-17 17:09:20.414605+00:002021-08-17 19:09:20.414513 [INFO] PUID not defined. Defaulting to root user 2021-08-17 17:09:20.429086+00:002021-08-17 19:09:20.428941 [INFO] PGID not defined. Defaulting to root group 2021-08-17 17:09:20.457204+00:002021-08-17 19:09:20.457076 [INFO] Starting OpenVPN... 2021-08-17 17:09:20.470559+00:002021-08-17 19:09:20 DEPRECATED OPTION: --cipher set to 'AES-256-CBC' but missing in --data-ciphers (AES-256-GCM:AES-128-GCM). Future OpenVPN version will ignore --cipher for cipher negotiations. Add 'AES-256-CBC' to --data-ciphers or change --cipher 'AES-256-CBC' to --data-ciphers-fallback 'AES-256-CBC' to silence this warning. 2021-08-17 17:09:20.470703+00:002021-08-17 19:09:20 WARNING: file 'credentials.conf' is group or others accessible 2021-08-17 17:09:20.470733+00:002021-08-17 19:09:20 OpenVPN 2.5.1 x86_64-pc-linux-gnu [SSL (OpenSSL)] [LZO] [LZ4] [EPOLL] [PKCS11] [MH/PKTINFO] [AEAD] built on May 14 2021 2021-08-17 17:09:20.470753+00:002021-08-17 19:09:20 library versions: OpenSSL 1.1.1k 25 Mar 2021, LZO 2.10 2021-08-17 17:09:20.473512+00:002021-08-17 19:09:20 RESOLVE: Cannot resolve host address: cz2-ovpn-udp.pointtoserver.com:53 (Temporary failure in name resolution) 2021-08-17 17:09:20.473577+00:002021-08-17 19:09:20 RESOLVE: Cannot resolve host address: cz2-ovpn-udp.pointtoserver.com:53 (Temporary failure in name resolution) 2021-08-17 17:09:20.473585+00:002021-08-17 19:09:20 Could not determine IPv4/IPv6 protocol 2021-08-17 17:09:20.473631+00:002021-08-17 19:09:20 SIGUSR1[soft,init_instance] received, process restarting 2021-08-17 17:09:25.474059+00:002021-08-17 19:09:25 RESOLVE: Cannot resolve host address: cz2-ovpn-udp.pointtoserver.com:53 (Temporary failure in name resolution) 2021-08-17 17:09:25.474145+00:002021-08-17 19:09:25 RESOLVE: Cannot resolve host address: cz2-ovpn-udp.pointtoserver.com:53 (Temporary failure in name resolution) 2021-08-17 17:09:25.474211+00:002021-08-17 19:09:25 Could not determine IPv4/IPv6 protocol 2021-08-17 17:09:25.474226+00:002021-08-17 19:09:25 SIGUSR1[soft,init_instance] received, process restarting 2021-08-17 17:09:30.474548+00:002021-08-17 19:09:30 RESOLVE: Cannot resolve host address: cz2-ovpn-udp.pointtoserver.com:53 (Temporary failure in name resolution) 2021-08-17 17:09:30.474635+00:002021-08-17 19:09:30 RESOLVE: Cannot resolve host address: cz2-ovpn-udp.pointtoserver.com:53 (Temporary failure in name resolution) 2021-08-17 17:09:30.474648+00:002021-08-17 19:09:30 Could not determine IPv4/IPv6 protocol 2021-08-17 17:09:30.474670+00:002021-08-17 19:09:30 SIGUSR1[soft,init_instance] received, process restarting 2021-08-17 17:09:35.475090+00:002021-08-17 19:09:35 RESOLVE: Cannot resolve host address: cz2-ovpn-udp.pointtoserver.com:53 (Temporary failure in name resolution) 2021-08-17 17:09:35.475171+00:002021-08-17 19:09:35 RESOLVE: Cannot resolve host address: cz2-ovpn-udp.pointtoserver.com:53 (Temporary failure in name resolution) 2021-08-17 17:09:35.475204+00:002021-08-17 19:09:35 Could not determine IPv4/IPv6 protocol 2021-08-17 17:09:35.475253+00:002021-08-17 19:09:35 SIGUSR1[soft,init_instance] received, process restarting 2021-08-17 17:09:40.475608+00:002021-08-17 19:09:40 RESOLVE: Cannot resolve host address: cz2-ovpn-udp.pointtoserver.com:53 (Temporary failure in name resolution) 2021-08-17 17:09:40.475672+00:002021-08-17 19:09:40 RESOLVE: Cannot resolve host address: cz2-ovpn-udp.pointtoserver.com:53 (Temporary failure in name resolution) 2021-08-17 17:09:40.475700+00:002021-08-17 19:09:40 Could not determine IPv4/IPv6 protocol 2021-08-17 17:09:40.475709+00:002021-08-17 19:09:40 SIGUSR1[soft,init_instance] received, process restarting

[alex171]

ok it did work for a moment there, but now it doesn't. I dug deeper and honestly, I'm a bit out of my league here. There seems to be a problem with the DEFAULT_GATEWAY variable. If I use your code in the jackettvpn container ip -4 route list 0/0 | cut -d ' ' -f 3 it gives me some address from the VPN (188.something) and if I do the same thing in the sabnzbdvpn container, it gives me a valid docker ip(172.16.0.1). The wrong gateway gives the Error:

Error: Nexthop has invalid gateway.

If I do a manual ip route add 192.168.2.0/24 via 172.16.0.1 dev eth0 there is no error. But container still isn't reachable from the home network and the container cannot ping anything either.

About the problem from above: As the container cannot ping anything in my network, it cannot speak to the local dns server (pihole), which the container got from the kubernetes default dns settings. I have overridden this, so it is online now.

[alex171]

I did finally roll back to the latest (for me) working state https://github.com/DyonR/docker-Jackettvpn/tree/8723449893b1fcf29877e69b95677b153c12a7b1 I built it, and added it to the dockerhub. For anyone with the same problem (web interface not responding/reachable), you can use this for the time being: https://hub.docker.com/r/alex171/jackettvpn. @DyonR if thats a problem, please tell me, then i can set the dockerhub repo to private.

[DyonR]

Hmmm, so the new stable version of Debian, bullseye, seems to have caused to problem. Very weird.
Good to see you figured it out though!

Of course you can have a copy of my Docker on Docker Hub that you have! It's licensed under the GPL license which allows commercial use, modification, distribution, use in patents or private use.

[alex171]

Yeah, seems that way! Very weird, I thought so too. Does it work for you though? I tested it in a ubuntu vm and got the same result. When I changed the route from inside the container, it worked. But not in TrueNAS Scale..

[DyonR]

Yeah, it works fine for me, I test all my containers on Unraid, which is the system I make them for, so I won't publish untested container. I can't test all systems like DSM (Synology), TrueNAS, Portainers, standalone Docker, nevertheless I try to help 😄
Maybe it is related to the kernel or Docker version? I have no idea.
My kernel version is 5.10.28-Unraid (uname -r)
My Docker version is Docker version 20.10.5, build 55c4c88 (docker -v)

I can also make a dyonr/jackettvpn:buster tag (buster being Debian 10), which you can use, I can also update that one whenever Jackett has an update mine can update too.

Edit: dyonr/jackettvpn:buster is live!
https://hub.docker.com/r/dyonr/jackettvpn/tags?page=1&ordering=last_updated
https://github.com/DyonR/docker-Jackettvpn/tree/buster

[alex171]

thanks mate, thats a great fix :) I'm running 5.10.42+truenas and Docker version 20.10.6, build 370c289.. Maybe it's just the kubernetes thing in TrueNAS or just the fact, that it is still in Beta and there are other bugs. *edit: buster tag works like a charm

[ThaDaVos]

I am having the same issue, is constantly says Network is down, exiting this Docker for me and it won't keep running - I tried the buster tag too but didn't solve my issue - I updated my UnRaid to the latest 6.10.0-rc1 to check if that solves it but also does not... (I was on 6.9.2) - what do you need from me to help you solve it?

[alex171]

hey @ThaDaVos , you can, in the meantime, try the container I made here. It's based on debian buster, everything else is the same. Worked for me. Some update killed the container for me again, so I switched back to the one I made.

[DyonR]

I am having the same issue, is constantly says Network is down, exiting this Docker for me and it won't keep running - I tried the buster tag too but didn't solve my issue - I updated my UnRaid to the latest 6.10.0-rc1 to check if that solves it but also does not... (I was on 6.9.2) - what do you need from me to help you solve it?

Can you perhaps open a new issue which also included the log file? Make sure to remove any sensitive info.
And also try the one of @alex171

[ThaDaVos]

Sorry for not commenting again... but the issue seems to be solved for me, I replaced my OpenVPN file, fixed my default gateway setting in UnRaid and it works again... - I entered my Gateway with leading zero's, so xxx.xxx.xxx.xxx instead of x.x.x.x - apparently this causes issues in the latest UnRaid with the Community Apps plugin too for DNS stuff

[theyogendraDanwar]

@alex171 i am getting an issue related to iptables in this docker container https://hub.docker.com/r/alex171/jackettvpn @DyonR added a fix for this couple of days ago will that be possible to add those changes in your docker as well i have created a ticket for this https://github.com/DyonR/docker-Jackettvpn/issues/55

[alex171]

@theyogendraDanwar you can try the update i pushed, see if it works for you.