yugabyte / yugabyte-db

YugabyteDB - the cloud native distributed SQL database for mission-critical applications.
https://www.yugabyte.com
Other
8.88k stars 1.05k forks source link

[Platform] During universe deployment, pre-flight network check fails when ICMP is blocked #9103

Open cigoldstein opened 3 years ago

cigoldstein commented 3 years ago

When attempting to deploy a universe in 2.4.5, the network check will fail if ICMP is blocked on the network. We can see below that it fails stating that "Internet Connection": false:

Failed to execute task java.util.concurrent.FutureTask@373cec36, hit error java.lang.RuntimeException: 
-----
Node <redacted nodename>failed preflight checks:
{
  "(Prometheus) /opt/prometheus is writable": true, 
  "(Prometheus) No Pre-existing Node Exporter Running": true, 
  "Internet Connection": false,
  "SSH Connection": true, 
  "(Prometheus) /var/run/prometheus is writable": true, 
  "(Prometheus) /etc/prometheus is writable": true, 
  "Sudo Access to Python": true, 
  "(Prometheus) /lib/systemd/system/node_exporter.service is writable": true, 
  "Yugabyte User in Yugabyte Group": true, 
  "(PAM Limits) /etc/security/limits.conf is writable": true, 
  "Try Ansible Command": true, 
  "(Mount Point) /app is writable": true, 
  "(Prometheus) /var/log/prometheus is writable": true, 
  "(Prometheus) /var/lib/prometheus is writable": true
}
-----

Ping test to yugabyte.com fails on this network (expected, since ICMP is blocked):

[root@<redacted> ~]# ping www.yugabyte.com
PING www.yugabyte.com (104.26.4.123) 56(84) bytes of data.
^C
--- www.yugabyte.com ping statistics ---
4 packets transmitted, 0 received, 100% packet loss, time 2999ms

Network connectivity works on 80/443:

[root@<redacted> ~]# nc -v www.yugabyte.com 80
Ncat: Version 7.50 ( https://nmap.org/ncat )
Ncat: Connected to 104.26.5.123:80.
^C

[root@<redacted> ~]# nc -v www.yugabyte.com 443
Ncat: Version 7.50 ( https://nmap.org/ncat )
Ncat: Connected to 172.67.73.218:443.
RunningJon commented 3 years ago

Maybe a more appropriate check would be a curl which uses TCP rather than ping which uses ICMP.

tylarb commented 3 years ago

Linked PR - https://github.com/yugabyte/yugabyte-db/pull/9133

ashu0008 commented 2 years ago

I see internet check is implemented using tcp check.

but some host may not open connection to /dev/tcp , which will cause installation to fail. I faced this issue in 2.8.1. So I had to remove internet check part from the preflight script.