search

GoogleCloudDataproc / bdutil

[DEPRECATED] Script used to manage Hadoop and Spark instances on Google Compute Engine
https://cloud.google.com/dataproc
Apache License 2.0
109 stars 94 forks source link

works with centos-6 but not centos-7 #22

Open bryan4887 opened 9 years ago

bryan4887 commented 9 years ago

This fails with centos-7 but not centos-6.

./bdutil -e platforms/hdp/ambari_env.sh deploy
Sat Mar 21 23:29:19 CST 2015: Using local tmp dir for staging files: /tmp/bdutil-20150321-232919-pZT
Sat Mar 21 23:29:19 CST 2015: Using custom environment-variable file(s): bdutil_env.sh platforms/hdp/ambari_env.sh
Sat Mar 21 23:29:19 CST 2015: Reading environment-variable file: ./bdutil_env.sh
Sat Mar 21 23:29:19 CST 2015: Reading environment-variable file: platforms/hdp/ambari_env.sh
Importing dependent env file: ./platforms/hdp/ambari_manual_env.sh
Importing dependent env file: ./hadoop2_env.sh
Importing dependent env file: ./platforms/hdp/ambari.conf
Importing dependent env file: ./platforms/hdp/ambari_functions.sh
Sat Mar 21 23:29:19 CST 2015: No explicit GCE_MASTER_MACHINE_TYPE provided; defaulting to value of GCE_MACHINE_TYPE: n1-standard-4
Deploy cluster with following settings?
      CONFIGBUCKET='net-xbhxxxxx'
      PROJECT='zeta-matrixxxxxxx'
      GCE_IMAGE='centos-7'
      GCE_ZONE='europe-west1-b'
      GCE_NETWORK='default'
      PREFIX='hadoop'
      NUM_WORKERS=4
      MASTER_HOSTNAME='hadoop-m'
      WORKERS='hadoop-w-0 hadoop-w-1 hadoop-w-2 hadoop-w-3'
      BDUTIL_GCS_STAGING_DIR='gs://net-bh/bdutil-staging/hadoop-m'
            MASTER_ATTACHED_PD='hadoop-m-pd'
      WORKER_ATTACHED_PDS='hadoop-w-0-pd hadoop-w-1-pd hadoop-w-2-pd hadoop-w-3-pd'
      (y/n) y

It seems to be an ssh problem. /tmp/bdutil-20150321-232919-pZT/debuginfo.txt:

ssh: connect to host 104.155.62.245 port 22: Connection refused
ERROR: (gcloud.compute.ssh) [/usr/bin/ssh] exited with return code [255].
dennishuo commented 9 years ago

You may find more useful errors further down the debuginfo.txt; that particular error you saw may be a red herring sometimes in a debuginfo.txt if it occurs while waiting on initial sshability. Alternatively, if that's the only error you see in debuginfo.txt, it may be a transient error being unable to SSH onto the VM.

I did manage to repro a failed ambari deployment on centos-7; my error is a bit different:

dhuo-ambari-w-1:    error reading information on service iptables: No such file or directory

This occurs inside ambari-setup.sh, on the line chkconfig iptables off:

[root@dhuo-ambari-w-1 dhuo]# chkconfig iptables off
error reading information on service iptables: No such file or directory

In general, CentOS 7 is not listed as an officially-supported OS for Ambari 1.7.0 and some discussion on StackOverflow seems to indicate that it wouldn't be expected to work even if the iptables command in this case is worked around. @seanorama could perhaps comment more on the timeline for official CentOS 7 support.