grycap / ansible-role-hadoop

Ansible Role to install a Hadoop Cluster
Apache License 2.0
15 stars 13 forks source link

Problem with "Set JAVA_HOME in hadoop-env.sh" #11

Closed Zsomx closed 5 years ago

Zsomx commented 6 years ago

Hi, There is a problem in the task "Set JAVA_HOME in hadoop-env.sh" on Ubuntu trusty. The console log:

` TASK [grycap.hadoop : Set JAVA_HOME in hadoop-env.sh] ** fatal: [10.6.14.40]: FAILED! => {"msg": "No file was found when using with_first_found. Use the 'skip: true' option to allow this task to be skipped if no files are found"} to retry, use: --limit @/Users/Zsomx/hadoop-installer---ansible/hadoop_namenode.retry

PLAY RECAP ***** 10.6.14.40 : ok=9 changed=7 unreachable=0 failed=1 `

My playbook is a simple one: `- hosts: hadoop_namenode become: yes vars: namenode_ip: 10.6.14.40 version: '2.9.0' roles:

Any suggestions?

Thanks!

micafer commented 6 years ago

HI @Zsomx, This role installs java 1.8 and it expects that is is installed in one of these two paths:

It seems that in your case java is not installed there.

What O.S. are you using? Where is java installed in your nodes?

Zsomx commented 6 years ago

Hi @micafer , I'm using Ubuntu 14.04 (trusty). I thought that this role installs Java, because of the logs:

TASK [grycap.hadoop : stat] ****************************************************************************************************************************************************
ok: [10.6.14.40]

TASK [grycap.hadoop : Yum install {{ item }}] **********************************************************************************************************************************
skipping: [10.6.14.40] => (item=[])

TASK [grycap.hadoop : Ubuntu 14 install software-properties-common] ************************************************************************************************************
changed: [10.6.14.40]

TASK [grycap.hadoop : Ubuntu 14 install OpenJDK PPA repo] **********************************************************************************************************************
changed: [10.6.14.40]

TASK [grycap.hadoop : Apt install {{ item }}] **********************************************************************************************************************************
changed: [10.6.14.40] => (item=['wget', 'tar', 'gzip', 'unzip', 'openjdk-8-jre'])

TASK [grycap.hadoop : set_fact] ************************************************************************************************************************************************
skipping: [10.6.14.40]

TASK [grycap.hadoop : set_fact] ************************************************************************************************************************************************
skipping: [10.6.14.40]

TASK [grycap.hadoop : Download 2.9.0] ******************************************************************************************************************************************
changed: [10.6.14.40]

TASK [grycap.hadoop : Unarchive hadoop tgz] ************************************************************************************************************************************
changed: [10.6.14.40]

TASK [grycap.hadoop : Create /opt/hadoop-2.X dir] ******************************************************************************************************************************
changed: [10.6.14.40]

TASK [grycap.hadoop : Create /opt/hadoop dir] **********************************************************************************************************************************
changed: [10.6.14.40]

TASK [grycap.hadoop : Set JAVA_HOME in hadoop-env.sh] **************************************************************************************************************************
fatal: [10.6.14.40]: FAILED! => {"msg": "No file was found when using with_first_found. Use the 'skip: true' option to allow this task to be skipped if no files are found"}
        to retry, use: --limit @/Users/Zsomx/hadoop-installer---ansible/hadoop_namenode.retry

PLAY RECAP *********************************************************************************************************************************************************************
10.6.14.40                 : ok=9    changed=7    unreachable=0    failed=1
micafer commented 6 years ago

Hi @Zsomx,

Yes it seems, and in Ubuntu 14 the location must be: "/usr/lib/jvm/java-8-openjdk-amd64". Can you check if this directory exists? Or what is the content of the dir "/usr/lib/jvm/"?

Zsomx commented 6 years ago

Hi @micafer , Here is the content of the directory:

lrwxrwxrwx 1 root root   20 Jul 26  2017 java-1.8.0-openjdk-amd64 -> java-8-openjdk-amd64
drwxr-xr-x 5 root root 4096 Feb 22 14:35 java-8-openjdk-amd64

the java-8-openjdk-amd64 dir:

|-- bin
|   |-- java -> ../jre/bin/java
|   |-- java-rmi.cgi
|   |-- jjs -> ../jre/bin/jjs
|   |-- keytool -> ../jre/bin/keytool
|   |-- orbd -> ../jre/bin/orbd
|   |-- pack200 -> ../jre/bin/pack200
|   |-- policytool -> ../jre/bin/policytool
|   |-- rmid -> ../jre/bin/rmid
|   |-- rmiregistry -> ../jre/bin/rmiregistry
|   |-- servertool -> ../jre/bin/servertool
|   |-- tnameserv -> ../jre/bin/tnameserv
|   `-- unpack200 -> ../jre/bin/unpack200
|-- docs -> ../../../share/doc/openjdk-8-jre-headless
|-- jre
|   |-- ASSEMBLY_EXCEPTION
|   |-- THIRD_PARTY_README
|   |-- bin
|   |   |-- java
|   |   |-- jjs
|   |   |-- keytool
|   |   |-- orbd
|   |   |-- pack200
|   |   |-- policytool
|   |   |-- rmid
|   |   |-- rmiregistry
|   |   |-- servertool
|   |   |-- tnameserv
|   |   `-- unpack200
|   |-- lib
|   |   |-- accessibility.properties -> /etc/java-8-openjdk/accessibility.properties
|   |   |-- amd64
|   |   |   |-- jli
|   |   |   |   `-- libjli.so
|   |   |   |-- jvm.cfg -> /etc/java-8-openjdk/jvm-amd64.cfg
|   |   |   |-- jvm.cfg-default
|   |   |   |-- libattach.so
|   |   |   |-- libawt.so
|   |   |   |-- libawt_headless.so
|   |   |   |-- libawt_xawt.so
|   |   |   |-- libdt_socket.so
|   |   |   |-- libfontmanager.so
|   |   |   |-- libhprof.so
|   |   |   |-- libicedtea-sound.so
|   |   |   |-- libinstrument.so
|   |   |   |-- libj2gss.so
|   |   |   |-- libj2pcsc.so
|   |   |   |-- libj2pkcs11.so
|   |   |   |-- libjaas_unix.so
|   |   |   |-- libjava.so
|   |   |   |-- libjava_crw_demo.so
|   |   |   |-- libjavajpeg.so
|   |   |   |-- libjavalcms.so
|   |   |   |-- libjawt.so
|   |   |   |-- libjdwp.so
|   |   |   |-- libjsdt.so
|   |   |   |-- libjsig.so
|   |   |   |-- libjsound.so
|   |   |   |-- libjsoundalsa.so
|   |   |   |-- libmanagement.so
|   |   |   |-- libmlib_image.so
|   |   |   |-- libnet.so
|   |   |   |-- libnio.so
|   |   |   |-- libnpt.so
|   |   |   |-- libsaproc.so
|   |   |   |-- libsctp.so
|   |   |   |-- libsplashscreen.so
|   |   |   |-- libsunec.so
|   |   |   |-- libunpack.so
|   |   |   |-- libverify.so
|   |   |   |-- libzip.so
|   |   |   `-- server
|   |   |       |-- Xusage.txt
|   |   |       |-- classes.jsa
|   |   |       |-- libjsig.so -> ../libjsig.so
|   |   |       `-- libjvm.so
|   |   |-- calendars.properties -> /etc/java-8-openjdk/calendars.properties
|   |   |-- charsets.jar
|   |   |-- classlist
|   |   |-- cmm
|   |   |   |-- CIEXYZ.pf
|   |   |   |-- GRAY.pf
|   |   |   |-- LINEAR_RGB.pf
|   |   |   |-- PYCC.pf
|   |   |   `-- sRGB.pf
|   |   |-- content-types.properties -> /etc/java-8-openjdk/content-types.properties
|   |   |-- currency.data
|   |   |-- ext
|   |   |   |-- cldrdata.jar
|   |   |   |-- dnsns.jar
|   |   |   |-- icedtea-sound.jar
|   |   |   |-- jaccess.jar
|   |   |   |-- java-atk-wrapper.jar -> ../../../../../../share/java/java-atk-wrapper.jar
|   |   |   |-- libatk-wrapper.so -> ../../../../../x86_64-linux-gnu/jni/libatk-wrapper.so
|   |   |   |-- localedata.jar
|   |   |   |-- nashorn.jar
|   |   |   |-- sunec.jar
|   |   |   |-- sunjce_provider.jar
|   |   |   |-- sunpkcs11.jar
|   |   |   `-- zipfs.jar
|   |   |-- flavormap.properties -> /etc/java-8-openjdk/flavormap.properties
|   |   |-- hijrah-config-umalqura.properties
|   |   |-- images
|   |   |   `-- cursors
|   |   |       |-- cursors.properties -> /etc/java-8-openjdk/images/cursors/cursors.properties
|   |   |       |-- invalid32x32.gif
|   |   |       |-- motif_CopyDrop32x32.gif
|   |   |       |-- motif_CopyNoDrop32x32.gif
|   |   |       |-- motif_LinkDrop32x32.gif
|   |   |       |-- motif_LinkNoDrop32x32.gif
|   |   |       |-- motif_MoveDrop32x32.gif
|   |   |       `-- motif_MoveNoDrop32x32.gif
|   |   |-- jar.binfmt
|   |   |-- jce.jar
|   |   |-- jexec
|   |   |-- jsse.jar
|   |   |-- jvm.hprof.txt
|   |   |-- logging.properties -> /etc/java-8-openjdk/logging.properties
|   |   |-- management
|   |   |   |-- jmxremote.access -> /etc/java-8-openjdk/management/jmxremote.access
|   |   |   |-- jmxremote.password -> /etc/java-8-openjdk/management/jmxremote.password
|   |   |   |-- management.properties -> /etc/java-8-openjdk/management/management.properties
|   |   |   `-- snmp.acl -> /etc/java-8-openjdk/management/snmp.acl
|   |   |-- management-agent.jar
|   |   |-- meta-index
|   |   |-- net.properties -> /etc/java-8-openjdk/net.properties
|   |   |-- psfont.properties.ja -> /etc/java-8-openjdk/psfont.properties.ja
|   |   |-- psfontj2d.properties -> /etc/java-8-openjdk/psfontj2d.properties
|   |   |-- resources.jar
|   |   |-- rt.jar
|   |   |-- security
|   |   |   |-- US_export_policy.jar
|   |   |   |-- blacklisted.certs -> /etc/java-8-openjdk/security/blacklisted.certs
|   |   |   |-- cacerts -> /etc/ssl/certs/java/cacerts
|   |   |   |-- java.policy -> /etc/java-8-openjdk/security/java.policy
|   |   |   |-- java.security -> /etc/java-8-openjdk/security/java.security
|   |   |   |-- local_policy.jar
|   |   |   `-- nss.cfg -> /etc/java-8-openjdk/security/nss.cfg
|   |   |-- sound.properties -> /etc/java-8-openjdk/sound.properties
|   |   |-- swing.properties -> /etc/java-8-openjdk/swing.properties
|   |   `-- tzdb.dat
|   `-- man
|       |-- ja -> ja_JP.UTF-8
|       |-- ja_JP.UTF-8
|       |   `-- man1
|       |       |-- java.1.gz
|       |       |-- jjs.1.gz
|       |       |-- keytool.1.gz
|       |       |-- orbd.1.gz
|       |       |-- pack200.1.gz
|       |       |-- policytool.1.gz
|       |       |-- rmid.1.gz
|       |       |-- rmiregistry.1.gz
|       |       |-- servertool.1.gz
|       |       |-- tnameserv.1.gz
|       |       `-- unpack200.1.gz
|       `-- man1
|           |-- java.1.gz
|           |-- jjs.1.gz
|           |-- keytool.1.gz
|           |-- orbd.1.gz
|           |-- pack200.1.gz
|           |-- policytool.1.gz
|           |-- rmid.1.gz
|           |-- rmiregistry.1.gz
|           |-- servertool.1.gz
|           |-- tnameserv.1.gz
|           `-- unpack200.1.gz
`-- man
    |-- ja -> ja_JP.UTF-8
    |-- ja_JP.UTF-8
    |   `-- man1
    |       |-- java.1.gz -> ../../../jre/man/ja_JP.UTF-8/man1/java.1.gz
    |       |-- jjs.1.gz -> ../../../jre/man/ja_JP.UTF-8/man1/jjs.1.gz
    |       |-- keytool.1.gz -> ../../../jre/man/ja_JP.UTF-8/man1/keytool.1.gz
    |       |-- orbd.1.gz -> ../../../jre/man/ja_JP.UTF-8/man1/orbd.1.gz
    |       |-- pack200.1.gz -> ../../../jre/man/ja_JP.UTF-8/man1/pack200.1.gz
    |       |-- policytool.1.gz -> ../../../jre/man/ja_JP.UTF-8/man1/policytool.1.gz
    |       |-- rmid.1.gz -> ../../../jre/man/ja_JP.UTF-8/man1/rmid.1.gz
    |       |-- rmiregistry.1.gz -> ../../../jre/man/ja_JP.UTF-8/man1/rmiregistry.1.gz
    |       |-- servertool.1.gz -> ../../../jre/man/ja_JP.UTF-8/man1/servertool.1.gz
    |       |-- tnameserv.1.gz -> ../../../jre/man/ja_JP.UTF-8/man1/tnameserv.1.gz
    |       `-- unpack200.1.gz -> ../../../jre/man/ja_JP.UTF-8/man1/unpack200.1.gz
    `-- man1
        |-- java.1.gz -> ../../jre/man/man1/java.1.gz
        |-- jjs.1.gz -> ../../jre/man/man1/jjs.1.gz
        |-- keytool.1.gz -> ../../jre/man/man1/keytool.1.gz
        |-- orbd.1.gz -> ../../jre/man/man1/orbd.1.gz
        |-- pack200.1.gz -> ../../jre/man/man1/pack200.1.gz
        |-- policytool.1.gz -> ../../jre/man/man1/policytool.1.gz
        |-- rmid.1.gz -> ../../jre/man/man1/rmid.1.gz
        |-- rmiregistry.1.gz -> ../../jre/man/man1/rmiregistry.1.gz
        |-- servertool.1.gz -> ../../jre/man/man1/servertool.1.gz
        |-- tnameserv.1.gz -> ../../jre/man/man1/tnameserv.1.gz
        `-- unpack200.1.gz -> ../../jre/man/man1/unpack200.1.gz

Seems fine.

guitmz commented 6 years ago

I have the same problem for ubuntu 16.04.

I also tried changing the to /usr/lib/jvm/java-1.8.0-openjdk-amd64 just in case and it still fails.

At this point, Java should be already installed

amazon-ebs-builder: TASK [hadoop : Apt install {{ item }}] *****************************************
amazon-ebs-builder: changed: [default] => (item=['wget', 'tar', 'gzip', 'unzip', 'openjdk-8-jre'])

I was able to overcome this issue by manually specifying the path in the lineinfile action. It looks like an Ansible bug? I'm using 2.4