search

mesosphere-backup / hdfs-deprecated

[DEPRECATED] This project is deprecated. It will be archived on December 1, 2017.
Apache License 2.0
147 stars 52 forks source link

Namenodes fails to start #242

Open manuchandraprasad opened 8 years ago

manuchandraprasad commented 8 years ago

I'm trying to load HDFS into a mesos cluster with 4 16GB , 8CPU slaves. No matter what i do namenode fails to start.

Here is my mesos-site.xml

<configuration>
  <property>
    <name>mesos.hdfs.data.dir</name>
    <description>The primary data directory in HDFS</description>
    <value>/root/hdfsdata</value>
  </property>

  <!-- Uncomment this to enable secondary data dir
  <property>
    <name>mesos.hdfs.secondary.data.dir</name>
    <description>The secondary data directory in HDFS</description>
    <value>/var/lib/hdfs/data2</value>
  </property>
  -->

  <property>
    <name>mesos.hdfs.domain.socket.dir</name>
    <description>The secondary data directory in HDFS</description>
    <value>/var/run/hadoop-hdfs</value>
  </property>

  <!-- Uncomment this to enable backup
  <property>
    <name>mesos.hdfs.backup.dir</name>
    <description>Backup dir for HDFS</description>
    <value>/tmp/nfs</value>
  </property>
  -->

  <property>
    <name>mesos.hdfs.native-hadoop-binaries</name>
    <description>Mark true if you have hadoop pre-installed on your host machines (otherwise it will be distributed by the scheduler)</description>
    <value>false</value>
  </property>

  <property>
    <name>mesos.hdfs.framework.mnt.path</name>
    <description>Mount location (if mesos.hdfs.native-hadoop-binaries is marked false)</description>
    <value>/opt/mesosphere</value>
  </property>

  <property>
    <name>mesos.hdfs.state.zk</name>
    <description>Comma-separated hostname-port pairs of zookeeper node locations for HDFS framework state information</description>
    <value>mesos-master:2181</value>
  </property>

  <property>
    <name>mesos.master.uri</name>
    <description>Zookeeper entry for mesos master location</description>
    <value>zk://mesos-master:2181/mesos</value>
  </property>

  <property>
    <name>mesos.hdfs.zkfc.ha.zookeeper.quorum</name>
    <description>Comma-separated list of zookeeper hostname-port pairs for HDFS HA features</description>
    <value>mesos-master:2181</value>
  </property>

  <property>
    <name>mesos.hdfs.framework.name</name>
    <description>Your Mesos framework name and cluster name when accessing files (hdfs://YOUR_NAME)</description>
    <value>hdfs</value>
  </property>

  <property>
    <name>mesos.hdfs.mesosdns</name>
    <description>Whether to use Mesos DNS for service discovery within HDFS</description>
    <value>false</value>
  </property>

  <property>
    <name>mesos.hdfs.mesosdns.domain</name>
    <description>Root domain name of Mesos DNS (usually 'mesos')</description>
    <value>mesos</value>
  </property>

  <property>
    <name>mesos.native.library</name>
    <description>Location of libmesos.so</description>
    <value>/usr/local/lib/libmesos.so</value>
  </property>

  <property>
    <name>mesos.hdfs.journalnode.count</name>
    <description>Number of journal nodes (must be odd)</description>
    <value>3</value>
  </property>

  <!-- Additional settings for fine-tuning -->
  <!--
  <property>
    <name>mesos.hdfs.jvm.overhead</name>
    <description>Multiplier on resources reserved in order to account for JVM allocation</description>
    <value>1.35</value>
  </property>

  <property>
    <name>mesos.hdfs.hadoop.heap.size</name>
    <value>512</value>
  </property>

  <property>
    <name>mesos.hdfs.namenode.heap.size</name>
    <value>4096</value>
  </property>

  <property>
    <name>mesos.hdfs.datanode.heap.size</name>
    <value>1024</value>
  </property>

  <property>
    <name>mesos.hdfs.executor.heap.size</name>
    <value>256</value>
  </property>

  <property>
    <name>mesos.hdfs.executor.cpus</name>
    <value>0.5</value>
  </property>

  <property>
    <name>mesos.hdfs.namenode.cpus</name>
    <value>1</value>
  </property>

  <property>
    <name>mesos.hdfs.journalnode.cpus</name>
    <value>1</value>
  </property>

  <property>
    <name>mesos.hdfs.datanode.cpus</name>
    <value>1</value>
  </property>

  <property>
    <name>mesos.hdfs.user</name>
    <value>hdfs</value>
  </property>

  <property>
    <name>mesos.hdfs.role</name>
    <value>*</value>
  </property>

  <property>
    <name>mesos.hdfs.datanode.exclusive</name>
    WARNING-It is not advisable to run the datanode on same slave because of performance issues
    <description>Whether to run the datanode on slave different from namenode and journal nodes</description>
    <value>true</value>
  </property>
  -->
</configuration>

Here is sandbox log for a slave

/************************************************************
STARTUP_MSG: Starting NameNode
STARTUP_MSG:   host = mesos-slave-02/159.203.117.223
STARTUP_MSG:   args = []
STARTUP_MSG:   version = 2.5.0-cdh5.3.1
STARTUP_MSG:   classpath = /tmp/mesos/slaves/9463b2ce-a133-4a39-a7cd-423ea9e884bc-S39/frameworks/9463b2ce-a133-4a39-a7cd-423ea9e884bc-0000/executors/executor.namenode.NameNodeExecutor.1449343025206/runs/802437d8-46d5-47d1-abd5-739850a5f3fc/hdfs-mesos-executor-0.1.5/etc/hadoop:/tmp/mesos/slaves/9463b2ce-a133-4a39-a7cd-423ea9e884bc-S39/frameworks/9463b2ce-a133-4a39-a7cd-423ea9e884bc-0000/executors/executor.namenode.NameNodeExecutor.1449343025206/runs/802437d8-46d5-47d1-abd5-739850a5f3fc/hdfs-mesos-executor-0.1.5/share/hadoop/common/lib/jackson-core-asl-1.8.8.jar:/tmp/mesos/slaves/9463b2ce-a133-4a39-a7cd-423ea9e884bc-S39/frameworks/9463b2ce-a133-4a39-a7cd-423ea9e884bc-0000/executors/executor.namenode.NameNodeExecutor.1449343025206/runs/802437d8-46d5-47d1-abd5-739850a5f3fc/hdfs-mesos-executor-0.1.5/share/hadoop/common/lib/paranamer-2.3.jar:/tmp/mesos/slaves/9463b2ce-a133-4a39-a7cd-423ea9e884bc-S39/frameworks/9463b2ce-a133-4a39-a7cd-423ea9e884bc-0000/executors/executor.namenode.NameNodeExecutor.1449343025206/runs/802437d8-46d5-47d1-abd5-739850a5f3fc/hdfs-mesos-executor-0.1.5/share/hadoop/common/lib/api-asn1-api-1.0.0-M20.jar:/tmp/mesos/slaves/9463b2ce-a133-4a39-a7cd-423ea9e884bc-S39/frameworks/9463b2ce-a133-4a39-a7cd-423ea9e884bc-0000/executors/executor.namenode.NameNodeExecutor.1449343025206/runs/802437d8-46d5-47d1-abd5-739850a5f3fc/hdfs-mesos-executor-0.1.5/share/hadoop/common/lib/zookeeper-3.4.5-cdh5.3.1.jar:/tmp/mesos/slaves/9463b2ce-a133-4a39-a7cd-423ea9e884bc-S39/frameworks/9463b2ce-a133-4a39-a7cd-423ea9e884bc-0000/executors/executor.namenode.NameNodeExecutor.1449343025206/runs/802437d8-46d5-47d1-abd5-739850a5f3fc/hdfs-mesos-executor-0.1.5/share/hadoop/common/lib/curator-recipes-2.6.0.jar:/tmp/mesos/slaves/9463b2ce-a133-4a39-a7cd-423ea9e884bc-S39/frameworks/9463b2ce-a133-4a39-a7cd-423ea9e884bc-0000/executors/executor.namenode.NameNodeExecutor.1449343025206/runs/802437d8-46d5-47d1-abd5-739850a5f3fc/hdfs-mesos-executor-0.1.5/share/hadoop/common/lib/servlet-api-2.5.jar:/tmp/mesos/slaves/9463b2ce-a133-4a39-a7cd-423ea9e884bc-S39/frameworks/9463b2ce-a133-4a39-a7cd-423ea9e884bc-0000/executors/executor.namenode.NameNodeExecutor.1449343025206/runs/802437d8-46d5-47d1-abd5-739850a5f3fc/hdfs-mesos-executor-0.1.5/share/hadoop/common/lib/commons-beanutils-1.7.0.jar:/tmp/mesos/slaves/9463b2ce-a133-4a39-a7cd-423ea9e884bc-S39/frameworks/9463b2ce-a133-4a39-a7cd-423ea9e884bc-0000/executors/executor.namenode.NameNodeExecutor.1449343025206/runs/802437d8-46d5-47d1-abd5-739850a5f3fc/hdfs-mesos-executor-0.1.5/share/hadoop/common/lib/protobuf-java-2.5.0.jar:/tmp/mesos/slaves/9463b2ce-a133-4a39-a7cd-423ea9e884bc-S39/frameworks/9463b2ce-a133-4a39-a7cd-423ea9e884bc-0000/executors/executor.namenode.NameNodeExecutor.1449343025206/runs/802437d8-46d5-47d1-abd5-739850a5f3fc/hdfs-mesos-executor-0.1.5/share/hadoop/common/lib/commons-collections-3.2.1.jar:/tmp/mesos/slaves/9463b2ce-a133-4a39-a7cd-423ea9e884bc-S39/frameworks/9463b2ce-a133-4a39-a7cd-423ea9e884bc-0000/executors/executor.namenode.NameNodeExecutor.1449343025206/runs/802437d8-46d5-47d1-abd5-739850a5f3fc/hdfs-mesos-executor-0.1.5/share/hadoop/common/lib/commons-compress-1.4.1.jar:/tmp/mesos/slaves/9463b2ce-a133-4a39-a7cd-423ea9e884bc-S39/frameworks/9463b2ce-a133-4a39-a7cd-423ea9e884bc-0000/executors/executor.namenode.NameNodeExecutor.1449343025206/runs/802437d8-46d5-47d1-abd5-739850a5f3fc/hdfs-mesos-executor-0.1.5/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar:/tmp/mesos/slaves/9463b2ce-a133-4a39-a7cd-423ea9e884bc-S39/frameworks/9463b2ce-a133-4a39-a7cd-423ea9e884bc-0000/executors/executor.namenode.NameNodeExecutor.1449343025206/runs/802437d8-46d5-47d1-abd5-739850a5f3fc/hdfs-mesos-executor-0.1.5/share/hadoop/common/lib/java-xmlbuilder-0.4.jar:/tmp/mesos/slaves/9463b2ce-a133-4a39-a7cd-423ea9e884bc-S39/frameworks/9463b2ce-a133-4a39-a7cd-423ea9e884bc-0000/executors/executor.namenode.NameNodeExecutor.1449343025206/runs/802437d8-46d5-47d1-abd5-739850a5f3fc/hdfs-mesos-executor-0.1.5/share/hadoop/common/lib/netty-3.6.2.Final.jar:/tmp/mesos/slaves/9463b2ce-a133-4a39-a7cd-423ea9e884bc-S39/frameworks/9463b2ce-a133-4a39-a7cd-423ea9e884bc-0000/executors/executor.namenode.NameNodeExecutor.1449343025206/runs/802437d8-46d5-47d1-abd5-739850a5f3fc/hdfs-mesos-executor-0.1.5/share/hadoop/common/lib/junit-4.11.jar:/tmp/mesos/slaves/9463b2ce-a133-4a39-a7cd-423ea9e884bc-S39/frameworks/9463b2ce-a133-4a39-a7cd-423ea9e884bc-0000/executors/executor.namenode.NameNodeExecutor.1449343025206/runs/802437d8-46d5-47d1-abd5-739850a5f3fc/hdfs-mesos-executor-0.1.5/share/hadoop/common/lib/jaxb-api-2.2.2.jar:/tmp/mesos/slaves/9463b2ce-a133-4a39-a7cd-423ea9e884bc-S39/frameworks/9463b2ce-a133-4a39-a7cd-423ea9e884bc-0000/executors/executor.namenode.NameNodeExecutor.1449343025206/runs/802437d8-46d5-47d1-abd5-739850a5f3fc/hdfs-mesos-executor-0.1.5/share/hadoop/common/lib/slf4j-api-1.7.5.jar:/tmp/mesos/slaves/9463b2ce-a133-4a39-a7cd-423ea9e884bc-S39/frameworks/9463b2ce-a133-4a39-a7cd-423ea9e884bc-0000/executors/executor.namenode.NameNodeExecutor.1449343025206/runs/802437d8-46d5-47d1-abd5-739850a5f3fc/hdfs-mesos-executor-0.1.5/share/hadoop/common/lib/apacheds-kerberos-codec-2.0.0-M15.jar:/tmp/mesos/slaves/9463b2ce-a133-4a39-a7cd-423ea9e884bc-S39/frameworks/9463b2ce-a133-4a39-a7cd-423ea9e884bc-0000/executors/executor.namenode.NameNodeExecutor.1449343025206/runs/802437d8-46d5-47d1-abd5-739850a5f3fc/hdfs-mesos-executor-0.1.5/share/hadoop/common/lib/stax-api-1.0-2.jar:/tmp/mesos/slaves/9463b2ce-a133-4a39-a7cd-423ea9e884bc-S39/frameworks/9463b2ce-a133-4a39-a7cd-423ea9e884bc-0000/executors/executor.namenode.NameNodeExecutor.1449343025206/runs/802437d8-46d5-47d1-abd5-739850a5f3fc/hdfs-mesos-executor-0.1.5/share/hadoop/common/lib/jasper-compiler-5.5.23.jar:/tmp/mesos/slaves/9463b2ce-a133-4a39-a7cd-423ea9e884bc-S39/frameworks/9463b2ce-a133-4a39-a7cd-423ea9e884bc-0000/executors/executor.namenode.NameNodeExecutor.1449343025206/runs/802437d8-46d5-47d1-abd5-739850a5f3fc/hdfs-mesos-executor-0.1.5/share/hadoop/common/lib/snappy-java-1.0.4.1.jar:/tmp/mesos/slaves/9463b2ce-a133-4a39-a7cd-423ea9e884bc-S39/frameworks/9463b2ce-a133-4a39-a7cd-423ea9e884bc-0000/executors/executor.namenode.NameNodeExecutor.1449343025206/runs/802437d8-46d5-47d1-abd5-739850a5f3fc/hdfs-mesos-executor-0.1.5/share/hadoop/common/lib/jackson-jaxrs-1.8.8.jar:/tmp/mesos/slaves/9463b2ce-a133-4a39-a7cd-423ea9e884bc-S39/frameworks/9463b2ce-a133-4a39-a7cd-423ea9e884bc-0000/executors/executor.namenode.NameNodeExecutor.1449343025206/runs/802437d8-46d5-47d1-abd5-739850a5f3fc/hdfs-mesos-executor-0.1.5/share/hadoop/common/lib/jackson-mapper-asl-1.8.8.jar:/tmp/mesos/slaves/9463b2ce-a133-4a39-a7cd-423ea9e884bc-S39/frameworks/9463b2ce-a133-4a39-a7cd-423ea9e884bc-0000/executors/executor.namenode.NameNodeExecutor.1449343025206/runs/802437d8-46d5-47d1-abd5-739850a5f3fc/hdfs-mesos-executor-0.1.5/share/hadoop/common/lib/commons-digester-1.8.jar:/tmp/mesos/slaves/9463b2ce-a133-4a39-a7cd-423ea9e884bc-S39/frameworks/9463b2ce-a133-4a39-a7cd-423ea9e884bc-0000/executors/executor.namenode.NameNodeExecutor.1449343025206/runs/802437d8-46d5-47d1-abd5-739850a5f3fc/hdfs-mesos-executor-0.1.5/share/hadoop/common/lib/commons-net-3.1.jar:/tmp/mesos/slaves/9463b2ce-a133-4a39-a7cd-423ea9e884bc-S39/frameworks/9463b2ce-a133-4a39-a7cd-423ea9e884bc-0000/executors/executor.namenode.NameNodeExecutor.1449343025206/runs/802437d8-46d5-47d1-abd5-739850a5f3fc/hdfs-mesos-executor-0.1.5/share/hadoop/common/lib/asm-3.2.jar:/tmp/mesos/slaves/9463b2ce-a133-4a39-a7cd-423ea9e884bc-S39/frameworks/9463b2ce-a133-4a39-a7cd-423ea9e884bc-0000/executors/executor.namenode.NameNodeExecutor.1449343025206/runs/802437d8-46d5-47d1-abd5-739850a5f3fc/hdfs-mesos-executor-0.1.5/share/hadoop/common/lib/jetty-6.1.26.cloudera.4.jar:/tmp/mesos/slaves/9463b2ce-a133-4a39-a7cd-423ea9e884bc-S39/frameworks/9463b2ce-a133-4a39-a7cd-423ea9e884bc-0000/executors/executor.namenode.NameNodeExecutor.1449343025206/runs/802437d8-46d5-47d1-abd5-739850a5f3fc/hdfs-mesos-executor-0.1.5/share/hadoop/common/lib/commons-httpclient-3.1.jar:/tmp/mesos/slaves/9463b2ce-a133-4a39-a7cd-423ea9e884bc-S39/frameworks/9463b2ce-a133-4a39-a7cd-423ea9e884bc-0000/executors/executor.namenode.NameNodeExecutor.1449343025206/runs/802437d8-46d5-47d1-abd5-739850a5f3fc/hdfs-mesos-executor-0.1.5/share/hadoop/common/lib/hadoop-auth-2.5.0-cdh5.3.1.jar:/tmp/mesos/slaves/9463b2ce-a133-4a39-a7cd-423ea9e884bc-S39/frameworks/9463b2ce-a133-4a39-a7cd-423ea9e884bc-0000/executors/executor.namenode.NameNodeExecutor.1449343025206/runs/802437d8-46d5-47d1-abd5-739850a5f3fc/hdfs-mesos-executor-0.1.5/share/hadoop/common/lib/xz-1.0.jar:/tmp/mesos/slaves/9463b2ce-a133-4a39-a7cd-423ea9e884bc-S39/frameworks/9463b2ce-a133-4a39-a7cd-423ea9e884bc-0000/executors/executor.namenode.NameNodeExecutor.1449343025206/runs/802437d8-46d5-47d1-abd5-739850a5f3fc/hdfs-mesos-executor-0.1.5/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar:/tmp/mesos/slaves/9463b2ce-a133-4a39-a7cd-423ea9e884bc-S39/frameworks/9463b2ce-a133-4a39-a7cd-423ea9e884bc-0000/executors/executor.namenode.NameNodeExecutor.1449343025206/runs/802437d8-46d5-47d1-abd5-739850a5f3fc/hdfs-mesos-executor-0.1.5/share/hadoop/common/lib/hamcrest-core-1.3.jar:/tmp/mesos/slaves/9463b2ce-a133-4a39-a7cd-423ea9e884bc-S39/frameworks/9463b2ce-a133-4a39-a7cd-423ea9e884bc-0000/executors/executor.namenode.NameNodeExecutor.1449343025206/runs/802437d8-46d5-47d1-abd5-739850a5f3fc/hdfs-mesos-executor-0.1.5/share/hadoop/common/lib/apacheds-i18n-2.0.0-M15.jar:/tmp/mesos/slaves/9463b2ce-a133-4a39-a7cd-423ea9e884bc-S39/frameworks/9463b2ce-a133-4a39-a7cd-423ea9e884bc-0000/executors/executor.namenode.NameNodeExecutor.1449343025206/runs/802437d8-46d5-47d1-abd5-739850a5f3fc/hdfs-mesos-executor-0.1.5/share/hadoop/common/lib/jsr305-1.3.9.jar:/tmp/mesos/slaves/9463b2ce-a133-4a39-a7cd-423ea9e884bc-S39/frameworks/9463b2ce-a133-4a39-a7cd-423ea9e884bc-0000/executors/executor.namenode.NameNodeExecutor.1449343025206/runs/802437d8-46d5-47d1-abd5-739850a5f3fc/hdfs-mesos-executor-0.1.5/share/hadoop/common/lib/commons-cli-1.2.jar:/tmp/mesos/slaves/9463b2ce-a133-4a39-a7cd-423ea9e884bc-S39/frameworks/9463b2ce-a133-4a39-a7cd-423ea9e884bc-0000/executors/executor.namenode.NameNodeExecutor.1449343025206/runs/802437d8-46d5-47d1-abd5-739850a5f3fc/hdfs-mesos-executor-0.1.5/share/hadoop/common/lib/jetty-util-6.1.26.cloudera.4.jar:/tmp/mesos/slaves/9463b2ce-a133-4a39-a7cd-423ea9e884bc-S39/frameworks/9463b2ce-a133-4a39-a7cd-423ea9e884bc-0000/executors/executor.namenode.NameNodeExecutor.1449343025206/runs/802437d8-46d5-47d1-abd5-739850a5f3fc/hdfs-mesos-executor-0.1.5/share/hadoop/common/lib/jersey-core-1.9.jar:/tmp/mesos/slaves/9463b2ce-a133-4a39-a7cd-423ea9e884bc-S39/frameworks/9463b2ce-a133-4a39-a7cd-423ea9e884bc-0000/executors/executor.namenode.NameNodeExecutor.1449343025206/runs/802437d8-46d5-47d1-abd5-739850a5f3fc/hdfs-mesos-executor-0.1.5/share/hadoop/common/lib/guava-11.0.2.jar:/tmp/mesos/slaves/9463b2ce-a133-4a39-a7cd-423ea9e884bc-S39/frameworks/9463b2ce-a133-4a39-a7cd-423ea9e884bc-0000/executors/executor.namenode.NameNodeExecutor.1449343025206/runs/802437d8-46d5-47d1-abd5-739850a5f3fc/hdfs-mesos-executor-0.1.5/share/hadoop/common/lib/jersey-json-1.9.jar:/tmp/mesos/slaves/9463b2ce-a133-4a39-a7cd-423ea9e884bc-S39/frameworks/9463b2ce-a133-4a39-a7cd-423ea9e884bc-0000/executors/executor.namenode.NameNodeExecutor.1449343025206/runs/802437d8-46d5-47d1-abd5-739850a5f3fc/hdfs-mesos-executor-0.1.5/share/hadoop/common/lib/hadoop-annotations-2.5.0-cdh5.3.1.jar:/tmp/mesos/slaves/9463b2ce-a133-4a39-a7cd-423ea9e884bc-S39/frameworks/9463b2ce-a133-4a39-a7cd-423ea9e884bc-0000/executors/executor.namenode.NameNodeExecutor.1449343025206/runs/802437d8-46d5-47d1-abd5-739850a5f3fc/hdfs-mesos-executor-0.1.5/share/hadoop/common/lib/xmlenc-0.52.jar:/tmp/mesos/slaves/9463b2ce-a133-4a39-a7cd-423ea9e884bc-S39/frameworks/9463b2ce-a133-4a39-a7cd-423ea9e884bc-0000/executors/executor.namenode.NameNodeExecutor.1449343025206/runs/802437d8-46d5-47d1-abd5-739850a5f3fc/hdfs-mesos-executor-0.1.5/share/hadoop/common/lib/httpcore-4.2.5.jar:/tmp/mesos/slaves/9463b2ce-a133-4a39-a7cd-423ea9e884bc-S39/frameworks/9463b2ce-a133-4a39-a7cd-423ea9e884bc-0000/executors/executor.namenode.NameNodeExecutor.1449343025206/runs/802437d8-46d5-47d1-abd5-739850a5f3fc/hdfs-mesos-executor-0.1.5/share/hadoop/common/lib/commons-codec-1.4.jar:/tmp/mesos/slaves/9463b2ce-a133-4a39-a7cd-423ea9e884bc-S39/frameworks/9463b2ce-a133-4a39-a7cd-423ea9e884bc-0000/executors/executor.namenode.NameNodeExecutor.1449343025206/runs/802437d8-46d5-47d1-abd5-739850a5f3fc/hdfs-mesos-executor-0.1.5/share/hadoop/common/lib/jackson-xc-1.8.8.jar:/tmp/mesos/slaves/9463b2ce-a133-4a39-a7cd-423ea9e884bc-S39/frameworks/9463b2ce-a133-4a39-a7cd-423ea9e884bc-0000/executors/executor.namenode.NameNodeExecutor.1449343025206/runs/802437d8-46d5-47d1-abd5-739850a5f3fc/hdfs-mesos-executor-0.1.5/share/hadoop/common/lib/commons-el-1.0.jar:/tmp/mesos/slaves/9463b2ce-a133-4a39-a7cd-423ea9e884bc-S39/frameworks/9463b2ce-a133-4a39-a7cd-423ea9e884bc-0000/executors/executor.namenode.NameNodeExecutor.1449343025206/runs/802437d8-46d5-47d1-abd5-739850a5f3fc/hdfs-mesos-executor-0.1.5/share/hadoop/common/lib/jsch-0.1.42.jar:/tmp/mesos/slaves/9463b2ce-a133-4a39-a7cd-423ea9e884bc-S39/frameworks/9463b2ce-a133-4a39-a7cd-423ea9e884bc-0000/executors/executor.namenode.NameNodeExecutor.1449343025206/runs/802437d8-46d5-47d1-abd5-739850a5f3fc/hdfs-mesos-executor-0.1.5/share/hadoop/common/lib/api-util-1.0.0-M20.jar:/tmp/mesos/slaves/9463b2ce-a133-4a39-a7cd-423ea9e884bc-S39/frameworks/9463b2ce-a133-4a39-a7cd-423ea9e884bc-0000/executors/executor.namenode.NameNodeExecutor.1449343025206/runs/802437d8-46d5-47d1-abd5-739850a5f3fc/hdfs-mesos-executor-0.1.5/share/hadoop/common/lib/avro-1.7.6-cdh5.3.1.jar:/tmp/mesos/slaves/9463b2ce-a133-4a39-a7cd-423ea9e884bc-S39/frameworks/9463b2ce-a133-4a39-a7cd-423ea9e884bc-0000/executors/executor.namenode.NameNodeExecutor.1449343025206/runs/802437d8-46d5-47d1-abd5-739850a5f3fc/hdfs-mesos-executor-0.1.5/share/hadoop/common/lib/gson-2.2.4.jar:/tmp/mesos/slaves/9463b2ce-a133-4a39-a7cd-423ea9e884bc-S39/frameworks/9463b2ce-a133-4a39-a7cd-423ea9e884bc-0000/executors/executor.namenode.NameNodeExecutor.1449343025206/runs/802437d8-46d5-47d1-abd5-739850a5f3fc/hdfs-mesos-executor-0.1.5/share/hadoop/common/lib/jsp-api-2.1.jar:/tmp/mesos/slaves/9463b2ce-a133-4a39-a7cd-423ea9e884bc-S39/frameworks/9463b2ce-a133-4a39-a7cd-423ea9e884bc-0000/executors/executor.namenode.NameNodeExecutor.1449343025206/runs/802437d8-46d5-47d1-abd5-739850a5f3fc/hdfs-mesos-executor-0.1.5/share/hadoop/common/lib/commons-io-2.4.jar:/tmp/mesos/slaves/9463b2ce-a133-4a39-a7cd-423ea9e884bc-S39/frameworks/9463b2ce-a133-4a39-a7cd-423ea9e884bc-0000/executors/executor.namenode.NameNodeExecutor.1449343025206/runs/802437d8-46d5-47d1-abd5-739850a5f3fc/hdfs-mesos-executor-0.1.5/share/hadoop/common/lib/log4j-1.2.17.jar:/tmp/mesos/slaves/9463b2ce-a133-4a39-a7cd-423ea9e884bc-S39/frameworks/9463b2ce-a133-4a39-a7cd-423ea9e884bc-0000/executors/executor.namenode.NameNodeExecutor.1449343025206/runs/802437d8-46d5-47d1-abd5-739850a5f3fc/hdfs-mesos-executor-0.1.5/share/hadoop/common/lib/commons-beanutils-core-1.8.0.jar:/tmp/mesos/slaves/9463b2ce-a133-4a39-a7cd-423ea9e884bc-S39/frameworks/9463b2ce-a133-4a39-a7cd-423ea9e884bc-0000/executors/executor.namenode.NameNodeExecutor.1449343025206/runs/802437d8-46d5-47d1-abd5-739850a5f3fc/hdfs-mesos-executor-0.1.5/share/hadoop/common/lib/commons-math3-3.1.1.jar:/tmp/mesos/slaves/9463b2ce-a133-4a39-a7cd-423ea9e884bc-S39/frameworks/9463b2ce-a133-4a39-a7cd-423ea9e884bc-0000/executors/executor.namenode.NameNodeExecutor.1449343025206/runs/802437d8-46d5-47d1-abd5-739850a5f3fc/hdfs-mesos-executor-0.1.5/share/hadoop/common/lib/commons-lang-2.6.jar:/tmp/mesos/slaves/9463b2ce-a133-4a39-a7cd-423ea9e884bc-S39/frameworks/9463b2ce-a133-4a39-a7cd-423ea9e884bc-0000/executors/executor.namenode.NameNodeExecutor.1449343025206/runs/802437d8-46d5-47d1-abd5-739850a5f3fc/hdfs-mesos-executor-0.1.5/share/hadoop/common/lib/commons-configuration-1.6.jar:/tmp/mesos/slaves/9463b2ce-a133-4a39-a7cd-423ea9e884bc-S39/frameworks/9463b2ce-a133-4a39-a7cd-423ea9e884bc-0000/executors/executor.namenode.NameNodeExecutor.1449343025206/runs/802437d8-46d5-47d1-abd5-739850a5f3fc/hdfs-mesos-executor-0.1.5/share/hadoop/common/lib/jettison-1.1.jar:/tmp/mesos/slaves/9463b2ce-a133-4a39-a7cd-423ea9e884bc-S39/frameworks/9463b2ce-a133-4a39-a7cd-423ea9e884bc-0000/executors/executor.namenode.NameNodeExecutor.1449343025206/runs/802437d8-46d5-47d1-abd5-739850a5f3fc/hdfs-mesos-executor-0.1.5/share/hadoop/common/lib/mockito-all-1.8.5.jar:/tmp/mesos/slaves/9463b2ce-a133-4a39-a7cd-423ea9e884bc-S39/frameworks/9463b2ce-a133-4a39-a7cd-423ea9e884bc-0000/executors/executor.namenode.NameNodeExecutor.1449343025206/runs/802437d8-46d5-47d1-abd5-739850a5f3fc/hdfs-mesos-executor-0.1.5/share/hadoop/common/lib/curator-client-2.6.0.jar:/tmp/mesos/slaves/9463b2ce-a133-4a39-a7cd-423ea9e884bc-S39/frameworks/9463b2ce-a133-4a39-a7cd-423ea9e884bc-0000/executors/executor.namenode.NameNodeExecutor.1449343025206/runs/802437d8-46d5-47d1-abd5-739850a5f3fc/hdfs-mesos-executor-0.1.5/share/hadoop/common/lib/jersey-server-1.9.jar:/tmp/mesos/slaves/9463b2ce-a133-4a39-a7cd-423ea9e884bc-S39/frameworks/9463b2ce-a133-4a39-a7cd-423ea9e884bc-0000/executors/executor.namenode.NameNodeExecutor.1449343025206/runs/802437d8-46d5-47d1-abd5-739850a5f3fc/hdfs-mesos-executor-0.1.5/share/hadoop/common/lib/httpclient-4.2.5.jar:/tmp/mesos/slaves/9463b2ce-a133-4a39-a7cd-423ea9e884bc-S39/frameworks/9463b2ce-a133-4a39-a7cd-423ea9e884bc-0000/executors/executor.namenode.NameNodeExecutor.1449343025206/runs/802437d8-46d5-47d1-abd5-739850a5f3fc/hdfs-mesos-executor-0.1.5/share/hadoop/common/lib/curator-framework-2.6.0.jar:/tmp/mesos/slaves/9463b2ce-a133-4a39-a7cd-423ea9e884bc-S39/frameworks/9463b2ce-a133-4a39-a7cd-423ea9e884bc-0000/executors/executor.namenode.NameNodeExecutor.1449343025206/runs/802437d8-46d5-47d1-abd5-739850a5f3fc/hdfs-mesos-executor-0.1.5/share/hadoop/common/lib/activation-1.1.jar:/tmp/mesos/slaves/9463b2ce-a133-4a39-a7cd-423ea9e884bc-S39/frameworks/9463b2ce-a133-4a39-a7cd-423ea9e884bc-0000/executors/executor.namenode.NameNodeExecutor.1449343025206/runs/802437d8-46d5-47d1-abd5-739850a5f3fc/hdfs-mesos-executor-0.1.5/share/hadoop/common/lib/commons-logging-1.1.3.jar:/tmp/mesos/slaves/9463b2ce-a133-4a39-a7cd-423ea9e884bc-S39/frameworks/9463b2ce-a133-4a39-a7cd-423ea9e884bc-0000/executors/executor.namenode.NameNodeExecutor.1449343025206/runs/802437d8-46d5-47d1-abd5-739850a5f3fc/hdfs-mesos-executor-0.1.5/share/hadoop/common/lib/jasper-runtime-5.5.23.jar:/tmp/mesos/slaves/9463b2ce-a133-4a39-a7cd-423ea9e884bc-S39/frameworks/9463b2ce-a133-4a39-a7cd-423ea9e884bc-0000/executors/executor.namenode.NameNodeExecutor.1449343025206/runs/802437d8-46d5-47d1-abd5-739850a5f3fc/hdfs-mesos-executor-0.1.5/share/hadoop/common/lib/jets3t-0.9.0.jar:/tmp/mesos/slaves/9463b2ce-a133-4a39-a7cd-423ea9e884bc-S39/frameworks/9463b2ce-a133-4a39-a7cd-423ea9e884bc-0000/executors/executor.namenode.NameNodeExecutor.1449343025206/runs/802437d8-46d5-47d1-abd5-739850a5f3fc/hdfs-mesos-executor-0.1.5/share/hadoop/common/hadoop-common-2.5.0-cdh5.3.1.jar:/tmp/mesos/slaves/9463b2ce-a133-4a39-a7cd-423ea9e884bc-S39/frameworks/9463b2ce-a133-4a39-a7cd-423ea9e884bc-0000/executors/executor.namenode.NameNodeExecutor.1449343025206/runs/802437d8-46d5-47d1-abd5-739850a5f3fc/hdfs-mesos-executor-0.1.5/share/hadoop/hdfs:/tmp/mesos/slaves/9463b2ce-a133-4a39-a7cd-423ea9e884bc-S39/frameworks/9463b2ce-a133-4a39-a7cd-423ea9e884bc-0000/executors/executor.namenode.NameNodeExecutor.1449343025206/runs/802437d8-46d5-47d1-abd5-739850a5f3fc/hdfs-mesos-executor-0.1.5/share/hadoop/hdfs/lib/jackson-core-asl-1.8.8.jar:/tmp/mesos/slaves/9463b2ce-a133-4a39-a7cd-423ea9e884bc-S39/frameworks/9463b2ce-a133-4a39-a7cd-423ea9e884bc-0000/executors/executor.namenode.NameNodeExecutor.1449343025206/runs/802437d8-46d5-47d1-abd5-739850a5f3fc/hdfs-mesos-executor-0.1.5/share/hadoop/hdfs/lib/servlet-api-2.5.jar:/tmp/mesos/slaves/9463b2ce-a133-4a39-a7cd-423ea9e884bc-S39/frameworks/9463b2ce-a133-4a39-a7cd-423ea9e884bc-0000/executors/executor.namenode.NameNodeExecutor.1449343025206/runs/802437d8-46d5-47d1-abd5-739850a5f3fc/hdfs-mesos-executor-0.1.5/share/hadoop/hdfs/lib/protobuf-java-2.5.0.jar:/tmp/mesos/slaves/9463b2ce-a133-4a39-a7cd-423ea9e884bc-S39/frameworks/9463b2ce-a133-4a39-a7cd-423ea9e884bc-0000/executors/executor.namenode.NameNodeExecutor.1449343025206/runs/802437d8-46d5-47d1-abd5-739850a5f3fc/hdfs-mesos-executor-0.1.5/share/hadoop/hdfs/lib/netty-3.6.2.Final.jar:/tmp/mesos/slaves/9463b2ce-a133-4a39-a7cd-423ea9e884bc-S39/frameworks/9463b2ce-a133-4a39-a7cd-423ea9e884bc-0000/executors/executor.namenode.NameNodeExecutor.1449343025206/runs/802437d8-46d5-47d1-abd5-739850a5f3fc/hdfs-mesos-executor-0.1.5/share/hadoop/hdfs/lib/jackson-mapper-asl-1.8.8.jar:/tmp/mesos/slaves/9463b2ce-a133-4a39-a7cd-423ea9e884bc-S39/frameworks/9463b2ce-a133-4a39-a7cd-423ea9e884bc-0000/executors/executor.namenode.NameNodeExecutor.1449343025206/runs/802437d8-46d5-47d1-abd5-739850a5f3fc/hdfs-mesos-executor-0.1.5/share/hadoop/hdfs/lib/asm-3.2.jar:/tmp/mesos/slaves/9463b2ce-a133-4a39-a7cd-423ea9e884bc-S39/frameworks/9463b2ce-a133-4a39-a7cd-423ea9e884bc-0000/executors/executor.namenode.NameNodeExecutor.1449343025206/runs/802437d8-46d5-47d1-abd5-739850a5f3fc/hdfs-mesos-executor-0.1.5/share/hadoop/hdfs/lib/jetty-6.1.26.cloudera.4.jar:/tmp/mesos/slaves/9463b2ce-a133-4a39-a7cd-423ea9e884bc-S39/frameworks/9463b2ce-a133-4a39-a7cd-423ea9e884bc-0000/executors/executor.namenode.NameNodeExecutor.1449343025206/runs/802437d8-46d5-47d1-abd5-739850a5f3fc/hdfs-mesos-executor-0.1.5/share/hadoop/hdfs/lib/jsr305-1.3.9.jar:/tmp/mesos/slaves/9463b2ce-a133-4a39-a7cd-423ea9e884bc-S39/frameworks/9463b2ce-a133-4a39-a7cd-423ea9e884bc-0000/executors/executor.namenode.NameNodeExecutor.1449343025206/runs/802437d8-46d5-47d1-abd5-739850a5f3fc/hdfs-mesos-executor-0.1.5/share/hadoop/hdfs/lib/commons-cli-1.2.jar:/tmp/mesos/slaves/9463b2ce-a133-4a39-a7cd-423ea9e884bc-S39/frameworks/9463b2ce-a133-4a39-a7cd-423ea9e884bc-0000/executors/executor.namenode.NameNodeExecutor.1449343025206/runs/802437d8-46d5-47d1-abd5-739850a5f3fc/hdfs-mesos-executor-0.1.5/share/hadoop/hdfs/lib/jetty-util-6.1.26.cloudera.4.jar:/tmp/mesos/slaves/9463b2ce-a133-4a39-a7cd-423ea9e884bc-S39/frameworks/9463b2ce-a133-4a39-a7cd-423ea9e884bc-0000/executors/executor.namenode.NameNodeExecutor.1449343025206/runs/802437d8-46d5-47d1-abd5-739850a5f3fc/hdfs-mesos-executor-0.1.5/share/hadoop/hdfs/lib/jersey-core-1.9.jar:/tmp/mesos/slaves/9463b2ce-a133-4a39-a7cd-423ea9e884bc-S39/frameworks/9463b2ce-a133-4a39-a7cd-423ea9e884bc-0000/executors/executor.namenode.NameNodeExecutor.1449343025206/runs/802437d8-46d5-47d1-abd5-739850a5f3fc/hdfs-mesos-executor-0.1.5/share/hadoop/hdfs/lib/guava-11.0.2.jar:/tmp/mesos/slaves/9463b2ce-a133-4a39-a7cd-423ea9e884bc-S39/frameworks/9463b2ce-a133-4a39-a7cd-423ea9e884bc-0000/executors/executor.namenode.NameNodeExecutor.1449343025206/runs/802437d8-46d5-47d1-abd5-739850a5f3fc/hdfs-mesos-executor-0.1.5/share/hadoop/hdfs/lib/xmlenc-0.52.jar:/tmp/mesos/slaves/9463b2ce-a133-4a39-a7cd-423ea9e884bc-S39/frameworks/9463b2ce-a133-4a39-a7cd-423ea9e884bc-0000/executors/executor.namenode.NameNodeExecutor.1449343025206/runs/802437d8-46d5-47d1-abd5-739850a5f3fc/hdfs-mesos-executor-0.1.5/share/hadoop/hdfs/lib/commons-codec-1.4.jar:/tmp/mesos/slaves/9463b2ce-a133-4a39-a7cd-423ea9e884bc-S39/frameworks/9463b2ce-a133-4a39-a7cd-423ea9e884bc-0000/executors/executor.namenode.NameNodeExecutor.1449343025206/runs/802437d8-46d5-47d1-abd5-739850a5f3fc/hdfs-mesos-executor-0.1.5/share/hadoop/hdfs/lib/commons-el-1.0.jar:/tmp/mesos/slaves/9463b2ce-a133-4a39-a7cd-423ea9e884bc-S39/frameworks/9463b2ce-a133-4a39-a7cd-423ea9e884bc-0000/executors/executor.namenode.NameNodeExecutor.1449343025206/runs/802437d8-46d5-47d1-abd5-739850a5f3fc/hdfs-mesos-executor-0.1.5/share/hadoop/hdfs/lib/commons-daemon-1.0.13.jar:/tmp/mesos/sla7cd-423ea9e884bc-S39/frameworks/9463b2ce-a133-4a39-a7cd-423ea9e884bc-0000/executors/executor.namenode.NameNodeExecutor.1449343025206/runs/802437d8-46d5-47d1-abd5-739850a5f3fc/hdfs-mesos-executor-0.1.5/share/hadoop/hdfs/lib/jsp-api-2.1.jar:/tmp/mesos/slaves/9463b2ce-a133-4a39-a7cd-423ea9e884bc-S39/frameworks/9463b2ce-a133-4a39-a7cd-423ea9e884bc-0000/executors/executor.namenode.NameNodeExecutor.1449343025206/runs/802437d8-46d5-47d1-abd5-739850a5f3fc/hdfs-mesos-executor-0.1.5/share/hadoop/hdfs/lib/commons-io-2.4.jar:/tmp/mesos/slaves/9463b2ce-a133-4a39-a7cd-423ea9e884bc-S39/frameworks/9463b2ce-a133-4a39-a7cd-423ea9e884bc-0000/executors/executor.namenode.NameNodeExecutor.1449343025206/runs/802437d8-46d5-47d1-abd5-739850a5f3fc/hdfs-mesos-executor-0.1.5/share/hadoop/hdfs/lib/log4j-1.2.17.jar:/tmp/mesos/slaves/9463b2ce-a133-4a39-a7cd-423ea9e884bc-S39/frameworks/9463b2ce-a133-4a39-a7cd-423ea9e884bc-0000/executors/executor.namenode.NameNodeExecutor.1449343025206/runs/802437d8-46d5-47d1-abd5-739850a5f3fc/hdfs-mesos-executor-0.1.5/share/hadoop/hdfs/lib/commons-lang-2.6.jar:/tmp/mesos/slaves/9463b2ce-a133-4a39-a7cd-423ea9e884bc-S39/frameworks/9463b2ce-a133-4a39-a7cd-423ea9e884bc-0000/executors/executor.namenode.NameNodeExecutor.1449343025206/runs/802437d8-46d5-47d1-abd5-739850a5f3fc/hdfs-mesos-executor-0.1.5/share/hadoop/hdfs/lib/jersey-server-1.9.jar:/tmp/mesos/slaves/9463b2ce-a133-4a39-a7cd-423ea9e884bc-S39/frameworks/9463b2ce-a133-4a39-a7cd-423ea9e884bc-0000/executors/executor.namenode.NameNodeExecutor.1449343025206/runs/802437d8-46d5-47d1-abd5-739850a5f3fc/hdfs-mesos-executor-0.1.5/share/hadoop/hdfs/lib/commons-logging-1.1.3.jar:/tmp/mesos/slaves/9463b2ce-a133-4a39-a7cd-423ea9e884bc-S39/frameworks/9463b2ce-a133-4a39-a7cd-423ea9e884bc-0000/executors/executor.namenode.NameNodeExecutor.1449343025206/runs/802437d8-46d5-47d1-abd5-739850a5f3fc/hdfs-mesos-executor-0.1.5/share/hadoop/hdfs/lib/jasper-runtime-5.5.23.jar:/tmp/mesos/slaves/9463b2ce-a133-4a39-a7cd-423ea9e884bc-S39/frameworks/9463b2ce-a133-4a39-a7cd-423ea9e884bc-0000/executors/executor.namenode.NameNodeExecutor.1449343025206/runs/802437d8-46d5-47d1-abd5-739850a5f3fc/hdfs-mesos-executor-0.1.5/share/hadoop/hdfs/hadoop-hdfs-2.5.0-cdh5.3.1.jar:/share/hadoop/yarn/*:/share/hadoop/mapreduce/*:/contrib/capacity-scheduler/*.jar
STARTUP_MSG:   build = http://github.com/cloudera/hadoop -r 4cda8416c73034b59cc8baafbe3666b074472846; compiled by 'jenkins' on 2015-01-28T00:37Z
STARTUP_MSG:   java = 1.7.0_76
************************************************************/
15/12/05 15:08:06 INFO namenode.NameNode: registered UNIX signal handlers for [TERM, HUP, INT]
15/12/05 15:08:06 INFO namenode.NameNode: createNameNode []
15/12/05 15:08:06 INFO ipc.Client: Retrying connect to server: mesos-slave-02/159.203.117.223:50071. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=1, sleepTime=1000 MILLISECONDS)
15/12/05 15:08:06 WARN ha.HealthMonitor: Transport-level exception trying to monitor health of NameNode at mesos-slave-02/159.203.117.223:50071: Call From mesos-slave-02/159.203.117.223 to mesos-slave-02:50071 failed on connection exception: java.net.ConnectException: Connection refused; For more details see:  http://wiki.apache.org/hadoop/ConnectionRefused
15/12/05 15:08:07 INFO impl.MetricsConfig: loaded properties from hadoop-metrics2.properties
15/12/05 15:08:07 INFO impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s).
15/12/05 15:08:07 INFO impl.MetricsSystemImpl: NameNode metrics system started
15/12/05 15:08:07 INFO namenode.NameNode: fs.defaultFS is file:///
15/12/05 15:08:08 INFO hdfs.DFSUtil: Starting web server as: ${dfs.web.authentication.kerberos.principal}
15/12/05 15:08:08 INFO hdfs.DFSUtil: Starting Web-server for hdfs at: http://mesos-slave-02:50070
15/12/05 15:08:08 INFO mortbay.log: Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
15/12/05 15:08:08 INFO http.HttpRequestLog: Http request log for http.requests.namenode is not defined
15/12/05 15:08:08 INFO http.HttpServer2: Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter)
15/12/05 15:08:08 INFO http.HttpServer2: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context hdfs
15/12/05 15:08:08 INFO http.HttpServer2: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs
15/12/05 15:08:08 INFO http.HttpServer2: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static
15/12/05 15:08:08 INFO http.HttpServer2: Added filter 'org.apache.hadoop.hdfs.web.AuthFilter' (class=org.apache.hadoop.hdfs.web.AuthFilter)
15/12/05 15:08:08 INFO http.HttpServer2: addJerseyResourcePackage: packageName=org.apache.hadoop.hdfs.server.namenode.web.resources;org.apache.hadoop.hdfs.web.resources, pathSpec=/webhdfs/v1/*
15/12/05 15:08:08 INFO http.HttpServer2: Jetty bound to port 50070
15/12/05 15:08:08 INFO mortbay.log: jetty-6.1.26.cloudera.4
15/12/05 15:08:09 INFO ipc.Client: Retrying connect to server: mesos-slave-02/159.203.117.223:50071. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=1, sleepTime=1000 MILLISECONDS)
15/12/05 15:08:09 WARN ha.HealthMonitor: Transport-level exception trying to monitor health of NameNode at mesos-slave-02/159.203.117.223:50071: Call From mesos-slave-02/159.203.117.223 to mesos-slave-02:50071 failed on connection exception: java.net.ConnectException: Connection refused; For more details see:  http://wiki.apache.org/hadoop/ConnectionRefused
15/12/05 15:08:09 INFO mortbay.log: Started HttpServer2$SelectChannelConnectorWithSafeStartup@mesos-slave-02:50070
15/12/05 15:08:09 WARN common.Util: Path /root/hdfsdata/name should be specified as a URI in configuration files. Please update hdfs configuration.
15/12/05 15:08:09 WARN common.Util: Path /root/hdfsdata/name should be specified as a URI in configuration files. Please update hdfs configuration.
15/12/05 15:08:09 WARN namenode.FSNamesystem: Only one image storage directory (dfs.namenode.name.dir) configured. Beware of data loss due to lack of redundant storage directories!
15/12/05 15:08:09 WARN common.Util: Path /root/hdfsdata/name should be specified as a URI in configuration files. Please update hdfs configuration.
15/12/05 15:08:09 WARN common.Util: Path /root/hdfsdata/name should be specified as a URI in configuration files. Please update hdfs configuration.
15/12/05 15:08:09 INFO namenode.FSNamesystem: No KeyProvider found.
15/12/05 15:08:09 INFO namenode.FSNamesystem: fsLock is fair:true
15/12/05 15:08:09 INFO blockmanagement.DatanodeManager: dfs.block.invalidate.limit=1000
15/12/05 15:08:09 INFO blockmanagement.DatanodeManager: dfs.namenode.datanode.registration.ip-hostname-check=false
15/12/05 15:08:09 INFO blockmanagement.BlockManager: dfs.namenode.startup.delay.block.deletion.sec is set to 000:00:00:00.000
15/12/05 15:08:09 INFO blockmanagement.BlockManager: The block deletion will start around 2015 Dec 05 15:08:09
15/12/05 15:08:09 INFO util.GSet: Computing capacity for map BlocksMap
15/12/05 15:08:09 INFO util.GSet: VM type       = 64-bit
15/12/05 15:08:09 INFO util.GSet: 2.0% max memory 3.9 GB = 80.6 MB
15/12/05 15:08:09 INFO util.GSet: capacity      = 2^23 = 8388608 entries
15/12/05 15:08:09 INFO blockmanagement.BlockManager: dfs.block.access.token.enable=false
15/12/05 15:08:09 INFO blockmanagement.BlockManager: defaultReplication         = 3
15/12/05 15:08:09 INFO blockmanagement.BlockManager: maxReplication             = 512
15/12/05 15:08:09 INFO blockmanagement.BlockManager: minReplication             = 1
15/12/05 15:08:09 INFO blockmanagement.BlockManager: maxReplicationStreams      = 2
15/12/05 15:08:09 INFO blockmanagement.BlockManager: shouldCheckForEnoughRacks  = false
15/12/05 15:08:09 INFO blockmanagement.BlockManager: replicationRecheckInterval = 3000
15/12/05 15:08:09 INFO blockmanagement.BlockManager: encryptDataTransfer        = false
15/12/05 15:08:09 INFO blockmanagement.BlockManager: maxNumBlocksToLog          = 1000
15/12/05 15:08:09 INFO namenode.FSNamesystem: fsOwner             = root (auth:SIMPLE)
15/12/05 15:08:09 INFO namenode.FSNamesystem: supergroup          = supergroup
15/12/05 15:08:09 INFO namenode.FSNamesystem: isPermissionEnabled = false
15/12/05 15:08:09 INFO namenode.FSNamesystem: Determined nameservice ID: hdfs
15/12/05 15:08:09 INFO namenode.FSNamesystem: HA Enabled: true
15/12/05 15:08:09 INFO namenode.FSNamesystem: Append Enabled: true
15/12/05 15:08:09 INFO util.GSet: Computing capacity for map INodeMap
15/12/05 15:08:09 INFO util.GSet: VM type       = 64-bit
15/12/05 15:08:09 INFO util.GSet: 1.0% max memory 3.9 GB = 40.3 MB
15/12/05 15:08:09 INFO util.GSet: capacity      = 2^22 = 4194304 entries
15/12/05 15:08:09 INFO namenode.NameNode: Caching file names occuring more than 10 times
15/12/05 15:08:09 INFO util.GSet: Computing capacity for map cachedBlocks
15/12/05 15:08:09 INFO util.GSet: VM type       = 64-bit
15/12/05 15:08:09 INFO util.GSet: 0.25% max memory 3.9 GB = 10.1 MB
15/12/05 15:08:09 INFO util.GSet: capacity      = 2^20 = 1048576 entries
15/12/05 15:08:09 INFO namenode.FSNamesystem: dfs.namenode.safemode.threshold-pct = 0.8999999761581421
15/12/05 15:08:09 INFO namenode.FSNamesystem: dfs.namenode.safemode.min.datanodes = 0
15/12/05 15:08:09 INFO namenode.FSNamesystem: dfs.namenode.safemode.extension     = 30000
15/12/05 15:08:09 INFO namenode.FSNamesystem: Retry cache on namenode is enabled
15/12/05 15:08:09 INFO namenode.FSNamesystem: Retry cache will use 0.03 of total heap and retry cache entry expiry time is 600000 millis
15/12/05 15:08:09 INFO util.GSet: Computing capacity for map NameNodeRetryCache
15/12/05 15:08:09 INFO util.GSet: VM type       = 64-bit
15/12/05 15:08:09 INFO util.GSet: 0.029999999329447746% max memory 3.9 GB = 1.2 MB
15/12/05 15:08:09 INFO util.GSet: capacity      = 2^17 = 131072 entries
15/12/05 15:08:09 INFO namenode.NNConf: ACLs enabled? false
15/12/05 15:08:09 INFO namenode.NNConf: XAttrs enabled? true
15/12/05 15:08:09 INFO namenode.NNConf: Maximum size of an xattr: 16384
15/12/05 15:08:10 INFO common.Storage: Lock on /root/hdfsdata/name/in_use.lock acquired by nodename 30704@mesos-slave-02
15/12/05 15:08:10 WARN namenode.FSNamesystem: Encountered exception loading fsimage
java.io.IOException: NameNode is not formatted.
    at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:212)
    at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:1005)
    at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:735)
    at org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:531)
    at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:587)
    at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:754)
    at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:738)
    at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1427)
    at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1493)
15/12/05 15:08:10 INFO mortbay.log: Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@mesos-slave-02:50070
15/12/05 15:08:10 INFO impl.MetricsSystemImpl: Stopping NameNode metrics system...
15/12/05 15:08:10 INFO impl.MetricsSystemImpl: NameNode metrics system stopped.
15/12/05 15:08:10 INFO impl.MetricsSystemImpl: NameNode metrics system shutdown complete.
15/12/05 15:08:10 FATAL namenode.NameNode: Failed to start namenode.
java.io.IOException: NameNode is not formatted.
    at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:212)
    at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:1005)
    at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:735)
    at org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:531)
    at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:587)
    at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:754)
    at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:738)
    at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1427)
    at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1493)
15/12/05 15:08:10 INFO util.ExitUtil: Exiting with status 1
15/12/05 15:08:10 INFO namenode.NameNode: SHUTDOWN_MSG: 
/************************************************************
SHUTDOWN_MSG: Shutting down NameNode at mesos-slave-02/159.203.117.223
************************************************************/
asdfsx commented 8 years ago

I met the same problem!!!

christobill commented 8 years ago

I had the same problem, 3 changes solved the issue for me:

  1. mesosdns in mesos-site.xml changed to true (but I guess you will need to deploy mesos-dns in marathon)
  2. all possible resources in mesos-site.xml to 256k and 0.05 cpu (!!)
  3. add 50070-50071 to mesos-slave resources (and 8020 if you need remote access)
masayoshi-louis commented 8 years ago

same problem

solved by using mesos-dns