Hadoop Q&A list

[lqshow]

hadoop fs -ls 提示错误

Call From linqiongs-MacBook-Pro.local/192.168.0.101 to localhost:9000 failed on connection exception: java.net.ConnectException: Connection refused; For more details see:  http://wiki.apache.org/hadoop/ConnectionRefused

A：未启动hadoop

Name node is in safe mode

# A: 
hadoop dfsadmin -safemode leave

启动hadoop时，log中出现：java.io.IOException: NameNode is not formatted

访问localhost:50070失败，说明namenode启动失败

本机环境不考虑数据情况下，暴力解决

hadoop namenode -format

如果直接格式化还是不行，则需要先删除namenode历史的文件，再格式化

namenode启动日志：/opt/hadoop/hadoop-2.6.0-cdh5.6.0/logs/yarn-linqiong-nodemanager-linqiongs-MacBook-Pro.local.log

hadoop No FileSystem for scheme: file

Java程序读写HDFS文件的错误

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-shade-plugin</artifactId>
    <version>2.3</version>
    <executions>
      <execution>
        <phase>package</phase>
        <goals>
          <goal>shade</goal>
        </goals>
        <configuration>
          <transformers>
            <transformer implementation="org.apache.maven.plugins.shade.resource.ServicesResourceTransformer"/>
          </transformers>
        </configuration>
      </execution>
    </executions>
  </plugin>

Reference

hadoop No FileSystem for scheme: file