[EXPERIMENTAL] This repo includes deployment instructions for running HDFS/Spark inside docker containers. Also includes spark-notebook and HDFS FileBrowser.
689
stars
374
forks
source link
Directory /hadoop/dfs/name is in an inconsistent state: storage directory does not exist or is not accessible. #65
My solution for now: Changed the docker-compose.yml file and added "priviledged: true" setting to the nodes configured there
namenode | org.apache.hadoop.hdfs.server.common.InconsistentFSStateException: Directory /hadoop/dfs/name is in an inconsistent state: storage directory does not exist or is not accessible.
namenode | at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverStorageDirs(FSImage.java:322)
namenode | at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:210)
namenode | at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:1012)
namenode | at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:691)
namenode | at org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:634)
namenode | at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:695)
namenode | at org.apache.hadoop.hdfs.server.namenode.NameNode.(NameNode.java:898)
namenode | at org.apache.hadoop.hdfs.server.namenode.NameNode.(NameNode.java:877)
namenode | at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1603)
namenode | at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1671)
Problem - see stacktrace below
My solution for now: Changed the docker-compose.yml file and added "priviledged: true" setting to the nodes configured there
namenode | org.apache.hadoop.hdfs.server.common.InconsistentFSStateException: Directory /hadoop/dfs/name is in an inconsistent state: storage directory does not exist or is not accessible. namenode | at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverStorageDirs(FSImage.java:322) namenode | at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:210) namenode | at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:1012) namenode | at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:691) namenode | at org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:634) namenode | at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:695) namenode | at org.apache.hadoop.hdfs.server.namenode.NameNode.(NameNode.java:898)
namenode | at org.apache.hadoop.hdfs.server.namenode.NameNode.(NameNode.java:877)
namenode | at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1603)
namenode | at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1671)