search

Intel-bigdata / HiBench

HiBench is a big data benchmark suite.
Other
1.45k stars 761 forks source link

failed to run some workloads: #448

Open tyn513 opened 7 years ago

tyn513 commented 7 years ago

In prepare process, same Exception thrown as show in following:

nutchindexing

17/04/19 10:11:06 WARN mapreduce.JobResourceUploader: Hadoop command-line option parsing not performed. Implemen$ 17/04/19 10:11:09 INFO mapred.FileInputFormat: Total input paths to process : 1 17/04/19 10:11:09 INFO mapreduce.JobSubmitter: Cleaning up the staging area /tmp/hadoop-yarn/staging/hdfs/.stagi$ Exception in thread "main" org.apache.hadoop.ipc.RemoteException(java.lang.NullPointerException): java.lang.Null$ at org.apache.hadoop.hdfs.server.blockmanagement.DatanodeManager.sortLocatedBlocks(DatanodeManager.java:$ at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1769) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getBlockLocations(NameNodeRpcServer.java:588) ...

Pagerank:

17/04/19 10:58:54 INFO HiBench.PagerankData: Running Job: Create pagerank nodes 17/04/19 10:58:54 INFO HiBench.PagerankData: Dummy file hdfs://Master00:9000/HiBench/Pagerank/temp/dummy as input 17/04/19 10:58:54 INFO HiBench.PagerankData: Vertices file hdfs://Master00:9000/HiBench/Pagerank/Input/vertices $ 17/04/19 10:58:55 INFO client.RMProxy: Connecting to ResourceManager at Master00/192.168.2.81:8032 17/04/19 10:58:55 INFO client.RMProxy: Connecting to ResourceManager at Master00/192.168.2.81:8032 17/04/19 10:58:55 WARN mapreduce.JobResourceUploader: Hadoop command-line option parsing not performed. Implemen$ 17/04/19 10:58:58 INFO mapred.FileInputFormat: Total input paths to process : 1 17/04/19 10:58:58 INFO mapreduce.JobSubmitter: Cleaning up the staging area /tmp/hadoop-yarn/staging/hdfs/.stagi$ Exception in thread "main" org.apache.hadoop.ipc.RemoteException(java.lang.NullPointerException): java.lang.Null$ at org.apache.hadoop.hdfs.server.blockmanagement.DatanodeManager.sortLocatedBlocks(DatanodeManager.java:$ at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1769) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getBlockLocations(NameNodeRpcServer.java:588) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getBlockLocations(Clie$ at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlock$ at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616) ...

Only a few workload such as wordcount,sord can successfully finish Please help me! thank you a lot

carsonwang commented 7 years ago

This looks like your own environment issue. Please check our HDFS status.