search

Intel-bigdata / HiBench

HiBench is a big data benchmark suite.
Other
1.45k stars 761 forks source link

I am facing the below mentioned issue while trying to execute the pagerank algorithm in hadoop from hibench. Please give a solution. #710

Closed MeAnkitaDutta closed 2 years ago

MeAnkitaDutta commented 2 years ago

patching args= Parsing conf: /home/hadoopusr/HiBench-7.1.1/conf/hadoop.conf Parsing conf: /home/hadoopusr/HiBench-7.1.1/conf/hibench.conf Parsing conf: /home/hadoopusr/HiBench-7.1.1/conf/workloads/websearch/pagerank.conf probe sleep jar: /usr/local/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-3.2.2-tests.jar start HadoopPreparePagerank bench hdfs rm -r: /usr/local/hadoop/bin/hadoop --config /usr/local/hadoop/etc/hadoop fs -rm -r -skipTrash hdfs://ankita-OptiPlex-9020:9000/HiBench/Pagerank/Input rm: `hdfs://ankita-OptiPlex-9020:9000/HiBench/Pagerank/Input': No such file... Submit MapReduce Job: /usr/local/hadoop/bin/hadoop --config /usr/local/hadoop/etc/hadoop jar /home/hadoopusr/HiBench-7.1.1/autogen/target/autogen-7.1.1-jar-with-dependencies.jar HiBench.DataGen -t pagerank -b hdfs://ankita-OptiPlex-9020:9000/HiBench/Pagerank -n Input -m 8 -r 8 -p 50 -pbalance -pbalance -o text at org.apache.hadoop.util.RunJar.main(RunJar.java:236) ERROR: Hadoop job /home/hadoopusr/HiBench-7.1.1/autogen/target/autogen-7.1.1-jar-with-dependencies.jar HiBench.DataGen failed to run successfully. Hint: You can goto /home/hadoopusr/HiBench-7.1.1/report/pagerank/prepare/conf/../bench.log to check for detailed log. Opening log tail for you:

at HiBench.PagerankData.generate(PagerankData.java:311)
at HiBench.DataGen.run(DataGen.java:24)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
at HiBench.DataGen.main(DataGen.java:45)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:323)
at org.apache.hadoop.util.RunJar.main(RunJar.java:236)
MeAnkitaDutta commented 2 years ago

The issue is resolved now. It occurred because HiBench 7.1.1 is not supported by Hadoop 3.2.2. It is only supported by Hadoop 2.x .