Open ehiggs opened 7 years ago
Hi @ehiggs I also have same problem
Wei
hi,
I have compiled HiBench with Hadoop-3.0.0-alpha3.
One thing you need to change is to add the hadoop-3.0 dependency to pom.xml at top folder:
One thing you need to change is to add the hadoop-3.0 dependency to pom.xml at top folder:
<hadoop.mr2.version>3.0.0-alpha3</hadoop.mr2.version>
With maven, this can generally be done using -Dhadoop.mr2.version=3.0.0-alpha3
for example. Then you don't need to edit the pom.xml
and can build from the command line.
Hello, I also have same problem,do you have solved this problem?if have, please give me some advice, THANK YOU!
This seems to happen with hadoop 3. I solved it removing a hadoop-hdfs-2.4.0.jar
that was in the classpath.
@beltran where is the classpath? Hibench or hadoop. Could you provide the details? Thanks!
@qiongsheng I had this problem while using https://github.com/hortonworks/hive-testbench. When running the benchmarks a bunch of jars are downloaded and placed in the classpath.
You may find the responsible jar by doing find ./HiBench -name hadoop-hdfs*.jar
. Rename it if something comes back and try again.
@beltran, I also have this issue in using hive-testbench, I move hadoop-hdfs*.jar to somewhere else. It works.
Hi. Is anyone having luck here? Thanks!
@ehiggs Do you have the solution about this problem?
Hello, I also have same problem,do you have solved this problem?if have, please give me some advice, THANK YOU!
@chengaofeng1 Do you have the solution about this problem?
@beltran, I also have this issue in using hive-testbench, I move hadoop-hdfs*.jar to somewhere else. It works.
But I can not find the jar in the HiBench.
@godtree, did you try to find the jar after building HiBench? I am not sure whether what I did is applicable to fixing your problem.
@godtree, did you try to find the jar after building HiBench? I am not sure whether what I did is applicable to fixing your problem.
@PHILO-HE There is not the jar after building HiBench. So it can not fix the problem. Thank you all the same.
I understand that HiBench currently only supports Hadoop2. But with Hadoop 3 being released soon it makes sense to have HiBench available for developers and administrators to know that there are no major regressions that would block the release or an upgrade. In fact, if, upon investigation, you see that these are regressions in compatibility, perhaps tickets should be files against the Apache Hadoop JIRA.
When I run dfsioe on a Hadoop 3 alpha4 (4b4a6524f2df3a891e9d5486ec39f7987766d84f) cluster built from Apache HEAD I get the following:
Environment information:
Thanks!