search

Intel-bigdata / HiBench

HiBench is a big data benchmark suite.
Other
1.46k stars 765 forks source link

Can Hibench run on a hadoop cluster with only one datanode? #680

Open Crazyoumashu opened 3 years ago

Crazyoumashu commented 3 years ago

I start a hadoop cluster with one namenode and one datanode. When I try to run prepare.sh of hibench, the mapreduce task is stuck and can not make any progress. Is this a feature of Hibench or just my priblem?

BEbillionaireUSD commented 3 years ago

It can but you need to stop the yarn mode, that is, keep the mapred.xml file empty and use your one local host.

BEbillionaireUSD commented 3 years ago

I found you are a Chinese and I do not want to challenge myself in English, so... 首先，配置hadoop的mapred.xml文件时，可以直接让它空白。然后就是core-site.xml和hdfs-site.xml都用localhost HiBench的配置文件中不要用yarn-client模式

Crazyoumashu commented 3 years ago

I found you are a Chinese and I do not want to challenge myself in English, so... 首先，配置hadoop的mapred.xml文件时，可以直接让它空白。然后就是core-site.xml和hdfs-site.xml都用localhost HiBench的配置文件中不要用yarn-client模式

谢谢老铁。你说的这种解决方案是单个机器既当namenode又当datanode的情况吧，确实可以解决问题。但是我初始是两台机器，一台当namenode，一台当datanode，这样的设定下是否是不能成功运行HiBench的？这个问题只是好奇，因为我可以用单机器跑实验了，再次谢谢您的帮助。

BEbillionaireUSD commented 3 years ago

I found you are a Chinese and I do not want to challenge myself in English, so... 首先，配置hadoop的mapred.xml文件时，可以直接让它空白。然后就是core-site.xml和hdfs-site.xml都用localhost HiBench的配置文件中不要用yarn-client模式

谢谢老铁。你说的这种解决方案是单个机器既当namenode又当datanode的情况吧，确实可以解决问题。但是我初始是两台机器，一台当namenode，一台当datanode，这样的设定下是否是不能成功运行HiBench的？这个问题只是好奇，因为我可以用单机器跑实验了，再次谢谢您的帮助。

我是一台同时拥有namenode和datanode，另一台只有datanode，你说的这种情况我也不太清楚，不好意思学艺不精😂