search

XiaoMi / minos

Minos is beyond a hadoop deployment system.
Apache License 2.0
522 stars 200 forks source link

add HADOOP_YARN_HOME for %service_env in deploy_yarn.py #16

Open hpttlook opened 10 years ago

hpttlook commented 10 years ago

Cloudera CDH4 is based on apache hadoop 2.0.0 version. Currently the latest version of hadoop is apache hadoop 2.2.0, big differences are exists between this two versions. In my test with minos + apache hadoop 2.2.0, the mapreduce job always failed due to some error like, "cannot find the main class xxxx, ClassNotDefined Exception".

After some debug things, it is proved the problem is caused by %service_env.

In hadoop 2.0.0, the follwing environment variables are used, HADOOP_HDFS_HOME HADOOP_COMMON_HOME YARN_HOME

But in hadoop 2.2.0, the following variales are used HADOOP_HDFS_HOME HADOOP_COMMON_HOME HADOOP_YARN_HOME HADOOP_MAPRED_HOME,

so adding HADOOP_YARN_HOME and HADOOP_MAPRED_HOME to service_env makes it works for both CDH4 and apache hadoop 2.2.0 versions.

wuzesheng commented 10 years ago

@renozhang Can you make sure of this issue? BTW: Thanks @hpttlook