search

IKANOW / Aleph2

The IKANOW v2 meta-database and analytics platform
Apache License 2.0
2 stars 1 forks source link

HDP Version gets overwritten by Ambari #61

Closed robgil closed 8 years ago

robgil commented 8 years ago

The hdp version in the mapred-site.xml currently needs to be manually changed via the following sed script.

sed -i s/\${hdp.version}/2.2.4.2-2/g /usr/hdp/current/hadoop-client/conf/mapred-site.xml

The version is dependent on what is installed. This makes it variable between clusters when running different versions and across customer installs.

The overwrite by ambari causes parse errors as the settings are reverted. Since these are ambari generated configs, they should be loadable without errors or have a configurable version. Perhaps default to the latest installed version.

Error: Unable to parse '/hdp/apps/${hdp.version}/mapreduce/mapreduce.tar.gz#mr-framework'

This is only an issue when using ambari to manage the cluster and configs. Ambari explicitly manages configs and will overwrite them when anything changes within Ambari. Copying the configs and changing them brings us further into the quagmire as they then have to be manually managed.

Alex-Ikanow commented 8 years ago

I think we should set -Dhdp.version=$HDP_VERSION in the ikanow aleph2 service (or arguably the /etc/sysconfig/ikanow-aleph2 file when that exists), and then manager HDP_VERSION in salt

I'm just trying that out now by hand

Alex-Ikanow commented 8 years ago

(tested the above -D syntax and it worked)

Alex-Ikanow commented 8 years ago

Confirmed fixed