resource_management.core.exceptions.Fail: Configuration parameter 'yarn-env' was not found in configurations dictionary!

[huanganqing]

the result of apply_blueprint.sh:

Traceback (most recent call last): File "/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/YARN/package/scripts/mapreduce2_client.py", line 93, in MapReduce2Client().execute() File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py", line 351, in execute method(env) File "/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/YARN/package/scripts/mapreduce2_client.py", line 39, in install import params File "/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/YARN/package/scripts/params.py", line 29, in from params_linux import * File "/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/YARN/package/scripts/params_linux.py", line 43, in import status_params File "/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/YARN/package/scripts/status_params.py", line 41, in yarn_pid_dir = format("{yarn_pid_dir_prefix}/{yarn_user}") File "/usr/lib/ambari-agent/lib/resource_management/libraries/functions/format.py", line 95, in format return ConfigurationFormatter().format(format_string, args, **result) File "/usr/lib/ambari-agent/lib/resource_management/libraries/functions/format.py", line 59, in format result_protected = self.vformat(format_string, args, all_params) File "/usr/lib/python2.7/string.py", line 563, in vformat result = self._vformat(format_string, args, kwargs, used_args, 2) File "/usr/lib/python2.7/string.py", line 596, in _vformat result.append(self.format_field(obj, format_spec)) File "/usr/lib/python2.7/string.py", line 613, in format_field return format(value, format_spec) File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/config_dictionary.py", line 73, in getattr raise Fail("Configuration parameter '" + self.name + "' was not found in configurations dictionary!") resource_management.core.exceptions.Fail: Configuration parameter 'yarn-env' was not found in configurations dictionary!

My config:

• host_group: "hdp-master" clients: ['ZOOKEEPER_CLIENT', 'HDFS_CLIENT', 'MAPREDUCE2_CLIENT'] services:

  • ZOOKEEPER_SERVER
  • NAMENODE
  • SECONDARY_NAMENODE
  • HISTORYSERVER
  • AMBARI_SERVER
  • METRICS_COLLECTOR
  • METRICS_GRAFANA
  • METRICS_MONITOR

• host_group: "hdp-slave" clients: ['ZOOKEEPER_CLIENT', 'HDFS_CLIENT', 'MAPREDUCE2_CLIENT'] services:

  • DATANODE
  • METRICS_MONITOR

The only thing I found related to this is:

[jira] [Updated] (AMBARI-21173) Not able to start Yarn services after restoring the configs to initial value http://mail-archives.apache.org/mod_mbox/ambari-issues/201708.mbox/%3CJIRA.13076667.1496353683000.5741.1502409960768@Atlassian.JIRA%3E

[alexandruanghel]

You have HISTORYSERVER which needs YARN. The dependencies are not automatically added when doing blueprint deployments.

[alexandruanghel]

Also, remove 'MAPREDUCE2_CLIENT' from the clients or simply add the YARN components.

[huanganqing]

Thanks a lots ! I It works.

And when I try to enable all services and install all kinds of client with "database: 'mysql'" and external_hostname: ''", the following is what i get from apply_blueprint.

Traceback (most recent call last): File "/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/HIVE/package/scripts/hive_metastore.py", line 201, in HiveMetastore().execute() File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py", line 351, in execute method(env) File "/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/HIVE/package/scripts/hive_metastore.py", line 61, in start create_metastore_schema() # execute without config lock File "/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/HIVE/package/scripts/hive.py", line 483, in create_metastore_schema user = params.hive_user File "/usr/lib/ambari-agent/lib/resource_management/core/base.py", line 166, in init self.env.run() File "/usr/lib/ambari-agent/lib/resource_management/core/environment.py", line 160, in run self.run_action(resource, action) File "/usr/lib/ambari-agent/lib/resource_management/core/environment.py", line 124, in run_action provider_action() File "/usr/lib/ambari-agent/lib/resource_management/core/providers/system.py", line 263, in action_run returns=self.resource.returns) File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 72, in inner result = function(command, kwargs) File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 102, in checked_call tries=tries, try_sleep=try_sleep, timeout_kill_strategy=timeout_kill_strategy, returns=returns) File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 150, in _call_wrapper result = _call(command, kwargs_copy) File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 314, in _call raise ExecutionFailed(err_msg, code, out, err) resource_management.core.exceptions.ExecutionFailed: Execution of 'export HIVE_CONF_DIR=/usr/hdp/current/hive-metastore/conf/ ; /usr/hdp/current/hive-server2/bin/schematool -initSchema -dbType mysql -userName hive -passWord [PROTECTED] -verbose' returned 1. SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/usr/hdp/3.0.1.0-187/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/usr/hdp/3.0.1.0-187/hadoop/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory] Metastore connection URL: jdbc:mysql://HDP-0.local:3306/hive Metastore Connection Driver : com.mysql.jdbc.Driver Metastore connection User: hive Sat Oct 13 02:56:04 CST 2018 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification. org.apache.hadoop.hive.metastore.HiveMetaException: Failed to get schema version. Underlying cause: java.sql.SQLException : Access denied for user 'hive'@'HDP-0' (using password: YES) SQL Error code: 1045 org.apache.hadoop.hive.metastore.HiveMetaException: Failed to get schema version. at org.apache.hadoop.hive.metastore.tools.HiveSchemaHelper.getConnectionToMetastore(HiveSchemaHelper.java:94) at org.apache.hive.beeline.HiveSchemaTool.getConnectionToMetastore(HiveSchemaTool.java:169) at org.apache.hive.beeline.HiveSchemaTool.testConnectionToMetastore(HiveSchemaTool.java:475) at org.apache.hive.beeline.HiveSchemaTool.doInit(HiveSchemaTool.java:581) at org.apache.hive.beeline.HiveSchemaTool.doInit(HiveSchemaTool.java:567) at org.apache.hive.beeline.HiveSchemaTool.main(HiveSchemaTool.java:1539) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.util.RunJar.run(RunJar.java:318) at org.apache.hadoop.util.RunJar.main(RunJar.java:232) Caused by: java.sql.SQLException: Access denied for user 'hive'@'HDP-0' (using password: YES) at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:957) at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3878) at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3814) at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:871) at com.mysql.jdbc.MysqlIO.proceedHandshakeWithPluggableAuthentication(MysqlIO.java:1694) at com.mysql.jdbc.MysqlIO.doHandshake(MysqlIO.java:1215) at com.mysql.jdbc.ConnectionImpl.coreConnect(ConnectionImpl.java:2255) at com.mysql.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:2286) at com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2085) at com.mysql.jdbc.ConnectionImpl.(ConnectionImpl.java:795) at com.mysql.jdbc.JDBC4Connection.(JDBC4Connection.java:44) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.mysql.jdbc.Util.handleNewInstance(Util.java:404) at com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:400) at com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:327) at java.sql.DriverManager.getConnection(DriverManager.java:664) at java.sql.DriverManager.getConnection(DriverManager.java:247) at org.apache.hadoop.hive.metastore.tools.HiveSchemaHelper.getConnectionToMetastore(HiveSchemaHelper.java:88) ... 11 more schemaTool failed

Underlying cause: java.sql.SQLException : Access denied for user 'hive'@'HDP-0' (using password: YES)

It seems i have to manually allow login from anywhere as user hive before apply blueprint. Am i right ?

[alexandruanghel]

Well, I assume this is a totally new cluster? Only this way it will work. Once Hive is on the blueprint_dynamic variable then by running ./prepare_nodes.sh the database will prepare.

Otherwise, if you're adding Hive after a full install, then you'll have to follow: https://docs.hortonworks.com/HDPDocuments/Ambari-2.7.1.0/administering-ambari/content/amb_using_hive_with_mysql.html

[huanganqing]

Yes, It's a new cluster and the error message is what i got from apply_blueprint.sh after finished prepare_nodes.sh, install_ambari.sh, configure_ambari.sh. It seems the database is installed in install_ambari.sh stage but the mysql is not allow hive to login.

Underlying cause: java.sql.SQLException : Access denied for user 'hive'@'HDP-0' (using password: YES)

I am not sure what's the right way to use this repository if hive need to be add after full install. If so, I can only allow to install basic HDFS cluster and manually add HIVE and other services right? Or I can still add service by apply_blueprint ? Thanks a lot.

[huanganqing]

Since i left external_hostname as blank and this repo does what he claims to install mysql on my ambari node, but i think there's something wrong about the creation of user/passwd/database.

[huanganqing]

after apply_blueprint and i got the following error. I check the databases installed on ambari node (due to blank external_hostname) and not found any hive database was created.

So my question is how do I perform the mysq_play.yml to perform the job it should be done before i execute apply_blueprint ? Thanks.

sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.mysql.jdbc.Util.handleNewInstance(Util.java:404) at com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:400) at com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:327) at java.sql.DriverManager.getConnection(DriverManager.java:664) at java.sql.DriverManager.getConnection(DriverManager.java:247) at org.apache.hadoop.hive.metastore.tools.HiveSchemaHelper.getConnectionToMetastore(HiveSchemaHelper.java:88) ... 11 more schemaTool failed

[alexandruanghel]

Hi, the following is the flow taken for the hive database and perms to be created, somewhere you have an issue. Please double check everything as this works fine for everyone else!

1) in the group_vars/all file you need to have HIVE_SERVER or HIVE_METASTORE as part of the blueprint variable. And of course blueprint_file set to 'blueprint_dynamic.j2'. The databases are not created when using a static blueprint since it's missing the information to do it.

2) this information is used to construct the hiveserver_hosts variable: https://github.com/hortonworks/ansible-hortonworks/blob/master/playbooks/set_variables.yml#L147

3) then you have to execute the prepare_nodes.sh script which applies the database role: https://github.com/hortonworks/ansible-hortonworks/blob/master/playbooks/prepare_nodes.yml#L19

4) in the database role, the hive database is created, only when the hiveserver_hosts variable is defined and not empty: https://github.com/hortonworks/ansible-hortonworks/blob/master/playbooks/roles/database/tasks/main.yml#L194

[huanganqing]

Hi, thanks for reply. Yes, I have HIVE_SERVER and HIVE_METASTORE in my blueprint_dynamic. I am running the script as the following sequence:

1. prepare_node.sh
2. install_ambari.sh
3. configure_ambari.sh
4. apply_blueprint.sh

so you mean i have to re-run prepare_node again before running apply_blueprint ? @@

[alexandruanghel]

No, I meant if all the variables are set as explained then it will work if you execute the scripts in that order (or simply ./install_cluster.sh).

But you cannot come back and edit stuff. These Ansible playbooks are for building a complete cluster, any further changes or additions have to be done after, outside these playbooks. You cannot use these playbooks to add new services, etc.

If you want more help I will first need these from you: 1) what OS you use or any public cloud? 2) the full contents of the group_vars/all file

If you want a more friendly way to install and manage clusters, including a GUI, you should check cloudbreak: https://docs.hortonworks.com/HDPDocuments/Cloudbreak/Cloudbreak-2.7.2/content/index.html https://github.com/hortonworks/cloudbreak

This is a fully supported Hortonworks product (unlike these Ansible playbooks), much more complete and much easier to use.

[alexandruanghel]

Hi, I haven't heard a reply back so I assume all is good now!? I'll close this but you can reopen it if you still have the issue and give the required information I previously asked. If this a genuine issue I would be more than happy to fix it, but I have to replicate and test first.