search

perwendel / spark

A simple expressive web framework for java. Spark has a kotlin DSL https://github.com/perwendel/spark-kotlin
Apache License 2.0
9.64k stars 1.56k forks source link

kerberos authentication in spark 2.2.0 #972

Closed ghost closed 5 years ago

ghost commented 6 years ago

version of spark: 2.2.0 the method to commit spark task is: new SparkLauncher(); we get exception like： Exception in thread "main" org.apache.spark.SparkException: Unable to load YARN support at org.apache.spark.deploy.SparkHadoopUtil$.liftedTree1$1(SparkHadoopUtil.scala:405) Caused by: java.lang.IllegalArgumentException: Can't get Kerberos realm Caused by: KrbException: Cannot locate default realm at sun.security.krb5.Config.getDefaultRealm(Config.java:1006) ... 31 more Caused by: KrbException: Generic error (description in e-text) (60) - Unable to locate Kerberos realm at sun.security.krb5.Config.getRealmFromDNS(Config.java:1102)

FireArrow commented 6 years ago

This is the web framework Spark, not to be confused with Apache Spark (https://spark.apache.org/). You have problem with Apache Spark. (You seem to be missing /etc/krb.conf, but I'm not sure, and I can't help you further here).