Closed Rjiliang closed 4 years ago
@renrr This error does not affect the execution of the task. If the table does not exist, it is likely that the hive conf configuration is not configured properly.
@renrr This error does not affect the execution of the task. If the table does not exist, it is likely that the hive conf configuration is not configured properly.
@chaogefeng Thank you for your reply. Which path should I check for hive configuration
Describe the bug when I upgrap hive to 3.0.0, rebuild links, query in Scripts an warn occurs: WARN [hiveEngineEngine-Thread-2] com.webank.wedatasphere.linkis.engine.hive.executor.HiveDriverProxy$ 84 apply - java.lang.ClassNotFoundException: org.apache.hadoop.hive.ql.CommandNeedRetryException
To Reproduce upgrade hadoop version (HDP-3.1) hadoop-3.1.1 hive-3.0.0 rebulid DSS and links, Query in Scripts :select * from demo.dept limit 100
Screenshots error log: 44385: 2020-03-31 23:10:38-957 INFO [hiveEngineEngine-Thread-2] com.webank.wedatasphere.linkis.engine.hive.executor.HiveEngineExecutor com.webank.wedatasphere.linkis.engine.hive.executor.HiveEngineExecutor.executeLine(HiveEngineExecutor.scala:108) executeLine - hive client begins to run hql code: 44385: select * from demo.dept limit 100 44385: 2020-03-31 23:10:38-994 WARN [hiveEngineEngine-Thread-2] com.webank.wedatasphere.linkis.engine.hive.executor.HiveDriverProxy$ com.webank.wedatasphere.linkis.common.utils.Utils$$anonfun$tryAndWarn$1.apply(Utils.scala:84) apply - java.lang.ClassNotFoundException: org.apache.hadoop.hive.ql.CommandNeedRetryException 44385: at java.net.URLClassLoader.findClass(URLClassLoader.java:382) ~[?:1.8.0_201] 44385: at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[?:1.8.0_201] 44385: at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[?:1.8.0_201] 44385: at com.webank.wedatasphere.linkis.engine.hive.executor.HiveDriverProxy$$anonfun$2.apply(HiveEngineExecutor.scala:374) ~[linkis-hive-engine-0.9.3.jar:?] 44385: at com.webank.wedatasphere.linkis.engine.hive.executor.HiveDriverProxy$$anonfun$2.apply(HiveEngineExecutor.scala:374) ~[linkis-hive-engine-0.9.3.jar:?] 44385: at com.webank.wedatasphere.linkis.common.utils.Utils$.tryCatch(Utils.scala:48) [linkis-common-0.9.3.jar:?] 44385: at com.webank.wedatasphere.linkis.common.utils.Utils$.tryAndWarn(Utils.scala:74) [linkis-common-0.9.3.jar:?] 44385: at com.webank.wedatasphere.linkis.engine.hive.executor.HiveDriverProxy$.(HiveEngineExecutor.scala:372) [linkis-hive-engine-0.9.3.jar:?]
44385: at com.webank.wedatasphere.linkis.engine.hive.executor.HiveDriverProxy$.(HiveEngineExecutor.scala) [linkis-hive-engine-0.9.3.jar:?]
44385: at com.webank.wedatasphere.linkis.engine.hive.executor.HiveEngineExecutor$$anon$1.run(HiveEngineExecutor.scala:124) [linkis-hive-engine-0.9.3.jar:?]
44385: at com.webank.wedatasphere.linkis.engine.hive.executor.HiveEngineExecutor$$anon$1.run(HiveEngineExecutor.scala:121) [linkis-hive-engine-0.9.3.jar:?]
44385: at java.security.AccessController.doPrivileged(Native Method) [?:1.8.0_201]
44385: at javax.security.auth.Subject.doAs(Subject.java:422) [?:1.8.0_201]
44385: at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729) [hadoop-common-3.1.1.jar:?]
44385: at com.webank.wedatasphere.linkis.engine.hive.executor.HiveEngineExecutor.executeLine(HiveEngineExecutor.scala:121) [linkis-hive-engine-0.9.3.jar:?]
44385: at com.webank.wedatasphere.linkis.engine.execute.EngineExecutor$$anonfun$execute$1$$anonfun$apply$9$$anonfun$apply$10.apply(EngineExecutor.scala:141) [linkis-ujes-engine-0.9.3.jar:?]
44385: at com.webank.wedatasphere.linkis.engine.execute.EngineExecutor$$anonfun$execute$1$$anonfun$apply$9$$anonfun$apply$10.apply(EngineExecutor.scala:140) [linkis-ujes-engine-0.9.3.jar:?]
44385: at com.webank.wedatasphere.linkis.common.utils.Utils$.tryCatch(Utils.scala:48) [linkis-common-0.9.3.jar:?]
44385: at com.webank.wedatasphere.linkis.engine.execute.EngineExecutor$$anonfun$execute$1$$anonfun$apply$9.apply(EngineExecutor.scala:141) [linkis-ujes-engine-0.9.3.jar:?]
44385: at com.webank.wedatasphere.linkis.engine.execute.EngineExecutor$$anonfun$execute$1$$anonfun$apply$9.apply(EngineExecutor.scala:136) [linkis-ujes-engine-0.9.3.jar:?]
44385: at scala.collection.immutable.Range.foreach(Range.scala:160) [scala-library-2.11.8.jar:?]
44385: at com.webank.wedatasphere.linkis.engine.execute.EngineExecutor$$anonfun$execute$1.apply(EngineExecutor.scala:136) [linkis-ujes-engine-0.9.3.jar:?]
44385: at com.webank.wedatasphere.linkis.engine.execute.EngineExecutor$$anonfun$execute$1.apply(EngineExecutor.scala:118) [linkis-ujes-engine-0.9.3.jar:?]
44385: at com.webank.wedatasphere.linkis.common.utils.Utils$.tryFinally(Utils.scala:62) [linkis-common-0.9.3.jar:?]
44385: at com.webank.wedatasphere.linkis.scheduler.executer.AbstractExecutor.ensureIdle(AbstractExecutor.scala:60) [linkis-scheduler-0.9.3.jar:?]
44385: at com.webank.wedatasphere.linkis.scheduler.executer.AbstractExecutor.ensureIdle(AbstractExecutor.scala:54) [linkis-scheduler-0.9.3.jar:?]
44385: at com.webank.wedatasphere.linkis.engine.execute.EngineExecutor.ensureOp$1(EngineExecutor.scala:117) [linkis-ujes-engine-0.9.3.jar:?]
44385: at com.webank.wedatasphere.linkis.engine.execute.EngineExecutor.execute(EngineExecutor.scala:118) [linkis-ujes-engine-0.9.3.jar:?]
44385: at com.webank.wedatasphere.linkis.scheduler.queue.Job$$anonfun$3.apply(Job.scala:254) [linkis-scheduler-0.9.3.jar:?]
44385: at com.webank.wedatasphere.linkis.scheduler.queue.Job$$anonfun$3.apply(Job.scala:254) [linkis-scheduler-0.9.3.jar:?]
44385: at com.webank.wedatasphere.linkis.common.utils.Utils$.tryCatch(Utils.scala:48) [linkis-common-0.9.3.jar:?]
44385: at com.webank.wedatasphere.linkis.scheduler.queue.Job.run(Job.scala:254) [linkis-scheduler-0.9.3.jar:?]
44385: at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_201]
44385: at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_201]
44385: at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_201]
44385: at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_201]
44385: at java.lang.Thread.run(Thread.java:748) [?:1.8.0_201]
44385: Caused by: org.apache.hadoop.hive.ql.parse.SemanticException: Line 1:14 Table not found 'dept' 44385: at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.getMetaData(SemanticAnalyzer.java:2118) ~[hive-exec-3.0.0.jar:3.0.0] 44385: at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.getMetaData(SemanticAnalyzer.java:2038) ~[hive-exec-3.0.0.jar:3.0.0] 44385: at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genResolvedParseTree(SemanticAnalyzer.java:11954) ~[hive-exec-3.0.0.jar:3.0.0] 44385: at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:12049) ~[hive-exec-3.0.0.jar:3.0.0] 44385: at org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:334) ~[hive-exec-3.0.0.jar:3.0.0] 44385: at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:288) ~[hive-exec-3.0.0.jar:3.0.0] 44385: at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:644) ~[hive-exec-3.0.0.jar:3.0.0] 44385: at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1684) ~[hive-exec-3.0.0.jar:3.0.0] 44385: at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1631) ~[hive-exec-3.0.0.jar:3.0.0] 44385: at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1626) ~[hive-exec-3.0.0.jar:3.0.0] 44385: at org.apache.hadoop.hive.ql.reexec.ReExecDriver.compileAndRespond(ReExecDriver.java:126) ~[hive-exec-3.0.0.jar:3.0.0] 44385: at org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:214) ~[hive-exec-3.0.0.jar:3.0.0] 44385: at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_201] 44385: at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_201] 44385: at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_201] 44385: at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_201] 44385: at com.webank.wedatasphere.linkis.engine.hive.executor.HiveDriverProxy$$anonfun$run$3.apply(HiveEngineExecutor.scala:348) ~[linkis-hive-engine-0.9.3.jar:?] 44385: at com.webank.wedatasphere.linkis.engine.hive.executor.HiveDriverProxy$$anonfun$run$3.apply(HiveEngineExecutor.scala:348) ~[linkis-hive-engine-0.9.3.jar:?] 44385: at com.webank.wedatasphere.linkis.common.utils.Utils$.tryCatch(Utils.scala:48) ~[linkis-common-0.9.3.jar:?] 44385: at com.webank.wedatasphere.linkis.common.utils.Utils$.tryAndWarn(Utils.scala:74) ~[linkis-common-0.9.3.jar:?] 44385: at com.webank.wedatasphere.linkis.engine.hive.executor.HiveDriverProxy.run(HiveEngineExecutor.scala:347) ~[linkis-hive-engine-0.9.3.jar:?] 44385: at com.webank.wedatasphere.linkis.engine.hive.executor.HiveEngineExecutor$$anon$1.run(HiveEngineExecutor.scala:137) ~[linkis-hive-engine-0.9.3.jar:?] 44385: ... 27 more