search

apache / linkis

Apache Linkis builds a computation middleware layer to facilitate connection, governance and orchestration between the upper applications and the underlying data engines.
https://linkis.apache.org/
Apache License 2.0
3.3k stars 1.17k forks source link

visual scriptis explore data, sparkenginemanage is not log，in visualis server log show error #180

Closed xccoder closed 4 years ago

xccoder commented 4 years ago

Describe the bug

2019-12-11 14:12:52,258 INFO  (Visualis_rootConsumerThread) INFO EntranceExecutionJob - job:Visualis_root_21 is closing
java.lang.reflect.UndeclaredThrowableException
    at com.webank.wedatasphere.dss.visualis.entrance.spark.SparkEntranceExecutor$$EnhancerBySpringCGLIB$$c66038cf.querySQLWithResultSetLocation(<generated>)
    at edp.davinci.service.impl.ViewServiceImpl.buildScala(ViewServiceImpl.java:788)
    at edp.davinci.service.impl.ViewServiceImpl.getResultDataList(ViewServiceImpl.java:698)
    at edp.davinci.service.impl.ViewServiceImpl.getData(ViewServiceImpl.java:568)2019-12-11 14:12:52,259 INFO  (Visualis_rootConsumerThread) INFO VisualisJob - Visualis_root_21 execute failed. Reason: Request engine failed, possibly due to insufficient resources or background process error(请求引擎失败，可能是由于资源不足或后台进程错误)!.
java.lang.NoSuchMethodError: org.json4s.jackson.JsonMethods$.parse(Lorg/json4s/JsonInput;Z)Lorg/json4s/JsonAST$JValue;
    at com.webank.wedatasphere.linkis.rpc.transform.JavaMapSerializer$$anonfun$$lessinit$greater$2$$anonfun$apply$4.applyOrElse(JavaCollectionSerializer.scala:39)
    at scala.PartialFunction$OrElse.apply(PartialFunction.scala:167)
    at scala.PartialFunction$class.applyOrElse(PartialFunction.scala:123)
    at scala.collection.AbstractMap.applyOrElse(Map.scala:59)
    at scala.PartialFunction$OrElse.apply(PartialFunction.scala:167)
    at org.json4s.Extraction$.internalDecomposeWithBuilder(Extraction.scala:146)
    at org.json4s.Extraction$.addField$1(Extraction.scala:110)
    at org.json4s.Extraction$.decomposeObject$1(Extraction.scala:140)
    at org.json4s.Extraction$.internalDecomposeWithBuilder(Extraction.scala:228)
    at org.json4s.Extraction$.decomposeWithBuilder(Extraction.scala:64)
    at org.json4s.Extraction$.decompose(Extraction.scala:242)
    at org.json4s.jackson.Serialization$.write(Serialization.scala:22)
    at com.webank.wedatasphere.linkis.rpc.transform.RPCProduct$$anon$1.toMessage(RPCProduct.scala:61)
    at com.webank.wedatasphere.linkis.rpc.BaseRPCSender$$anonfun$ask$1.apply(BaseRPCSender.scala:84)
    at com.webank.wedatasphere.linkis.rpc.interceptor.BaseRPCInterceptorExchange.invoke(RPCInterceptorExchange.scala:32)
    at com.webank.wedatasphere.linkis.rpc.interceptor.common.CommonRPCInterceptor.intercept(CommonRPCInterceptor.scala:29)
    at com.webank.wedatasphere.linkis.rpc.interceptor.BaseRPCInterceptorChain.handle(RPCInterceptorChain.scala:35)
    at com.webank.wedatasphere.linkis.rpc.interceptor.common.RetryableRPCInterceptor$$anonfun$intercept$1.apply(RetryableRPCInterceptor.scala:53)
    at com.webank.wedatasphere.linkis.common.utils.Utils$.tryCatch(Utils.scala:48)
    at com.webank.wedatasphere.linkis.common.utils.RetryHandler$class.retry(RetryHandler.scala:56)
    at com.webank.wedatasphere.linkis.rpc.interceptor.common.RetryableRPCInterceptor$RPCRetryHandler.retry(RetryableRPCInterceptor.scala:58)
    at com.webank.wedatasphere.linkis.rpc.interceptor.common.RetryableRPCInterceptor.intercept(RetryableRPCInterceptor.scala:53)
    at com.webank.wedatasphere.linkis.rpc.interceptor.BaseRPCInterceptorChain.handle(RPCInterceptorChain.scala:35)
    at com.webank.wedatasphere.linkis.rpc.interceptor.common.CacheableRPCInterceptor.intercept(CacheableRPCInterceptor.scala:57)
    at com.webank.wedatasphere.linkis.rpc.interceptor.BaseRPCInterceptorChain.handle(RPCInterceptorChain.scala:35)
    at com.webank.wedatasphere.linkis.rpc.interceptor.common.BroadcastRPCInterceptor.intercept(BroadcastRPCInterceptor.scala:74)
    at com.webank.wedatasphere.linkis.rpc.interceptor.BaseRPCInterceptorChain.handle(RPCInterceptorChain.scala:35)
    at com.webank.wedatasphere.linkis.rpc.BaseRPCSender.execute(BaseRPCSender.scala:79)
    at com.webank.wedatasphere.linkis.rpc.BaseRPCSender.ask(BaseRPCSender.scala:83)
    at com.webank.wedatasphere.linkis.entrance.execute.EngineRequester.request(EngineRequester.scala:51)
    at com.webank.wedatasphere.linkis.entrance.execute.EntranceExecutorManager.createExecutor(EntranceExecutorManager.scala:66)
    at com.webank.wedatasphere.linkis.entrance.execute.EntranceExecutorManager$$anonfun$askExecutor$1.apply(EntranceExecutorManager.scala:112)
    at com.webank.wedatasphere.linkis.entrance.execute.EntranceExecutorManager$$anonfun$askExecutor$1.apply(EntranceExecutorManager.scala:111)
    at scala.Option.orElse(Option.scala:289)
    at com.webank.wedatasphere.linkis.entrance.execute.EntranceExecutorManager.askExecutor(EntranceExecutorManager.scala:111)
    at com.webank.wedatasphere.linkis.entrance.execute.EntranceExecutorManager$$anonfun$4.apply(EntranceExecutorManager.scala:129)
    at com.webank.wedatasphere.linkis.entrance.execute.EntranceExecutorManager$$anonfun$4.apply(EntranceExecutorManager.scala:129)
    at com.webank.wedatasphere.linkis.common.utils.Utils$.tryCatch(Utils.scala:48)
    at com.webank.wedatasphere.linkis.entrance.execute.EntranceExecutorManager.askExecutor(EntranceExecutorManager.scala:129)
    at com.webank.wedatasphere.linkis.scheduler.queue.fifoqueue.FIFOUserConsumer$$anonfun$loop$3$$anonfun$apply$3$$anonfun$apply$mcV$sp$1$$anonfun$apply$mcZ$sp$2.apply(FIFOUserConsumer.scala:113)
    at com.webank.wedatasphere.linkis.scheduler.queue.fifoqueue.FIFOUserConsumer$$anonfun$loop$3$$anonfun$apply$3$$anonfun$apply$mcV$sp$1$$anonfun$apply$mcZ$sp$2.apply(FIFOUserConsumer.scala:113)
    at com.webank.wedatasphere.linkis.common.utils.Utils$.tryCatch(Utils.scala:48)
    at com.webank.wedatasphere.linkis.scheduler.queue.fifoqueue.FIFOUserConsumer$$anonfun$loop$3$$anonfun$apply$3$$anonfun$apply$mcV$sp$1.apply$mcZ$sp(FIFOUserConsumer.scala:113)
    at com.webank.wedatasphere.linkis.common.utils.Utils$.aux$1(Utils.scala:191)
    at com.webank.wedatasphere.linkis.common.utils.Utils$.waitUntil(Utils.scala:204)
    at com.webank.wedatasphere.linkis.common.utils.Utils$.waitUntil(Utils.scala:207)
    at com.webank.wedatasphere.linkis.scheduler.queue.fifoqueue.FIFOUserConsumer$$anonfun$loop$3$$anonfun$apply$3.apply$mcV$sp(FIFOUserConsumer.scala:112)
    at com.webank.wedatasphere.linkis.scheduler.queue.fifoqueue.FIFOUserConsumer$$anonfun$loop$3$$anonfun$apply$3.apply(FIFOUserConsumer.scala:108)
    at com.webank.wedatasphere.linkis.scheduler.queue.fifoqueue.FIFOUserConsumer$$anonfun$loop$3$$anonfun$apply$3.apply(FIFOUserConsumer.scala:108)
    at com.webank.wedatasphere.linkis.common.utils.Utils$.tryCatch(Utils.scala:48)
    at com.webank.wedatasphere.linkis.scheduler.queue.fifoqueue.FIFOUserConsumer$$anonfun$loop$3.apply(FIFOUserConsumer.scala:133)
    at com.webank.wedatasphere.linkis.scheduler.queue.fifoqueue.FIFOUserConsumer$$anonfun$loop$3.apply(FIFOUserConsumer.scala:107)
    at scala.Option.foreach(Option.scala:257)
    at com.webank.wedatasphere.linkis.scheduler.queue.fifoqueue.FIFOUserConsumer.loop(FIFOUserConsumer.scala:107)
    at com.webank.wedatasphere.linkis.scheduler.queue.fifoqueue.FIFOUserConsumer$$anonfun$run$1.apply$mcV$sp(FIFOUserConsumer.scala:74)
    at com.webank.wedatasphere.linkis.scheduler.queue.fifoqueue.FIFOUserConsumer$$anonfun$run$1.apply(FIFOUserConsumer.scala:74)
    at com.webank.wedatasphere.linkis.scheduler.queue.fifoqueue.FIFOUserConsumer$$anonfun$run$1.apply(FIFOUserConsumer.scala:74)
    at com.webank.wedatasphere.linkis.common.utils.Utils$.tryCatch(Utils.scala:48)
    at com.webank.wedatasphere.linkis.common.utils.Utils$.tryAndError(Utils.scala:102)
    at com.webank.wedatasphere.linkis.scheduler.queue.fifoqueue.FIFOUserConsumer.run(FIFOUserConsumer.scala:74)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)

    at edp.davinci.service.impl.ViewServiceImpl$$FastClassBySpringCGLIB$$fc616aa2.invoke(<generated>)
    at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
    at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:684)
    at edp.davinci.service.impl.ViewServiceImpl$$EnhancerBySpringCGLIB$$c163cd33.getData(<generated>)
    at edp.davinci.controller.ViewController.getData(ViewController.java:312)
    at sun.reflect.GeneratedMethodAccessor269.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:209)
    at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:136)
    at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:97)
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:827)
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:738)
    at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85)2019-12-11 14:12:52,259 WARN  (Visualis_rootConsumerThread) WARN FIFOUserConsumer - Ask executor for Job Visualis_root_21 failed!
java.lang.NoSuchMethodError: org.json4s.jackson.JsonMethods$.parse(Lorg/json4s/JsonInput;Z)Lorg/json4s/JsonAST$JValue;

Analysis: by log judge should be json4s jar conflict

peacewong commented 4 years ago

Hello, we have optimized the package conflict problem