lordk911
commented
3 years ago 环境是HDP3.1.4 对应的版本是spark2.3.2，hadoop3, 需要在编译linkis时指定spark版本，否则json4s相关包版本有冲突。
重新安装links 后，还需要把hadoop-hdfs-client-x.x.x.jar 拷贝到linkis下，否则看不到查询结果。
查询出结果过后,点击“可视化”按钮，报错：在visualis-server下查看日志：
org.eclipse.jetty.server.HttpChannel 573 handleException - /api/rest_j/v1/visualis/view java.lang.NoSuchMethodError: javax.ws.rs.core.Application.getProperties()Ljava/util/Map
由于路径上有多个jersey版本，删掉1.x的版本，又由于 https://github.com/spring-cloud/spring-cloud-netflix/issues/2756 问题，需要升级eureka相关的包：
ll eureka*
-rw-r--r--. 1 bigtop bigtop 409687 Nov 26 2019 eureka-client-1.7.2.jar.bak
-rw-r--r--. 1 root root 438867 Apr 26 18:06 eureka-client-1.9.2.jar
-rw-r--r--. 1 bigtop bigtop 252899 Nov 26 2019 eureka-core-1.7.2.jar.bak
-rw-r--r--. 1 root root 253141 Apr 26 18:06 eureka-core-1.9.2.jar
重启visualis-server，点击“可视化”按钮，报错：
2021-04-26 18:10:30.985 INFO [qtp1425306446-95] com.webank.wedatasphere.linkis.server.restful.RestfulApplication 48 <init> - packages com.webank.wedatasphere.linkis.entrance.restful,com.webank.wedatasphere.dss.visualis.restful 2021-04-26 18:10:31.752 ERROR [qtp1425306446-95] com.webank.wedatasphere.dss.visualis.restful.ViewRestfulApi 119 createView - 可视化分析失败： java.lang.IndexOutOfBoundsException: Index: 0, Size: 0 at java.util.ArrayList.rangeCheck(ArrayList.java:659) ~[?:1.8.0_281] at java.util.ArrayList.get(ArrayList.java:435) ~[?:1.8.0_281] at com.webank.wedatasphere.dss.visualis.restful.ViewRestfulApi.createView(ViewRestfulApi.java:72) [visualis-server-0.5.0.jar:?] at com.webank.wedatasphere.dss.visualis.restful.ViewRestfulApi$$FastClassBySpringCGLIB$$b7006be3.invoke(<generated>) [visualis-server-0.5.0.jar:?] at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) [spring-core-5.0.8.RELEASE.jar:5.0.8.RELEASE] at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:684) [spring-aop-5.0.7.RELEASE.jar:5.0.7.RELEASE] at com.webank.wedatasphere.dss.visualis.restful.ViewRestfulApi$$EnhancerBySpringCGLIB$$2cecced1.createView(<generated>) [visualis-server-0.5.0.jar:?] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_281] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_281] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_281] at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_281] at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81) [jaxrs-ri-2.21.jar:2.21.] at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:144) [jaxrs-ri-2.21.jar:2.21.] at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:161) [jaxrs-ri-2.21.jar:2.21.] at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:160) [jaxrs-ri-2.21.jar:2.21.] at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:99) [jaxrs-ri-2.21.jar:2.21.] at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:389) [jaxrs-ri-2.21.jar:2.21.] at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:347) [jaxrs-ri-2.21.jar:2.21.] at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:102) [jaxrs-ri-2.21.jar:2.21.] at org.glassfish.jersey.server.ServerRuntime$2.run(ServerRuntime.java:309) [jaxrs-ri-2.21.jar:2.21.] at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271) [jaxrs-ri-2.21.jar:2.21.] at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267) [jaxrs-ri-2.21.jar:2.21.] at org.glassfish.jersey.internal.Errors.process(Errors.java:315) [jaxrs-ri-2.21.jar:2.21.] at org.glassfish.jersey.internal.Errors.process(Errors.java:297) [jaxrs-ri-2.21.jar:2.21.] at org.glassfish.jersey.internal.Errors.process(Errors.java:267) [jaxrs-ri-2.21.jar:2.21.] at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:317) [jaxrs-ri-2.21.jar:2.21.] at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:292) [jaxrs-ri-2.21.jar:2.21.] at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1139) [jaxrs-ri-2.21.jar:2.21.] at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:460) [jaxrs-ri-2.21.jar:2.21.] at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:386) [jaxrs-ri-2.21.jar:2.21.] at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:334) [jaxrs-ri-2.21.jar:2.21.] at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:221) [jaxrs-ri-2.21.jar:2.21.] at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:865) [jetty-servlet-9.4.11.v20180605.jar:9.4.11.v20180605] at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1655) [jetty-servlet-9.4.11.v20180605.jar:9.4.11.v20180605] at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:215) [websocket-server-9.4.11.v20180605.jar:9.4.11.v20180605] at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) [jetty-servlet-9.4.11.v20180605.jar:9.4.11.v20180605] at com.webank.wedatasphere.linkis.server.security.SecurityFilter.doFilter(SecurityFilter.scala:100) [linkis-module-0.9.3.jar:?] at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) [jetty-servlet-9.4.11.v20180605.jar:9.4.11.v20180605] at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:200) [spring-web-5.0.7.RELEASE.jar:5.0.7.RELEASE] at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-5.0.7.RELEASE.jar:5.0.7.RELEASE] at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) [jetty-servlet-9.4.11.v20180605.jar:9.4.11.v20180605] at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.filterAndRecordMetrics(WebMvcMetricsFilter.java:158) [spring-boot-actuator-2.0.3.RELEASE.jar:2.0.3.RELEASE] at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.filterAndRecordMetrics(WebMvcMetricsFilter.java:126) [spring-boot-actuator-2.0.3.RELEASE.jar:2.0.3.RELEASE] at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.doFilterInternal(WebMvcMetricsFilter.java:111) [spring-boot-actuator-2.0.3.RELEASE.jar:2.0.3.RELEASE] at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-5.0.7.RELEASE.jar:5.0.7.RELEASE] at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) [jetty-servlet-9.4.11.v20180605.jar:9.4.11.v20180605] at com.alibaba.druid.support.http.WebStatFilter.doFilter(WebStatFilter.java:124) [druid-1.1.20.jar:1.1.20] at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) [jetty-servlet-9.4.11.v20180605.jar:9.4.11.v20180605] at org.springframework.boot.actuate.web.trace.servlet.HttpTraceFilter.doFilterInternal(HttpTraceFilter.java:90) [spring-boot-actuator-2.0.3.RELEASE.jar:2.0.3.RELEASE] at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-5.0.7.RELEASE.jar:5.0.7.RELEASE] at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) [jetty-servlet-9.4.11.v20180605.jar:9.4.11.v20180605] at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:200) [spring-web-5.0.7.RELEASE.jar:5.0.7.RELEASE] at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-5.0.7.RELEASE.jar:5.0.7.RELEASE] at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) [jetty-servlet-9.4.11.v20180605.jar:9.4.11.v20180605] at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533) [jetty-servlet-9.4.11.v20180605.jar:9.4.11.v20180605] at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146) [jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605] at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548) [jetty-security-9.4.11.v20180605.jar:9.4.11.v20180605] at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) [jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605] at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257) [jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605] at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595) [jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605] at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) [jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605] at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1317) [jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605] at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203) [jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605] at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473) [jetty-servlet-9.4.11.v20180605.jar:9.4.11.v20180605] at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564) [jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605] at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201) [jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605] at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1219) [jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605] at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144) [jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605] at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) [jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605] at org.eclipse.jetty.server.Server.handle(Server.java:531) [jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605] at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:352) [jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605] at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260) [jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605] at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:281) [jetty-io-9.4.11.v20180605.jar:9.4.11.v20180605] at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102) [jetty-io-9.4.11.v20180605.jar:9.4.11.v20180605] at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118) [jetty-io-9.4.11.v20180605.jar:9.4.11.v20180605] at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762) [jetty-util-9.4.11.v20180605.jar:9.4.11.v20180605] at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680) [jetty-util-9.4.11.v20180605.jar:9.4.11.v20180605] at java.lang.Thread.run(Thread.java:748) [?:1.8.0_281]
qjws
我部署的0.9.1版本 这里可视化不能点击 你的是可以点的吗?
执行SparkSQL得到结果后，点击“可视化”按钮，报500错误：
找到dss/visualis-server/logs 下的日志：
2021-04-26 09:09:22.426 WARN [qtp895684328-1000] org.eclipse.jetty.server.HttpChannel 573 handleException - /api/rest_j/v1/visualis/view java.lang.NullPointerException: null at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:386) ~[jaxrs-ri-2.21.jar:2.21.] at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:334) ~[jaxrs-ri-2.21.jar:2.21.] at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:221) ~[jaxrs-ri-2.21.jar:2.21.] at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:865) ~[jetty-servlet-9.4.11.v20180605.jar:9.4.11.v20180605] at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1655) ~[jetty-servlet-9.4.11.v20180605.jar:9.4.11.v20180605] at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:215) ~[websocket-server-9.4.11.v20180605.jar:9.4.11.v20180605] at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) ~[jetty-servlet-9.4.11.v20180605.jar:9.4.11.v20180605] at com.webank.wedatasphere.linkis.server.security.SecurityFilter.doFilter(SecurityFilter.scala:100) ~[linkis-module-0.9.3.jar:?] at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) ~[jetty-servlet-9.4.11.v20180605.jar:9.4.11.v20180605] at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:200) ~[spring-web-5.0.7.RELEASE.jar:5.0.7.RELEASE] at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) ~[spring-web-5.0.7.RELEASE.jar:5.0.7.RELEASE] at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) ~[jetty-servlet-9.4.11.v20180605.jar:9.4.11.v20180605] at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.filterAndRecordMetrics(WebMvcMetricsFilter.java:158) ~[spring-boot-actuator-2.0.3.RELEASE.jar:2.0.3.RELEASE] at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.filterAndRecordMetrics(WebMvcMetricsFilter.java:126) ~[spring-boot-actuator-2.0.3.RELEASE.jar:2.0.3.RELEASE] at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.doFilterInternal(WebMvcMetricsFilter.java:111) ~[spring-boot-actuator-2.0.3.RELEASE.jar:2.0.3.RELEASE] at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) ~[spring-web-5.0.7.RELEASE.jar:5.0.7.RELEASE] at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) ~[jetty-servlet-9.4.11.v20180605.jar:9.4.11.v20180605] at com.alibaba.druid.support.http.WebStatFilter.doFilter(WebStatFilter.java:124) ~[druid-1.1.20.jar:1.1.20] at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) ~[jetty-servlet-9.4.11.v20180605.jar:9.4.11.v20180605] at org.springframework.boot.actuate.web.trace.servlet.HttpTraceFilter.doFilterInternal(HttpTraceFilter.java:90) ~[spring-boot-actuator-2.0.3.RELEASE.jar:2.0.3.RELEASE] at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) ~[spring-web-5.0.7.RELEASE.jar:5.0.7.RELEASE] at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) ~[jetty-servlet-9.4.11.v20180605.jar:9.4.11.v20180605] at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:200) ~[spring-web-5.0.7.RELEASE.jar:5.0.7.RELEASE] at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) ~[spring-web-5.0.7.RELEASE.jar:5.0.7.RELEASE] at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) ~[jetty-servlet-9.4.11.v20180605.jar:9.4.11.v20180605] at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533) ~[jetty-servlet-9.4.11.v20180605.jar:9.4.11.v20180605] at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146) ~[jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605] at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548) ~[jetty-security-9.4.11.v20180605.jar:9.4.11.v20180605] at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) ~[jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605] at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257) ~[jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605] at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595) ~[jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605] at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) ~[jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605] at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1317) ~[jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605] at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203) ~[jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605] at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473) ~[jetty-servlet-9.4.11.v20180605.jar:9.4.11.v20180605] at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564) ~[jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605] at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201) ~[jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605] at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1219) ~[jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605] at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144) ~[jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605] at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) ~[jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605] at org.eclipse.jetty.server.Server.handle(Server.java:531) ~[jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605] at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:352) [jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605] at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260) [jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605] at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:281) [jetty-io-9.4.11.v20180605.jar:9.4.11.v20180605] at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102) [jetty-io-9.4.11.v20180605.jar:9.4.11.v20180605] at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118) [jetty-io-9.4.11.v20180605.jar:9.4.11.v20180605] at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762) [jetty-util-9.4.11.v20180605.jar:9.4.11.v20180605] at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680) [jetty-util-9.4.11.v20180605.jar:9.4.11.v20180605] at java.lang.Thread.run(Thread.java:748) [?:1.8.0_281]要怎么进一步调试并发现错误的原因？