Qualitis is a one-stop data quality management platform that supports quality verification, notification, and management for various datasource. It is used to solve various data quality problems caused by data processing. https://github.com/WeBankFinTech/Qualitis
1.执行appconn-install.sh时。日志中的异常信息如下:
Step3: load the plugin of qualitis AppConn in DSS.
Now try to call dss-framework-project to reload the plugin of qualitis AppConn.
{"timestamp":1662013124121,"status":500,"error":"Internal Server Error","message":"","path":"/api/rest_j/v1/dss/framework/project/appconn/qualitis/load"}
Succeed to + reload the plugin of qualitis AppConn in dss-framework-project.
Now please wait for 5 minutes, then all of the DSS micro-services will refresh the qualitis AppConn plugin.
2.然后重启DSS,DSS-FRAMEWORK-PROJECT-SERVER服务启动失败。
部分日志信息如下:
2022-09-01 14:58:31.557 INFO [main] com.webank.wedatasphere.dss.standard.app.development.listener.scheduler.ListenerEventBusRefExecutionScheduler$$anon$1 41 info - Async-NodeExecution-Scheduler-ListenerBus add a new listener => class com.webank.wedatasphere.dss.standard.app.development.listener.scheduler.ListenerEventBusRefExecutionScheduler$$anon$3
2022-09-01 14:58:31.558 INFO [main] com.webank.wedatasphere.dss.framework.appconn.restful.AppConnManagerRestfulApi 59 lambda$init$1 - Try to check the quality of AppConn schedulis.
2022-09-01 14:58:31.569 INFO [main] com.webank.wedatasphere.dss.framework.appconn.restful.AppConnManagerRestfulApi 59 lambda$init$1 - Try to check the quality of AppConn qualitis.
2022-09-01 14:58:31.625 WARN [main] org.springframework.boot.web.servlet.context.AnnotationConfigServletWebServerApplicationContext 559 refresh - Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'appConnManagerRestfulApi': Invocation of init method failed; nested exception is java.lang.NoSuchMethodError: scala.Predef$.refArrayOps([Ljava/lang/Object;)Lscala/collection/mutable/ArrayOps;
2022-09-01 14:58:31.627 INFO [main] com.alibaba.druid.pool.DruidDataSource 2029 close - {dataSource-1} closing ...
2022-09-01 14:58:31.632 INFO [main] com.alibaba.druid.pool.DruidDataSource 2101 close - {dataSource-1} closed
2022-09-01 14:58:31.654 INFO [main] com.netflix.discovery.DiscoveryClient 935 shutdown - Shutting down DiscoveryClient ...
2022-09-01 14:58:34.655 INFO [main] com.netflix.discovery.DiscoveryClient 971 unregister - Unregistering ...
2022-09-01 14:58:34.661 INFO [main] com.netflix.discovery.DiscoveryClient 973 unregister - DiscoveryClient_DSS-FRAMEWORK-PROJECT-SERVER/h218:dss-framework-project-server:9202 - deregister status: 404
2022-09-01 14:58:34.689 INFO [main] com.netflix.discovery.DiscoveryClient 960 shutdown - Completed shut down of DiscoveryClient
2022-09-01 14:58:34.691 INFO [main] org.eclipse.jetty.server.session 149 stopScavenging - node0 Stopped scavenging
2022-09-01 14:58:34.692 INFO [main] org.eclipse.jetty.server.handler.ContextHandler 1154 doStop - Stopped o.s.b.w.e.j.JettyEmbeddedWebAppContext@1002d1c8{application,/,[file:///tmp/jetty-docbase.9202.9111122959891544011/],STOPPED}
2022-09-01 14:58:34.729 INFO [main] org.springframework.boot.autoconfigure.logging.ConditionEvaluationReportLoggingListener 136 logMessage -
Error starting ApplicationContext. To display the conditions report re-run your application with 'debug' enabled.
2022-09-01 14:58:34.747 ERROR [main] org.springframework.boot.diagnostics.LoggingFailureAnalysisReporter 40 report -
Description:
An attempt was made to call a method that does not exist. The attempt was made from the following location:
com.webank.wedatasphere.dss.standard.app.development.listener.scheduler.ListenerEventBusRefExecutionScheduler.(ListenerEventBusRefExecutionScheduler.scala:48)
The following method did not exist:
scala.Predef$.refArrayOps([Ljava/lang/Object;)Lscala/collection/mutable/ArrayOps;
The method's class, scala.Predef$, is available from the following locations:
jar:file:/home/data/dss-linkis/dss/lib/dss-commons/scala-library-2.12.10.jar!/scala/Predef$.class
The class hierarchy was loaded from the following locations:
scala.Predef$: file:/home/data/dss-linkis/dss/lib/dss-commons/scala-library-2.12.10.jar
scala.LowPriorityImplicits: file:/home/data/dss-linkis/dss/lib/dss-commons/scala-library-2.12.10.jar
Action:
Correct the classpath of your application so that it contains a single, compatible version of scala.Predef$
版本信息 qualitis 0.9.2、 dss 1.1.0 、linkis 1.1.1 、scala 2.12.10
问题 将qualitis接入dss工作流,执行失败。请问一下改怎么解决呢?详细的日志内容如下。
1.执行appconn-install.sh时。日志中的异常信息如下: Step3: load the plugin of qualitis AppConn in DSS. Now try to call dss-framework-project to reload the plugin of qualitis AppConn. {"timestamp":1662013124121,"status":500,"error":"Internal Server Error","message":"","path":"/api/rest_j/v1/dss/framework/project/appconn/qualitis/load"} Succeed to + reload the plugin of qualitis AppConn in dss-framework-project. Now please wait for 5 minutes, then all of the DSS micro-services will refresh the qualitis AppConn plugin.
2.然后重启DSS,DSS-FRAMEWORK-PROJECT-SERVER服务启动失败。 部分日志信息如下: 2022-09-01 14:58:31.557 INFO [main] com.webank.wedatasphere.dss.standard.app.development.listener.scheduler.ListenerEventBusRefExecutionScheduler$$anon$1 41 info - Async-NodeExecution-Scheduler-ListenerBus add a new listener => class com.webank.wedatasphere.dss.standard.app.development.listener.scheduler.ListenerEventBusRefExecutionScheduler$$anon$3 2022-09-01 14:58:31.558 INFO [main] com.webank.wedatasphere.dss.framework.appconn.restful.AppConnManagerRestfulApi 59 lambda$init$1 - Try to check the quality of AppConn schedulis. 2022-09-01 14:58:31.569 INFO [main] com.webank.wedatasphere.dss.framework.appconn.restful.AppConnManagerRestfulApi 59 lambda$init$1 - Try to check the quality of AppConn qualitis. 2022-09-01 14:58:31.625 WARN [main] org.springframework.boot.web.servlet.context.AnnotationConfigServletWebServerApplicationContext 559 refresh - Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'appConnManagerRestfulApi': Invocation of init method failed; nested exception is java.lang.NoSuchMethodError: scala.Predef$.refArrayOps([Ljava/lang/Object;)Lscala/collection/mutable/ArrayOps; 2022-09-01 14:58:31.627 INFO [main] com.alibaba.druid.pool.DruidDataSource 2029 close - {dataSource-1} closing ... 2022-09-01 14:58:31.632 INFO [main] com.alibaba.druid.pool.DruidDataSource 2101 close - {dataSource-1} closed 2022-09-01 14:58:31.654 INFO [main] com.netflix.discovery.DiscoveryClient 935 shutdown - Shutting down DiscoveryClient ... 2022-09-01 14:58:34.655 INFO [main] com.netflix.discovery.DiscoveryClient 971 unregister - Unregistering ... 2022-09-01 14:58:34.661 INFO [main] com.netflix.discovery.DiscoveryClient 973 unregister - DiscoveryClient_DSS-FRAMEWORK-PROJECT-SERVER/h218:dss-framework-project-server:9202 - deregister status: 404 2022-09-01 14:58:34.689 INFO [main] com.netflix.discovery.DiscoveryClient 960 shutdown - Completed shut down of DiscoveryClient 2022-09-01 14:58:34.691 INFO [main] org.eclipse.jetty.server.session 149 stopScavenging - node0 Stopped scavenging 2022-09-01 14:58:34.692 INFO [main] org.eclipse.jetty.server.handler.ContextHandler 1154 doStop - Stopped o.s.b.w.e.j.JettyEmbeddedWebAppContext@1002d1c8{application,/,[file:///tmp/jetty-docbase.9202.9111122959891544011/],STOPPED} 2022-09-01 14:58:34.729 INFO [main] org.springframework.boot.autoconfigure.logging.ConditionEvaluationReportLoggingListener 136 logMessage -
Error starting ApplicationContext. To display the conditions report re-run your application with 'debug' enabled. 2022-09-01 14:58:34.747 ERROR [main] org.springframework.boot.diagnostics.LoggingFailureAnalysisReporter 40 report -
Description: An attempt was made to call a method that does not exist. The attempt was made from the following location: com.webank.wedatasphere.dss.standard.app.development.listener.scheduler.ListenerEventBusRefExecutionScheduler.(ListenerEventBusRefExecutionScheduler.scala:48)
The following method did not exist:
scala.Predef$.refArrayOps([Ljava/lang/Object;)Lscala/collection/mutable/ArrayOps;
The method's class, scala.Predef$, is available from the following locations:
jar:file:/home/data/dss-linkis/dss/lib/dss-commons/scala-library-2.12.10.jar!/scala/Predef$.class
The class hierarchy was loaded from the following locations:
scala.Predef$: file:/home/data/dss-linkis/dss/lib/dss-commons/scala-library-2.12.10.jar
scala.LowPriorityImplicits: file:/home/data/dss-linkis/dss/lib/dss-commons/scala-library-2.12.10.jar
Action:
Correct the classpath of your application so that it contains a single, compatible version of scala.Predef$
3.经过查询dss安装路径下的scala-library-*.jar。并不存在多个不同版本的问题。