search

cubefs / compass

Compass is a task diagnosis platform for bigdata
Apache License 2.0
362 stars 136 forks source link

compass deployment problem #19

Open chenwenjob opened 1 year ago

chenwenjob commented 1 year ago

1、Basic environment confirmation: Do kafka redis es components need to be configured for compass with dolpinscheduler? (single-player mode)

2、Port 7050 is not found after startup, and no error message is displayed: Console log information is as follows:

Log information when start_all.sh is executed:

./bin/start_all.sh
/opt/compass/dist/compass/task-application 32066 /opt/compass/dist/compass/task-canal conf/metrics/ conf/example/ conf/spring/ conf/spring/tsdb/ conf/spring/tsdb/sql-map/ conf/spring/tsdb/sql/ conf/logback.xml conf/metrics/Canal_instances_tmpl.json conf/example/instance.properties conf/canal.properties conf/canal_local.properties conf/spring/default-instance.xml conf/spring/tsdb/mysql-tsdb.xml conf/spring/tsdb/sql-map/sqlmap_snapshot.xml conf/spring/tsdb/sql-map/sqlmap-config.xml conf/spring/tsdb/sql-map/sqlmap_history.xml conf/spring/tsdb/h2-tsdb.xml conf/spring/tsdb/sql/create_table.sql conf/spring/base-instance.xml conf/spring/group-instance.xml conf/spring/memory-instance.xml conf/spring/file-instance.xml

gzip: stdin: unexpected end of file tar: Unexpected EOF in archive tar: Error is not recoverable: exiting now

gzip: stdin: unexpected end of file tar: Unexpected EOF in archive tar: Error is not recoverable: exiting now plugin/ plugin/connector.rocketmq-1.1.6-jar-with-dependencies.jar

gzip: stdin: unexpected end of file tar: Unexpected EOF in archive tar: Unexpected EOF in archive tar: Error is not recoverable: exiting now cd to /opt/compass/dist/compass/task-canal/bin for workaround relative path LOG CONFIGURATION : /opt/compass/dist/compass/task-canal/bin/../conf/logback.xml canal conf : /opt/compass/dist/compass/task-canal/bin/../conf/canal.properties CLASSPATH :/opt/compass/dist/compass/task-canal/bin/../conf:/opt/compass/dist/compass/task-canal/bin/../lib/*:.:/usr/local/java/lib/dt.jar:/usr/local/java/lib/tools.jar cd to /opt/compass/dist/compass/task-canal for continue /opt/compass/dist/compass/task-canal-adapter cd to /opt/compass/dist/compass/task-canal-adapter/bin for workaround relative path CLASSPATH :/opt/compass/dist/compass/task-canal-adapter/bin/../conf:/opt/compass/dist/compass/task-canal-adapter/bin/../lib/zookeeper-3.4.5.jar:/opt/compass/dist/compass/task-canal-adapter/bin/../lib/zkclient-0.10.jar:/opt/compass/dist/compass/task-canal-adapter/bin/../lib/validation-api-2.0.1.Final.jar:/opt/compass/dist/compass/task-canal-adapter/bin/../lib/ucp-11.2.0.4.jar:/opt/compass/dist/compass/task-canal-adapter/bin/../lib/tomcat-embed-websocket-8.5.29.jar:/opt/compass/dist/compass/task-canal-adapter/bin/../lib/tomcat-embed-el-8.5.29.jar:/opt/compass/dist/compass/task-canal-adapter/bin/../lib/tomcat-embed-core-8.5.29.jar:/opt/compass/dist/compass/task-canal-adapter/bin/../lib/spring-webmvc-5.0.5.RELEASE.jar:/opt/compass/dist/compass/task-canal-adapter/bin/../lib/spring-web-5.0.5.RELEASE.jar:/opt/compass/dist/compass/task-canal-adapter/bin/../lib/spring-tx-5.0.5.RELEASE.jar:/opt/compass/dist/compass/task-canal-adapter/bin/../lib/spring-security-crypto-5.0.4.RELEASE.jar:/opt/compass/dist/compass/task-canal-adapter/bin/../lib/spring-orm-5.0.5.RELEASE.jar:/opt/compass/dist/compass/task-canal-adapter/bin/../lib/spring-jdbc-5.0.5.RELEASE.jar:/opt/compass/dist/compass/task-canal-adapter/bin/../lib/spring-jcl-5.0.5.RELEASE.jar:/opt/compass/dist/compass/task-canal-adapter/bin/../lib/spring-expression-5.0.5.RELEASE.jar:/opt/compass/dist/compass/task-canal-adapter/bin/../lib/spring-core-5.0.5.RELEASE.jar:/opt/compass/dist/compass/task-canal-adapter/bin/../lib/spring-context-5.0.5.RELEASE.jar:/opt/compass/dist/compass/task-canal-adapter/bin/../lib/spring-cloud-context-2.0.0.RELEASE.jar:/opt/compass/dist/compass/task-canal-adapter/bin/../lib/spring-boot-starter-web-2.0.1.RELEASE.jar:/opt/compass/dist/compass/task-canal-adapter/bin/../lib/spring-boot-starter-tomcat-2.0.1.RELEASE.jar:/opt/compass/dist/compass/task-canal-adapter/bin/../lib/spring-boot-starter-logging-2.0.1.RELEASE.jar:/opt/compass/dist/compass/task-canal-adapter/bin/../lib/spring-boot-starter-json-2.0.1.RELEASE.jar:/opt/compass/dist/compass/task-canal-adapter/bin/../lib/spring-boot-starter-2.0.1.RELEASE.jar:/opt/compass/dist/compass/task-canal-adapter/bin/../lib/spring-boot-autoconfigure-2.0.1.RELEASE.jar:/opt/compass/dist/compass/task-canal-adapter/bin/../lib/spring-boot-2.0.1.RELEASE.jar:/opt/compass/dist/compass/task-canal-adapter/bin/../lib/spring-beans-5.0.5.RELEASE.jar:/opt/compass/dist/compass/task-canal-adapter/bin/../lib/spring-aop-5.0.5.RELEASE.jar:/opt/compass/dist/compass/task-canal-adapter/bin/../lib/snakeyaml-1.19.jar:/opt/compass/dist/compass/task-canal-adapter/bin/../lib/slf4j-api-1.7.12.jar:/opt/compass/dist/compass/task-canal-adapter/bin/../lib/simplefan-11.2.0.4.jar:/opt/compass/dist/compass/task-canal-adapter/bin/../lib/protobuf-java-3.6.1.jar:/opt/compass/dist/compass/task-canal-adapter/bin/../lib/postgresql-42.1.4.jar:/opt/compass/dist/compass/task-canal-adapter/bin/../lib/osdt_core-11.2.0.4.jar:/opt/compass/dist/compass/task-canal-adapter/bin/../lib/osdt_cert-11.2.0.4.jar:/opt/compass/dist/compass/task-canal-adapter/bin/../lib/oro-2.0.8.jar:/opt/compass/dist/compass/task-canal-adapter/bin/../lib/oraclepki-11.2.0.4.jar:/opt/compass/dist/compass/task-canal-adapter/bin/../lib/ons-11.2.0.4.jar:/opt/compass/dist/compass/task-canal-adapter/bin/../lib/ojdbc6-11.2.0.4.jar:/opt/compass/dist/compass/task-canal-adapter/bin/../lib/netty-all-4.1.6.Final.jar:/opt/compass/dist/compass/task-canal-adapter/bin/../lib/netty-3.2.2.Final.jar:/opt/compass/dist/compass/task-canal-adapter/bin/../lib/mysql-connector-java-5.1.48.jar:/opt/compass/dist/compass/task-canal-adapter/bin/../lib/mssql-jdbc-7.0.0.jre8.jar:/opt/compass/dist/compass/task-canal-adapter/bin/../lib/logback-core-1.1.3.jar:/opt/compass/dist/compass/task-canal-adapter/bin/../lib/logback-classic-1.1.3.jar:/opt/compass/dist/compass/task-canal-adapter/bin/../lib/log4j-to-slf4j-2.17.0.jar:/opt/compass/dist/compass/task-canal-adapter/bin/../lib/log4j-api-2.17.0.jar:/opt/compass/dist/compass/task-canal-adapter/bin/../lib/log4j-1.2.17.jar:/opt/compass/dist/compass/task-canal-adapter/bin/../lib/jul-to-slf4j-1.7.25.jar:/opt/compass/dist/compass/task-canal-adapter/bin/../lib/jsr305-3.0.2.jar:/opt/compass/dist/compass/task-canal-adapter/bin/../lib/joda-time-2.9.4.jar:/opt/compass/dist/compass/task-canal-adapter/bin/../lib/jcl-over-slf4j-1.7.12.jar:/opt/compass/dist/compass/task-canal-adapter/bin/../lib/jboss-logging-3.3.2.Final.jar:/opt/compass/dist/compass/task-canal-adapter/bin/../lib/javax.annotation-api-1.3.2.jar:/opt/compass/dist/compass/task-canal-adapter/bin/../lib/jackson-module-parameter-names-2.9.5.jar:/opt/compass/dist/compass/task-canal-adapter/bin/../lib/jackson-datatype-jsr310-2.9.5.jar:/opt/compass/dist/compass/task-canal-adapter/bin/../lib/jackson-datatype-jdk8-2.9.5.jar:/opt/compass/dist/compass/task-canal-adapter/bin/../lib/jackson-databind-2.9.5.jar:/opt/compass/dist/compass/task-canal-adapter/bin/../lib/jackson-core-2.9.5.jar:/opt/compass/dist/compass/task-canal-adapter/bin/../lib/jackson-annotations-2.9.0.jar:/opt/compass/dist/compass/task-canal-adapter/bin/../lib/htrace-core-3.1.0-incubating.jar:/opt/compass/dist/compass/task-canal-adapter/bin/../lib/hibernate-validator-6.0.9.Final.jar:/opt/compass/dist/compass/task-canal-adapter/bin/../lib/hbase-shaded-client-1.1.2.jar:/opt/compass/dist/compass/task-canal-adapter/bin/../lib/guava-18.0.jar:/opt/compass/dist/compass/task-canal-adapter/bin/../lib/findbugs-annotations-1.3.9-1.jar:/opt/compass/dist/compass/task-canal-adapter/bin/../lib/fastjson2-2.0.7.jar:/opt/compass/dist/compass/task-canal-adapter/bin/../lib/druid-1.2.11.jar:/opt/compass/dist/compass/task-canal-adapter/bin/../lib/curator-recipes-2.10.0.jar:/opt/compass/dist/compass/task-canal-adapter/bin/../lib/curator-framework-2.10.0.jar:/opt/compass/dist/compass/task-canal-adapter/bin/../lib/curator-client-2.10.0.jar:/opt/compass/dist/compass/task-canal-adapter/bin/../lib/connector.core-1.1.6.jar:/opt/compass/dist/compass/task-canal-adapter/bin/../lib/commons-logging-1.2.jar:/opt/compass/dist/compass/task-canal-adapter/bin/../lib/commons-lang-2.6.jar:/opt/compass/dist/compass/task-canal-adapter/bin/../lib/commons-io-2.4.jar:/opt/compass/dist/compass/task-canal-adapter/bin/../lib/commons-codec-1.9.jar:/opt/compass/dist/compass/task-canal-adapter/bin/../lib/commons-beanutils-1.8.2.jar:/opt/compass/dist/compass/task-canal-adapter/bin/../lib/client-adapter.launcher-1.1.6.jar:/opt/compass/dist/compass/task-canal-adapter/bin/../lib/client-adapter.common-1.1.6.jar:/opt/compass/dist/compass/task-canal-adapter/bin/../lib/classmate-1.3.4.jar:/opt/compass/dist/compass/task-canal-adapter/bin/../lib/canal.protocol-1.1.6.jar:/opt/compass/dist/compass/task-canal-adapter/bin/../lib/canal.common-1.1.6.jar:/opt/compass/dist/compass/task-canal-adapter/bin/../lib/aviator-2.2.1.jar:.:/usr/local/java/lib/dt.jar:/usr/local/java/lib/tools.jar cd to /opt/compass/dist/compass/task-canal-adapter for continue /opt/compass/dist/compass/task-detect nohup: appending output to ohup.out 32222 /opt/compass/dist/compass/task-metadata 32251 /opt/compass/dist/compass/task-parser 32355 /opt/compass/dist/compass/task-portal 32413 /opt/compass/dist/compass/task-syncer 32489

Log information when stop_all.sh is executed:

./bin/stop_all.sh /opt/compass/dist/compass/task-application task-application 26076 /opt/compass/dist/compass/task-canal ambari-35.snowleopard.cn: stopping canal 26154 ... bin/stop.sh: line 68: kill: (26154) - No such process Oook! cost:0 /opt/compass/dist/compass/task-canal-adapter ambari-35.snowleopard.cn: stopping canal 26199 ... Oook! cost:6 /opt/compass/dist/compass/task-detect task-detect 26215 /opt/compass/dist/compass/task-metadata task-metadata 26267 /opt/compass/dist/compass/task-parser task-parser 26330 /opt/compass/dist/compass/task-portal task-portal 26403 /opt/compass/dist/compass/task-syncer task-syncer 26470

nilnon commented 1 year ago

@chenwenjob
Kafka redis es components are required.

gzip: stdin: unexpected end of file tar: Unexpected EOF in archive tar: Error is not recoverable: exiting now

It seems that the canal download failed, please download again and put it in the task-canal directory.

Port 7050 is not found after startup

Please check whether the task-portal process is alive.

chenwenjob commented 1 year ago

@nilnon With the es zk redis kafka component configured as recommended, the UI interface is already visible. But the platform has no data to show. Some task-portal.error.logs are intercepted, please help to troubleshoot errors:

2023-04-12 16:41:38,084 ERROR 26217 [http-nio-7075-exec-9] [] : [com.oppo.cloud.portal.config.GlobalExceptionConfig:41] GlobalException:
java.lang.NullPointerException: null at com.oppo.cloud.portal.service.impl.ReportServiceImpl.getStatisticsData(ReportServiceImpl.java:133) at com.oppo.cloud.portal.controller.ReportController.getStatisticData(ReportController.java:49) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205) at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:150) at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:117) at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:895) at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:808) at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87) at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1071) at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:964) at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006) at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:898) at javax.servlet.http.HttpServlet.service(HttpServlet.java:670) at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883) at javax.servlet.http.HttpServlet.service(HttpServlet.java:779) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:227) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.springframework.web.filter.CorsFilter.doFilterInternal(CorsFilter.java:91) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:214) at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:186) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:354) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:267) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.doFilterInternal(WebMvcMetricsFilter.java:96) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:177) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:97) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:541) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:135) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:78) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:360) at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:399) at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65) at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:891) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1784) at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191) at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:750) 2023-04-12 16:41:42,633 ERROR 26217 [http-nio-7075-exec-4] [] : [com.oppo.cloud.portal.config.GlobalExceptionConfig:41] GlobalException:
java.lang.NullPointerException: null at com.oppo.cloud.portal.service.impl.ElasticSearchServiceImpl.findValueByDayBuckets(ElasticSearchServiceImpl.java:106) at com.oppo.cloud.portal.service.impl.ElasticSearchServiceImpl.sumAggregationByDay(ElasticSearchServiceImpl.java:300) at com.oppo.cloud.portal.service.impl.TaskAppServiceImpl.getGraph(TaskAppServiceImpl.java:421) at com.oppo.cloud.portal.controller.AppController.getGraph(AppController.java:121) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205) at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:150) at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:117) at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:895) at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:808) at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87) at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1071) at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:964) at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006) at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:909) at javax.servlet.http.HttpServlet.service(HttpServlet.java:696) at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883) at javax.servlet.http.HttpServlet.service(HttpServlet.java:779) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:227) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.springframework.web.filter.CorsFilter.doFilterInternal(CorsFilter.java:91) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:214) at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:186) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:354) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:267) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.doFilterInternal(WebMvcMetricsFilter.java:96) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:177) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:97) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:541) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:135) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:78) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:360) at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:399) at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65) at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:891) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1784) at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191) at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:750)

Look forward to yours. Thank you！

nilnon commented 1 year ago

@chenwenjob The platform does not provide default data, the next thing you need to do is to synchronously schedule platform metadata for diagnosis.

chenwenjob commented 1 year ago

@nilnon I configured the dolphinschecler dispatch platform and I saw the dolphinschecler data synchronized in the compass database. image image

Please help to analyze the reason and solve the wrong idea, thank you

nilnon commented 1 year ago

@chenwenjob You need to make sure that each module is already working. It seems that you have successfully run task-canal/adapter, then you need to make sure that the task-syncer, task-application and other modules are running normally.

chenwenjob commented 1 year ago

@nilnon Ok, I'll try again! In addition, I did not configure the hadoop component information will not affect the platform work?

nilnon commented 1 year ago

@chenwenjob Hadoop components are necessary because the diagnostic platform is based on logs for diagnostic analysis.

chenwenjob commented 1 year ago

@nilnon Found some errors, please help to have a look:

I used the tool to try to connect redis. At the same time, I changed the version from 7.0 to 4.0, but the following error was displayed.

  1. task-application.error. , Caused by: org.springframework.data.redis.connection.PoolException: Could not get a resource from the pool; nested exception is io.lettuce.core.RedisException: Cannot obtain initial Redis Cluster topology at org.springframework.data.redis.connection.lettuce.LettucePoolingConnectionProvider.getConnection(LettucePoolingConnectionProvider.java:109) at org.springframework.data.redis.connection.lettuce.LettuceConnectionFactory$ExceptionTranslatingConnectionProvider.getConnection(LettuceConnectionFactory.java:1595) ... 23 common frames omitted Caused by: io.lettuce.core.RedisException: Cannot obtain initial Redis Cluster topology at io.lettuce.core.cluster.RedisClusterClient.lambda$getPartitions$0(RedisClusterClient.java:329) at io.lettuce.core.cluster.RedisClusterClient.get(RedisClusterClient.java:941) at io.lettuce.core.cluster.RedisClusterClient.getPartitions(RedisClusterClient.java:329) at org.springframework.data.redis.connection.lettuce.ClusterConnectionProvider.getConnectionAsync(ClusterConnectionProvider.java:92) at org.springframework.data.redis.connection.lettuce.ClusterConnectionProvider.getConnectionAsync(ClusterConnectionProvider.java:40) at org.springframework.data.redis.connection.lettuce.LettuceConnectionProvider.getConnection(LettuceConnectionProvider.java:53) at org.springframework.data.redis.connection.lettuce.LettucePoolingConnectionProvider.lambda$null$0(LettucePoolingConnectionProvider.java:97) at io.lettuce.core.support.ConnectionPoolSupport$RedisPooledObjectFactory.create(ConnectionPoolSupport.java:211) at io.lettuce.core.support.ConnectionPoolSupport$RedisPooledObjectFactory.create(ConnectionPoolSupport.java:201) at org.apache.commons.pool2.BasePooledObjectFactory.makeObject(BasePooledObjectFactory.java:70) at org.apache.commons.pool2.impl.GenericObjectPool.create(GenericObjectPool.java:571) at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:298) at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:223) at io.lettuce.core.support.ConnectionPoolSupport$1.borrowObject(ConnectionPoolSupport.java:122) at io.lettuce.core.support.ConnectionPoolSupport$1.borrowObject(ConnectionPoolSupport.java:117) at org.springframework.data.redis.connection.lettuce.LettucePoolingConnectionProvider.getConnection(LettucePoolingConnectionProvider.java:103) ... 24 common frames omitted Caused by: io.lettuce.core.cluster.topology.DefaultClusterTopologyRefresh$CannotRetrieveClusterPartitions: Cannot retrieve cluster partitions from [redis://192.168.5.88]

Details: [redis://192.168.5.88]: Unable to connect to [192.168.5.88:6379]: Connection closed prematurely

    Suppressed: io.lettuce.core.RedisConnectionException: Unable to connect to [192.168.5.88:6379]: Connection closed prematurely
            at io.lettuce.core.cluster.topology.DefaultClusterTopologyRefresh.lambda$openConnections$12(DefaultClusterTopologyRefresh.java:347)
            at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774)
            at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750)
            at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)
            at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1990)
            at io.lettuce.core.AbstractRedisClient.lambda$null$5(AbstractRedisClient.java:458)
            at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774)
            at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750)
            at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)
            at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1990)
            at io.lettuce.core.protocol.RedisHandshakeHandler.lambda$fail$4(RedisHandshakeHandler.java:131)
            at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590)
            at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557)
            at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492)
            at io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:185)
            at io.netty.channel.DefaultChannelPromise.addListener(DefaultChannelPromise.java:95)
            at io.netty.channel.DefaultChannelPromise.addListener(DefaultChannelPromise.java:30)
            at io.lettuce.core.protocol.RedisHandshakeHandler.fail(RedisHandshakeHandler.java:130)
            at io.lettuce.core.protocol.RedisHandshakeHandler.channelInactive(RedisHandshakeHandler.java:86)
            at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:305)
            at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:281)
            at io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:274)
            at io.netty.channel.ChannelInboundHandlerAdapter.channelInactive(ChannelInboundHandlerAdapter.java:81)
            at io.lettuce.core.ChannelGroupListener.channelInactive(ChannelGroupListener.java:69)
            at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:305)
            at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:281)
            at io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:274)
            at io.netty.channel.DefaultChannelPipeline$HeadContext.channelInactive(DefaultChannelPipeline.java:1405)
            at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:301)
            at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:281)
            at io.netty.channel.DefaultChannelPipeline.fireChannelInactive(DefaultChannelPipeline.java:901)
            at io.netty.channel.AbstractChannel$AbstractUnsafe$7.run(AbstractChannel.java:813)
            at io.netty.util.concurrent.AbstractEventExecutor.runTask(AbstractEventExecutor.java:174)
            at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:167)
            at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasksFrom(SingleThreadEventExecutor.java:426)
            at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:375)
            at io.netty.util.concurrent.SingleThreadEventExecutor.confirmShutdown(SingleThreadEventExecutor.java:763)
            at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:596)
            at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
            at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
            at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
            at java.lang.Thread.run(Thread.java:750)
    Caused by: io.lettuce.core.RedisConnectionException: Connection closed prematurely
            ... 24 common frames omitted

2023-04-13 09:26:55,372 ERROR 3574 [main] [] : [i.n.u.concurrent.DefaultPromise.rejectedExecution:864] Failed to submit a listener notification task. Event loop shut down? java.util.concurrent.RejectedExecutionException: event executor terminated at io.netty.util.concurrent.SingleThreadEventExecutor.reject(SingleThreadEventExecutor.java:934) at io.netty.util.concurrent.SingleThreadEventExecutor.offerTask(SingleThreadEventExecutor.java:351) at io.netty.util.concurrent.SingleThreadEventExecutor.addTask(SingleThreadEventExecutor.java:344)

2.task-detect.error.2023-04-13.0.log Caused by: org.springframework.data.redis.connection.PoolException: Could not get a resource from the pool; nested exception is io.lettuce.core.RedisException: Cannot obtain initial Redis Cluster topology at org.springframework.data.redis.connection.lettuce.LettucePoolingConnectionProvider.getConnection(LettucePoolingConnectionProvider.java:109) at org.springframework.data.redis.connection.lettuce.LettuceConnectionFactory$ExceptionTranslatingConnectionProvider.getConnection(LettuceConnectionFactory.java:1595) ... 23 common frames omitted Caused by: io.lettuce.core.RedisException: Cannot obtain initial Redis Cluster topology at io.lettuce.core.cluster.RedisClusterClient.lambda$getPartitions$0(RedisClusterClient.java:329) at io.lettuce.core.cluster.RedisClusterClient.get(RedisClusterClient.java:941) at io.lettuce.core.cluster.RedisClusterClient.getPartitions(RedisClusterClient.java:329) at org.springframework.data.redis.connection.lettuce.ClusterConnectionProvider.getConnectionAsync(ClusterConnectionProvider.java:92) at org.springframework.data.redis.connection.lettuce.ClusterConnectionProvider.getConnectionAsync(ClusterConnectionProvider.java:40) at org.springframework.data.redis.connection.lettuce.LettuceConnectionProvider.getConnection(LettuceConnectionProvider.java:53) at org.springframework.data.redis.connection.lettuce.LettucePoolingConnectionProvider.lambda$null$0(LettucePoolingConnectionProvider.java:97) at io.lettuce.core.support.ConnectionPoolSupport$RedisPooledObjectFactory.create(ConnectionPoolSupport.java:211) at io.lettuce.core.support.ConnectionPoolSupport$RedisPooledObjectFactory.create(ConnectionPoolSupport.java:201) at org.apache.commons.pool2.BasePooledObjectFactory.makeObject(BasePooledObjectFactory.java:70) at org.apache.commons.pool2.impl.GenericObjectPool.create(GenericObjectPool.java:571) at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:298) at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:223) at io.lettuce.core.support.ConnectionPoolSupport$1.borrowObject(ConnectionPoolSupport.java:122) at io.lettuce.core.support.ConnectionPoolSupport$1.borrowObject(ConnectionPoolSupport.java:117) at org.springframework.data.redis.connection.lettuce.LettucePoolingConnectionProvider.getConnection(LettucePoolingConnectionProvider.java:103) ... 24 common frames omitted Caused by: io.lettuce.core.cluster.topology.DefaultClusterTopologyRefresh$CannotRetrieveClusterPartitions: Cannot retrieve cluster partitions from [redis://192.168.5.88]

Details: [redis://192.168.5.88]: Unable to connect to [192.168.5.88:6379]: executor not accepting a task

    Suppressed: io.lettuce.core.RedisConnectionException: Unable to connect to [192.168.5.88:6379]: executor not accepting a task
            at io.lettuce.core.cluster.topology.DefaultClusterTopologyRefresh.lambda$openConnections$12(DefaultClusterTopologyRefresh.java:347)
            at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774)
            at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750)
            at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)
            at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1990)

Looking forward to your reply！

nilnon commented 1 year ago

@chenwenjob Redis is required for cluster mode.

chenwenjob commented 1 year ago

@nilnon Thank you for your advice. The Redis-related problems have been eliminated. Several questions remain: 1.task-metadata error indicates a zk problem. What is the version requirement? I use the highest version of zk. 2023-04-13 10:38:30,002 ERROR 27269 [scheduling-5] [] : [o.a.curator.framework.imps.CuratorFrameworkImpl:703] Ensure path threw exception org.apache.zookeeper.KeeperException$UnimplementedException: KeeperErrorCode = Unimplemented for /task-metadata at org.apache.zookeeper.KeeperException.create(KeeperException.java:106) at org.apache.zookeeper.KeeperException.create(KeeperException.java:54) at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:1403) at org.apache.curator.utils.ZKPaths.mkdirs(ZKPaths.java:308) at org.apache.curator.framework.imps.NamespaceImpl$1.call(NamespaceImpl.java:90) at org.apache.curator.connection.StandardConnectionHandlingPolicy.callWithRetry(StandardConnectionHandlingPolicy.java:64) at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:100) at org.apache.curator.framework.imps.NamespaceImpl.fixForNamespace(NamespaceImpl.java:83) at org.apache.curator.framework.imps.CuratorFrameworkImpl.fixForNamespace(CuratorFrameworkImpl.java:736) at org.apache.curator.framework.imps.WatcherRemovalFacade.fixForNamespace(WatcherRemovalFacade.java:176) at org.apache.curator.framework.imps.CreateBuilderImpl.forPath(CreateBuilderImpl.java:586) at org.apache.curator.framework.imps.CreateBuilderImpl.forPath(CreateBuilderImpl.java:575) at org.apache.curator.framework.imps.CreateBuilderImpl.forPath(CreateBuilderImpl.java:51) at org.apache.curator.framework.recipes.locks.StandardLockInternalsDriver.createsTheLock(StandardLockInternalsDriver.java:54) at org.apache.curator.framework.recipes.locks.LockInternals.attemptLock(LockInternals.java:225) at org.apache.curator.framework.recipes.locks.InterProcessMutex.internalLock(InterProcessMutex.java:237) at org.apache.curator.framework.recipes.locks.InterProcessMutex.acquire(InterProcessMutex.java:108) at com.oppo.cloud.meta.scheduler.SparkMetaScheduler.lock(SparkMetaScheduler.java:54) at com.oppo.cloud.meta.scheduler.SparkMetaScheduler.run(SparkMetaScheduler.java:45) at sun.reflect.GeneratedMethodAccessor14.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.springframework.scheduling.support.ScheduledMethodRunnable.run(ScheduledMethodRunnable.java:84) at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54) at org.springframework.scheduling.concurrent.ReschedulingRunnable.run(ReschedulingRunnable.java:95) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-04-13 10:38:30,002 ERROR 27269 [scheduling-9] [] : [o.a.curator.framework.imps.CuratorFrameworkImpl:703] Ensure path threw exception org.apache.zookeeper.KeeperException$UnimplementedException: KeeperErrorCode = Unimplemented for /task-metadata at org.apache.zookeeper.KeeperException.create(KeeperException.java:106) at org.apache.zookeeper.KeeperException.create(KeeperException.java:54) at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:1403) at org.apache.curator.utils.ZKPaths.mkdirs(ZKPaths.java:308) at org.apache.curator.framework.imps.NamespaceImpl$1.call(NamespaceImpl.java:90) at org.apache.curator.connection.StandardConnectionHandlingPolicy.callWithRetry(StandardConnectionHandlingPolicy.java:64) at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:100) at org.apache.curator.framework.imps.NamespaceImpl.fixForNamespace(NamespaceImpl.java:83) at org.apache.curator.framework.imps.CuratorFrameworkImpl.fixForNamespace(CuratorFrameworkImpl.java:736) at org.apache.curator.framework.imps.WatcherRemovalFacade.fixForNamespace(WatcherRemovalFacade.java:176) at org.apache.curator.framework.imps.CreateBuilderImpl.forPath(CreateBuilderImpl.java:586) at org.apache.curator.framework.imps.CreateBuilderImpl.forPath(CreateBuilderImpl.java:575) at org.apache.curator.framework.imps.CreateBuilderImpl.forPath(CreateBuilderImpl.java:51) at org.apache.curator.framework.recipes.locks.StandardLockInternalsDriver.createsTheLock(StandardLockInternalsDriver.java:54) at org.apache.curator.framework.recipes.locks.LockInternals.attemptLock(LockInternals.java:225) at org.apache.curator.framework.recipes.locks.InterProcessMutex.internalLock(InterProcessMutex.java:237) at org.apache.curator.framework.recipes.locks.InterProcessMutex.acquire(InterProcessMutex.java:108) at com.oppo.cloud.meta.scheduler.YarnMetaScheduler.lock(YarnMetaScheduler.java:56) at com.oppo.cloud.meta.scheduler.YarnMetaScheduler.run(YarnMetaScheduler.java:46) at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.springframework.scheduling.support.ScheduledMethodRunnable.run(ScheduledMethodRunnable.java:84) at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54) at org.springframework.scheduling.concurrent.ReschedulingRunnable.run(ReschedulingRunnable.java:95) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-04-13 10:38:35,105 ERROR 27269 [scheduling-5] [] : [com.oppo.cloud.meta.scheduler.SparkMetaScheduler:47] KeeperErrorCode = Unimplemented for /task-metadata/sparkMetaLock/_c_50d6b16d-9e1f-47cd-8297-531345420801-lock- 2023-04-13 10:38:35,105 ERROR 27269 [scheduling-9] [] : [com.oppo.cloud.meta.scheduler.YarnMetaScheduler:48] Exception: org.apache.zookeeper.KeeperException$UnimplementedException: KeeperErrorCode = Unimplemented for /task-metadata/yarnMetaLock/_c_af4251ce-a221-4182-aaf8-151186ff28cc-lock- at org.apache.zookeeper.KeeperException.create(KeeperException.java:106) at org.apache.zookeeper.KeeperException.create(KeeperException.java:54) at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:1506) at org.apache.curator.framework.imps.CreateBuilderImpl$17.call(CreateBuilderImpl.java:1180) at org.apache.curator.framework.imps.CreateBuilderImpl$17.call(CreateBuilderImpl.java:1156) at org.apache.curator.connection.StandardConnectionHandlingPolicy.callWithRetry(StandardConnectionHandlingPolicy.java:64) at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:100) at org.apache.curator.framework.imps.CreateBuilderImpl.pathInForeground(CreateBuilderImpl.java:1153) at org.apache.curator.framework.imps.CreateBuilderImpl.protectedPathInForeground(CreateBuilderImpl.java:607) at org.apache.curator.framework.imps.CreateBuilderImpl.forPath(CreateBuilderImpl.java:597) at org.apache.curator.framework.imps.CreateBuilderImpl.forPath(CreateBuilderImpl.java:575) at org.apache.curator.framework.imps.CreateBuilderImpl.forPath(CreateBuilderImpl.java:51) at org.apache.curator.framework.recipes.locks.StandardLockInternalsDriver.createsTheLock(StandardLockInternalsDriver.java:54) at org.apache.curator.framework.recipes.locks.LockInternals.attemptLock(LockInternals.java:225) at org.apache.curator.framework.recipes.locks.InterProcessMutex.internalLock(InterProcessMutex.java:237) at org.apache.curator.framework.recipes.locks.InterProcessMutex.acquire(InterProcessMutex.java:108) at com.oppo.cloud.meta.scheduler.YarnMetaScheduler.lock(YarnMetaScheduler.java:56) at com.oppo.cloud.meta.scheduler.YarnMetaScheduler.run(YarnMetaScheduler.java:46) at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source)

2.task-portal error 2023-04-13 10:28:02,632 ERROR 27410 [http-nio-7075-exec-2] [] : [c.oppo.cloud.portal.service.impl.ReportServiceImpl:130] get last week period failed,msg: java.util.concurrent.ExecutionException: java.lang.NullPointerException at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357) at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1908) at com.oppo.cloud.portal.service.impl.ReportServiceImpl.getStatisticsData(ReportServiceImpl.java:128) at com.oppo.cloud.portal.controller.ReportController.getStatisticData(ReportController.java:49) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205) at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:150) at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:117) at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:895) at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:808) at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87) at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1071) at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:964) at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006) at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:898) at javax.servlet.http.HttpServlet.service(HttpServlet.java:670) at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883) at javax.servlet.http.HttpServlet.service(HttpServlet.java:779) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:227) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.springframework.web.filter.CorsFilter.doFilterInternal(CorsFilter.java:91) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:214) at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:186) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:354) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:267) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.doFilterInternal(WebMvcMetricsFilter.java:96) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:177) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:97) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:541) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:135) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:78) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:360) at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:399) at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65) at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:891) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1784) at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191) at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:750) Caused by: java.lang.NullPointerException: null at com.oppo.cloud.portal.service.impl.ReportServiceImpl.getStaticsDataByReportData(ReportServiceImpl.java:252) at com.oppo.cloud.portal.service.impl.ReportServiceImpl.lambda$getStatisticsData$2(ReportServiceImpl.java:108) at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1604) at java.util.concurrent.CompletableFuture$AsyncSupply.exec(CompletableFuture.java:1596) at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289) at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1067) at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1703) at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:172) 2023-04-13 10:28:02,633 ERROR 27410 [http-nio-7075-exec-2] [] : [com.oppo.cloud.portal.config.GlobalExceptionConfig:41] GlobalException:
java.lang.NullPointerException: null at com.oppo.cloud.portal.service.impl.ReportServiceImpl.getStatisticsData(ReportServiceImpl.java:133) at com.oppo.cloud.portal.controller.ReportController.getStatisticData(ReportController.java:49) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498)

Looking forward to your reply, best wishes！

nilnon commented 1 year ago

@chenwenjob

pom.xml

4.2.0 3.7.1
chenwenjob commented 1 year ago

@nilnon The task-portal still reports an error, and no data can be found on the interface. It feels that the interface responds to 304 status. image

2.task-portal error ，Error message log

2023-04-13 13:10:25,011 ERROR 29412 [http-nio-7075-exec-4] [] : [com.oppo.cloud.portal.config.GlobalExceptionConfig:41] GlobalException:
java.lang.NullPointerException: null at com.oppo.cloud.portal.service.impl.ElasticSearchServiceImpl.findValueByDayBuckets(ElasticSearchServiceImpl.java:106) at com.oppo.cloud.portal.service.impl.ElasticSearchServiceImpl.sumAggregationByDay(ElasticSearchServiceImpl.java:300) at com.oppo.cloud.portal.service.impl.JobServiceImpl.getGraph(JobServiceImpl.java:291) at com.oppo.cloud.portal.controller.JobController.getGraph(JobController.java:118) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498)

2023-04-13 10:28:02,632 ERROR 27410 [http-nio-7075-exec-2] [] : [c.oppo.cloud.portal.service.impl.ReportServiceImpl:130] get last week period failed,msg: java.util.concurrent.ExecutionException: java.lang.NullPointerException at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357) at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1908) at com.oppo.cloud.portal.service.impl.ReportServiceImpl.getStatisticsData(ReportServiceImpl.java:128) at com.oppo.cloud.portal.controller.ReportController.getStatisticData(ReportController.java:49) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205) at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:150) at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:117) at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:895) at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:808) at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87) at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1071) at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:964) at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006) at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:898) at javax.servlet.http.HttpServlet.service(HttpServlet.java:670) at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883) at javax.servlet.http.HttpServlet.service(HttpServlet.java:779) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:227) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.springframework.web.filter.CorsFilter.doFilterInternal(CorsFilter.java:91) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:214) at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:186) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:354) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:267) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.doFilterInternal(WebMvcMetricsFilter.java:96) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:177) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:97) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:541) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:135) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:78) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:360) at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:399) at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65) at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:891) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1784) at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191) at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:750) Caused by: java.lang.NullPointerException: null at com.oppo.cloud.portal.service.impl.ReportServiceImpl.getStaticsDataByReportData(ReportServiceImpl.java:252) at com.oppo.cloud.portal.service.impl.ReportServiceImpl.lambda$getStatisticsData$2(ReportServiceImpl.java:108) at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1604) at java.util.concurrent.CompletableFuture$AsyncSupply.exec(CompletableFuture.java:1596) at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289) at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1067) at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1703) at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:172) 2023-04-13 10:28:02,633 ERROR 27410 [http-nio-7075-exec-2] [] : [com.oppo.cloud.portal.config.GlobalExceptionConfig:41] GlobalException: java.lang.NullPointerException: null at com.oppo.cloud.portal.service.impl.ReportServiceImpl.getStatisticsData(ReportServiceImpl.java:133) at com.oppo.cloud.portal.controller.ReportController.getStatisticData(ReportController.java:49) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498)

Looking forward to your reply, best wishes！

nilnon commented 1 year ago

@chenwenjob task-portal has no data, maybe there is no data in ES. You need to check if the modules task-applicaiton -> task-detect -> task-parser are working properly.

chenwenjob commented 1 year ago

@nilnon ES has the following indexes: image

kafka doesn't have any data, but canel doesn't report any errors

image

nilnon commented 1 year ago

@chenwenjob es index conf All indexes are configured under custom:elasticsearch. If canal is running normally, check whether there is new data in the source mysql.

chenwenjob commented 1 year ago

@nilnon A new problem appears in task-application.error， My hadoop configuration information is as follows

hadoop: namenodes:

  • nameservices: log-hdfs namenodesAddr: [ "192.168.5.31", "192.168.5.32", "192.168.5.33", "192.168.5.34", "192.168.5.35" ] namenodes: [ "192.168.5.31", "192.168.5.32" ] user: hdfs password: port: 8020

    scheduler platform hdfs log path keyword identification, used by task-application. flume

    matchPathKeys: [ "task-application" ]

yarn:

  • clusterName: "bigdata" resourceManager: [ "192.168.5.31:8088" ] jobHistoryServer: "192.168.5.32:19888"

spark: sparkHistoryServer: [ "localhost:18018" ]

application error info

2023-04-13 16:39:45,728 ERROR 23448 [org.springframework.kafka.KafkaListenerEndpointContainer#0-0-C-1] [] : [c.o.c.a.service.impl.LogParserServiceImpl:345] logPath: hdfs://log-hdfs:8020/flume/dolphinscheduler/2023-04-13/20230413/9183246776096_9-1630-13778_log does not have hadoop config 2023-04-13 16:39:45,738 ERROR 23448 [org.springframework.kafka.KafkaListenerEndpointContainer#0-0-C-1] [] : [c.o.c.a.service.impl.LogParserServiceImpl:345] logPath: hdfs://log-hdfs:8020/flume/dolphinscheduler/2023-04-13/20230413/9183246776096_9-1630-13778_log does not have hadoop config 2023-04-13 16:46:34,806 ERROR 23448 [org.springframework.kafka.KafkaListenerEndpointContainer#0-0-C-1] [] : [c.o.c.a.service.impl.LogParserServiceImpl:345] logPath: hdfs://log-hdfs:8020/flume/dolphinscheduler/2023-04-13/20230413/9208983174176_1-1631-13779_log does not have hadoop config 2023-04-13 16:46:36,648 ERROR 23448 [org.springframework.kafka.KafkaListenerEndpointContainer#0-0-C-1] [] : [c.o.c.a.service.impl.LogParserServiceImpl:345] logPath: hdfs://log-hdfs:8020/flume/dolphinscheduler/2023-04-13/20230413/9208983174176_1-1631-13779_log does not have hadoop config

At the same time, task-syncer also has an error message

2023-04-13 17:08:05,770 ERROR 23806 [org.springframework.kafka.KafkaListenerEndpointContainer#0-0-C-1] [] : [com.oppo.cloud.syncer.service.impl.CommonService:73] table: flow,queryForMap: select project.id as project_id, project.name as project_name from t_ds_process_definition as process inner join t_ds_project as project on process.project_code=project.code where process.id='11809', org.springframework.dao.EmptyResultDataAccessException: Incorrect result size: expected 1, actual 0 at org.springframework.dao.support.DataAccessUtils.nullableSingleResult(DataAccessUtils.java:97) at org.springframework.jdbc.core.JdbcTemplate.queryForObject(JdbcTemplate.java:509) at org.springframework.jdbc.core.JdbcTemplate.queryForMap(JdbcTemplate.java:502) at com.oppo.cloud.syncer.service.impl.CommonService.dataMapping(CommonService.java:70) at com.oppo.cloud.syncer.service.impl.FlowService.update(FlowService.java:68) at com.oppo.cloud.syncer.consumer.MessageConsumer.updateAction(MessageConsumer.java:116) at com.oppo.cloud.syncer.consumer.MessageConsumer.consumeMessage(MessageConsumer.java:93) at com.oppo.cloud.syncer.consumer.MessageConsumer.receive(MessageConsumer.java:78) at sun.reflect.GeneratedMethodAccessor28.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498)

nilnon commented 1 year ago

@chenwenjob

  1. your log path is hdfs://log-hdfs:8020/flume/dolphinscheduler/2023-04-13/20230413/9183246776096_9-1630-13778_log, so the matchPathKeys: [ "task-application" ] should be changed to matchPathKeys: [ "flume" ].

  2. You need to check whether process.id='11809' has been synchronized to mysql.

chenwenjob commented 1 year ago

@nilnon How to configure application-hadoop.yml when hadoop has permission to authenticate kerberos? The following error is seen in task-application:

2023-04-14 09:39:12,345 ERROR 12223 [delayed-queue-executor-1] [] : [c.o.c.a.service.impl.LogParserServiceImpl:362] filesPattern_error:java.io.IOException: Couldn't create proxy provider class org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider 2023-04-14 09:39:22,351 ERROR 12223 [delayed-queue-executor-3] [] : [c.o.c.a.service.impl.LogParserServiceImpl:362] filesPattern_error:java.io.IOException: Couldn't create proxy provider class org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider 2023-04-14 09:39:22,351 ERROR 12223 [delayed-queue-executor-3] [] : [com.oppo.cloud.application.task.DelayedTask:124] discard delay task:DelayedTaskInfo(key=0657186d-c42c-49c4-a305-ad6e8fc55ffc, tryTimes=21, taskInstance=TaskInstance [Hash = 95000617, id=13784, projectName=colin_test_414, flowName=com_test, taskName=compas_test_1, executionTime=null, startTime=null, endTime=null, taskState=17, taskType=SHELL, retryTimes=0, maxRetryTimes=0, workerGroup=default, triggerType=manual, createTime=null, updateTime=null, serialVersionUID=1], rawData={task_name=compas_test_1, retry_times=0, worker_group=default, task_state=17, flow_name=com_test, id=13784, task_type=SHELL, max_retry_times=0, project_name=colin_test_414})

nilnon commented 1 year ago

@chenwenjob At present, the hdfs client does not support configuring kerberos authentication. You can use the cluster test without authentication first, or modify the client configuration, or wait for our support. Thanks for your feedback.

nilnon commented 1 year ago

@chenwenjob We already support configuring kerberos, please check the latest version.