DTStack / Taier

Taier is a big data development platform for submission, scheduling, operation and maintenance, and indicator information display
https://dtstack.github.io/Taier/
Apache License 2.0
1.33k stars 334 forks source link

集群管理-配置yarn报错 #1112

Closed mfgcs closed 12 months ago

mfgcs commented 1 year ago

Search before asking

Description

通过docker-compose启动taier服务,配置yarn之后测试报错 image java.lang.reflect.UndeclaredThrowableException java.lang.reflect.UndeclaredThrowableException at com.sun.proxy.$Proxy213.testCon(Unknown Source) at com.dtstack.taier.scheduler.executor.DatasourceOperator.testConnect(DatasourceOperator.java:68) at com.dtstack.taier.develop.service.console.ConsoleComponentService.testConnect(ConsoleComponentService.java:1137) at com.dtstack.taier.develop.service.console.ConsoleComponentService.testComponentWithResult(ConsoleComponentService.java:1024) at com.dtstack.taier.develop.service.console.ConsoleComponentService.testConnect(ConsoleComponentService.java:971) at com.dtstack.taier.develop.service.console.ConsoleComponentService$$FastClassBySpringCGLIB$$e484d26.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:685) at com.dtstack.taier.develop.service.console.ConsoleComponentService$$EnhancerBySpringCGLIB$$865e2d9d.testConnect() at com.dtstack.taier.develop.controller.console.ComponentController.testConnect(ComponentController.java:146) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:190) at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:138) at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:106) at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:888) at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:793) at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87) at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1040) at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:943) at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006) at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:909) at javax.servlet.http.HttpServlet.service(HttpServlet.java:660) at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883) at javax.servlet.http.HttpServlet.service(HttpServlet.java:741) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at com.dtstack.taier.develop.filter.DtRequestWrapperFilter.doFilterInternal(DtRequestWrapperFilter.java:96) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:541) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343) at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:367) at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65) at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:860) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1598) at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:748) Caused by: java.util.concurrent.ExecutionException: java.lang.Exception: com.dtstack.taier.datasource.api.exception.SourceException: execute method error : Active nodes is empty. at java.util.concurrent.FutureTask.report(FutureTask.java:122) at java.util.concurrent.FutureTask.get(FutureTask.java:206) at com.dtstack.taier.datasource.api.utils.RetryUtils$AsyncRetry.call(RetryUtils.java:183) at com.dtstack.taier.datasource.api.utils.RetryUtils$Retry.doRetry(RetryUtils.java:99) at com.dtstack.taier.datasource.api.utils.RetryUtils.asyncExecuteWithRetry(RetryUtils.java:65) at com.dtstack.taier.datasource.api.proxy.ClientProxyInvocationHandle.invokeMethod(ClientProxyInvocationHandle.java:77) at com.dtstack.taier.datasource.api.proxy.ClientProxyInvocationHandle.invoke(ClientProxyInvocationHandle.java:58) ... 56 more Caused by: java.lang.Exception: com.dtstack.taier.datasource.api.exception.SourceException: execute method error : Active nodes is empty. at com.dtstack.taier.datasource.api.proxy.ClientProxyInvocationHandle.lambda$invokeMethod$0(ClientProxyInvocationHandle.java:82) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ... 1 more Caused by: com.dtstack.taier.datasource.api.exception.SourceException: execute method error : Active nodes is empty. at com.dtstack.taier.datasource.plugin.yarn.core.client.YarnClientImpl.lambda$getYarnClient$6(YarnClientImpl.java:450) at com.dtstack.taier.datasource.plugin.kerberos.core.util.SecurityUtils.login(SecurityUtils.java:80) at com.dtstack.taier.datasource.plugin.yarn.core.client.YarnClientImpl.getYarnClient(YarnClientImpl.java:444) at com.dtstack.taier.datasource.plugin.yarn.core.client.YarnClientImpl.testCon(YarnClientImpl.java:78) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.dtstack.taier.datasource.api.proxy.ClientProxyInvocationHandle.lambda$invokeMethod$0(ClientProxyInvocationHandle.java:79) ... 4 more Caused by: com.dtstack.taier.datasource.api.exception.SourceException: Active nodes is empty. at com.dtstack.taier.datasource.plugin.yarn.core.client.YarnClientImpl.lambda$testCon$0(YarnClientImpl.java:85) at com.dtstack.taier.datasource.plugin.yarn.core.client.YarnClientImpl.lambda$getYarnClient$6(YarnClientImpl.java:448) ... 12 more Caused by: java.net.ConnectException: Your endpoint configuration is wrong; For more details see: http://wiki.apache.org/hadoop/UnsetHostnameOrPort at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:824) at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:750) at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1495) at org.apache.hadoop.ipc.Client.call(Client.java:1437) at org.apache.hadoop.ipc.Client.call(Client.java:1347) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:228) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:116) at com.sun.proxy.$Proxy220.getClusterNodes(Unknown Source) at org.apache.hadoop.yarn.api.impl.pb.client.ApplicationClientProtocolPBClientImpl.getClusterNodes(ApplicationClientProtocolPBClientImpl.java:311) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:422) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:165) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:157) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95) at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:359) at com.sun.proxy.$Proxy221.getClusterNodes(Unknown Source) at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.getNodeReports(YarnClientImpl.java:618) at com.dtstack.taier.datasource.plugin.yarn.core.client.YarnClientImpl.lambda$testCon$0(YarnClientImpl.java:80) ... 13 more Caused by: java.net.ConnectException: Connection refused at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717) at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206) at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:531) at org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:685) at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:788) at org.apache.hadoop.ipc.Client$Connection.access$3500(Client.java:409) at org.apache.hadoop.ipc.Client.getConnection(Client.java:1552) at org.apache.hadoop.ipc.Client.call(Client.java:1383) ... 30 more

我的hadoop集群版本是3.36 部署了3台服务器,分别是hadoop100,hadoop102,hadoop102,我通过docker-compose启动的taier,docker-compose文件如下

version: '3' services: taier-db: image: dtopensource/taier-mysql:latest extra_hosts:

Code of Conduct

mfgcs commented 1 year ago

我配置sftp用hadoop100都能连通,所以应该不是网络问题! image

vainhope commented 1 year ago

Active nodes is empty 是指Yarn 获取到的runnig的节点为空 确认下yarn 节点是否都是正常的呢

trying1996 commented 1 year ago

我配置sftp用hadoop100都能连通,所以应该不是网络问题! image

兄弟,解决了吗,我跟你遇到一样的问题

vainhope commented 1 year ago

image 看下集群active node 是否正常

trying1996 commented 1 year ago

image 看下集群active node 是否正常

是正常的,都是active,当前hadoop是3.2.0版本 image image

vainhope commented 1 year ago

看下失败原因呢

trying1996 commented 1 year ago

image 看下集群active node 是否正常

进程也在跑

image image

vainhope commented 1 year ago

image

trying1996 commented 1 year ago

看下失败原因呢

这是web上的错误日志,请问该错误的日志是哪个文件

java.lang.reflect.UndeclaredThrowableException java.lang.reflect.UndeclaredThrowableException at com.sun.proxy.$Proxy213.testCon(Unknown Source) at com.dtstack.taier.scheduler.executor.DatasourceOperator.testConnect(DatasourceOperator.java:68) at com.dtstack.taier.develop.service.console.ConsoleComponentService.testConnect(ConsoleComponentService.java:1137) at com.dtstack.taier.develop.service.console.ConsoleComponentService.testComponentWithResult(ConsoleComponentService.java:1024) at com.dtstack.taier.develop.service.console.ConsoleComponentService.testConnect(ConsoleComponentService.java:971) at com.dtstack.taier.develop.service.console.ConsoleComponentService$$FastClassBySpringCGLIB$$e484d26.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:685) at com.dtstack.taier.develop.service.console.ConsoleComponentService$$EnhancerBySpringCGLIB$$bf462fca.testConnect() at com.dtstack.taier.develop.controller.console.ComponentController.testConnect(ComponentController.java:146) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:190) at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:138) at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:106) at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:888) at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:793) at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87) at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1040) at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:943) at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006) at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:909) at javax.servlet.http.HttpServlet.service(HttpServlet.java:660) at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883) at javax.servlet.http.HttpServlet.service(HttpServlet.java:741) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at com.dtstack.taier.develop.filter.DtRequestWrapperFilter.doFilterInternal(DtRequestWrapperFilter.java:96) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:541) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343) at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:367) at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65) at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:860) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1598) at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:748) Caused by: java.util.concurrent.ExecutionException: java.lang.Exception: com.dtstack.taier.datasource.api.exception.SourceException: execute method error : Active nodes is empty. at java.util.concurrent.FutureTask.report(FutureTask.java:122) at java.util.concurrent.FutureTask.get(FutureTask.java:206) at com.dtstack.taier.datasource.api.utils.RetryUtils$AsyncRetry.call(RetryUtils.java:183) at com.dtstack.taier.datasource.api.utils.RetryUtils$Retry.doRetry(RetryUtils.java:99) at com.dtstack.taier.datasource.api.utils.RetryUtils.asyncExecuteWithRetry(RetryUtils.java:65) at com.dtstack.taier.datasource.api.proxy.ClientProxyInvocationHandle.invokeMethod(ClientProxyInvocationHandle.java:77) at com.dtstack.taier.datasource.api.proxy.ClientProxyInvocationHandle.invoke(ClientProxyInvocationHandle.java:58) ... 56 more Caused by: java.lang.Exception: com.dtstack.taier.datasource.api.exception.SourceException: execute method error : Active nodes is empty. at com.dtstack.taier.datasource.api.proxy.ClientProxyInvocationHandle.lambda$invokeMethod$0(ClientProxyInvocationHandle.java:82) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ... 1 more Caused by: com.dtstack.taier.datasource.api.exception.SourceException: execute method error : Active nodes is empty. at com.dtstack.taier.datasource.plugin.yarn.core.client.YarnClientImpl.lambda$getYarnClient$6(YarnClientImpl.java:450) at com.dtstack.taier.datasource.plugin.kerberos.core.util.SecurityUtils.login(SecurityUtils.java:80) at com.dtstack.taier.datasource.plugin.yarn.core.client.YarnClientImpl.getYarnClient(YarnClientImpl.java:444) at com.dtstack.taier.datasource.plugin.yarn.core.client.YarnClientImpl.testCon(YarnClientImpl.java:78) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.dtstack.taier.datasource.api.proxy.ClientProxyInvocationHandle.lambda$invokeMethod$0(ClientProxyInvocationHandle.java:79) ... 4 more Caused by: com.dtstack.taier.datasource.api.exception.SourceException: Active nodes is empty. at com.dtstack.taier.datasource.plugin.yarn.core.client.YarnClientImpl.lambda$testCon$0(YarnClientImpl.java:85) at com.dtstack.taier.datasource.plugin.yarn.core.client.YarnClientImpl.lambda$getYarnClient$6(YarnClientImpl.java:448) ... 12 more Caused by: java.net.ConnectException: Your endpoint configuration is wrong; For more details see: http://wiki.apache.org/hadoop/UnsetHostnameOrPort at sun.reflect.GeneratedConstructorAccessor147.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:824) at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:750) at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1495) at org.apache.hadoop.ipc.Client.call(Client.java:1437) at org.apache.hadoop.ipc.Client.call(Client.java:1347) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:228) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:116) at com.sun.proxy.$Proxy220.getClusterNodes(Unknown Source) at org.apache.hadoop.yarn.api.impl.pb.client.ApplicationClientProtocolPBClientImpl.getClusterNodes(ApplicationClientProtocolPBClientImpl.java:311) at sun.reflect.GeneratedMethodAccessor231.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:422) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:165) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:157) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95) at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:359) at com.sun.proxy.$Proxy221.getClusterNodes(Unknown Source) at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.getNodeReports(YarnClientImpl.java:618) at com.dtstack.taier.datasource.plugin.yarn.core.client.YarnClientImpl.lambda$testCon$0(YarnClientImpl.java:80) ... 13 more Caused by: java.net.ConnectException: Connection refused at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717) at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206) at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:531) at org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:685) at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:788) at org.apache.hadoop.ipc.Client$Connection.access$3500(Client.java:409) at org.apache.hadoop.ipc.Client.getConnection(Client.java:1552) at org.apache.hadoop.ipc.Client.call(Client.java:1383) ... 29 more

vainhope commented 1 year ago

Caused by: java.net.ConnectException: Your endpoint configuration is wrong; 确认下rm的端口和ip taier是不是都能ping通

trying1996 commented 1 year ago

Caused by: java.net.ConnectException: Your endpoint configuration is wrong; 确认下rm的端口和ip taier是不是都能ping通

可以啊

image

trying1996 commented 1 year ago

Caused by: java.net.ConnectException: Your endpoint configuration is wrong; 确认下rm的端口和ip taier是不是都能ping通

你说的端口是指8088吧?telnet也可以

trying1996 commented 1 year ago

Caused by: java.net.ConnectException: Your endpoint configuration is wrong; 确认下rm的端口和ip taier是不是都能ping通

19888的historyserver端口不通,是这样原因吗

vainhope commented 1 year ago

和historyserver没有关系 你看下taier是在bigdata04的机器上吗 如果是单独的机器 和rm1,rm2的8032端口都通吗 host有配置吗

trying1996 commented 1 year ago

和historyserver没有关系 你看下taier是在bigdata04的机器上吗 如果是单独的机器 和rm1,rm2的8032端口都通吗 host有配置吗

少了些配置,现在可以通了,但又有新问题,页面上会一直有这个提示,并且无法选择任务类型 image image

vainhope commented 1 year ago

https://dtstack.github.io/Taier/docs/functions/multi-cluster 参考组件和任务对应关系配置下计算组件

hellomyheart commented 11 months ago

@trying1996 兄弟你的问题最后怎么解决的,我可以看一下你的配置吗?我和你一样,也是Active nodes is empty.

hellomyheart commented 11 months ago

@mfgcs 兄弟你的问题最后怎么解决的,我可以看一下你的配置吗?我和你一样,也是Active nodes is empty. 但是hadoop上看是正常的