search

apache / dolphinscheduler

Apache DolphinScheduler is the modern data orchestration platform. Agile to create high performance workflow with low-code
https://dolphinscheduler.apache.org/
Apache License 2.0
12.78k stars 4.59k forks source link

[Question] Create tenant to report an error #4328

Closed xiaot502 closed 3 years ago

xiaot502 commented 3 years ago

For better global communication, Please describe it in English. If you feel the description in English is not clear, then you can append description in Chinese(just for Mandarin(CN)), thx! Describe the question A clear and concise description of what the question is.

Which version of DolphinScheduler: -[1.3.3-preview]

Additional context

cat dolphinscheduler-api-server.2020-12-28_23.0.log
api日志报错信息如下： [INFO] 2020-12-28 23:38:58.327 org.eclipse.jetty.util.log:[193] - Logging initialized @18647ms to org.eclipse.jetty.util.log.Slf4jLog [INFO] 2020-12-28 23:38:59.082 org.eclipse.jetty.server.Server:[370] - jetty-9.4.14.v20181114; built: 2018-11-14T21:20:31.478Z; git: c4550056e785fb5665914545889f21dc136ad9e6; jvm 1.8.0_252-b09 [INFO] 2020-12-28 23:38:59.468 org.eclipse.jetty.server.handler.ContextHandler.application:[2345] - Initializing Spring embedded WebApplicationContext [INFO] 2020-12-28 23:38:59.716 org.eclipse.jetty.server.session:[365] - DefaultSessionIdManager workerName=node0 [INFO] 2020-12-28 23:38:59.716 org.eclipse.jetty.server.session:[370] - No SessionScavenger set, using defaults [INFO] 2020-12-28 23:38:59.720 org.eclipse.jetty.server.session:[149] - node0 Scavenging every 660000ms [INFO] 2020-12-28 23:38:59.743 org.eclipse.jetty.server.handler.ContextHandler:[855] - Started o.s.b.w.e.j.JettyEmbeddedWebAppContext@18d11527{application,/dolphinscheduler,[file:///tmp/jetty-docbase.1455525805337213889.12345/, jar:file:/opt/dolphinscheduler/lib/springfox-swagger-ui-2.9.2.jar!/META-INF/resources, jar:file:/opt/dolphinscheduler/lib/swagger-bootstrap-ui-1.9.3.jar!/META-INF/resources],AVAILABLE} [INFO] 2020-12-28 23:38:59.757 org.eclipse.jetty.server.Server:[407] - Started @20082ms [INFO] 2020-12-28 23:39:05.541 org.apache.dolphinscheduler.service.zk.ZookeeperOperator:[84] - zookeeper registry center init, server lists is: sv0:2183,sv1:2183,sv2:2183. [INFO] 2020-12-28 23:39:05.654 org.apache.curator.utils.Compatibility:[48] - Running in ZooKeeper 3.4.x compatibility mode [INFO] 2020-12-28 23:39:05.656 org.apache.curator.utils.Compatibility:[61] - Using emulated InjectSessionExpiration [INFO] 2020-12-28 23:39:05.708 org.apache.curator.framework.imps.CuratorFrameworkImpl:[308] - Starting [INFO] 2020-12-28 23:39:05.754 org.apache.curator.framework.imps.CuratorFrameworkImpl:[356] - Default schema [INFO] 2020-12-28 23:39:05.807 org.apache.curator.framework.state.ConnectionStateManager:[251] - State change: CONNECTED [INFO] 2020-12-28 23:39:05.814 org.apache.dolphinscheduler.service.zk.ZookeeperOperator:[84] - zookeeper registry center init, server lists is: sv0:2183,sv1:2183,sv2:2183. [INFO] 2020-12-28 23:39:05.815 org.apache.curator.framework.imps.CuratorFrameworkImpl:[308] - Starting [INFO] 2020-12-28 23:39:05.818 org.apache.curator.framework.imps.CuratorFrameworkImpl:[356] - Default schema [INFO] 2020-12-28 23:39:05.860 org.apache.curator.framework.state.ConnectionStateManager:[251] - State change: CONNECTED [INFO] 2020-12-28 23:39:05.877 org.apache.dolphinscheduler.service.zk.ZookeeperCachedOperator:[55] - add listener to zk path: /dolphinscheduler [INFO] 2020-12-28 23:39:06.187 org.apache.dolphinscheduler.service.zk.ZookeeperOperator:[84] - zookeeper registry center init, server lists is: sv0:2183,sv1:2183,sv2:2183. [INFO] 2020-12-28 23:39:06.188 org.apache.curator.framework.imps.CuratorFrameworkImpl:[308] - Starting [INFO] 2020-12-28 23:39:06.193 org.apache.curator.framework.imps.CuratorFrameworkImpl:[356] - Default schema [INFO] 2020-12-28 23:39:06.215 org.apache.curator.framework.state.ConnectionStateManager:[251] - State change: CONNECTED [INFO] 2020-12-28 23:39:06.216 org.apache.dolphinscheduler.service.zk.ZookeeperCachedOperator:[55] - add listener to zk path: /dolphinscheduler [INFO] 2020-12-28 23:39:11.220 springfox.documentation.spring.web.plugins.DocumentationPluginsBootstrapper:[160] - Context refreshed [INFO] 2020-12-28 23:39:11.329 springfox.documentation.spring.web.plugins.DocumentationPluginsBootstrapper:[163] - Found 1 custom documentation plugin(s) [INFO] 2020-12-28 23:39:11.634 springfox.documentation.spring.web.scanners.ApiListingReferenceScanner:[41] - Scanning for api listing references [INFO] 2020-12-28 23:39:12.467 springfox.documentation.spring.web.readers.operation.CachingOperationNameGenerator:[40] - Generating unique operation named: viewTreeUsingGET_1 [INFO] 2020-12-28 23:39:12.861 org.eclipse.jetty.server.handler.ContextHandler.application:[2345] - Initializing Spring DispatcherServlet 'dispatcherServlet' [INFO] 2020-12-28 23:39:12.899 org.mortbay.log:[67] - Logging to Logger[org.mortbay.log] via org.mortbay.log.Slf4jLog [WARN] 2020-12-28 23:39:12.964 org.apache.jasper.EmbeddedServletOptions:[76] - PWC5963: The scratchDir you specified: /tmp/jsp is unusable. [INFO] 2020-12-28 23:39:13.024 org.eclipse.jetty.server.AbstractConnector:[292] - Started ServerConnector@5724c7da{HTTP/1.1,[http/1.1]}{0.0.0.0:12345} [INFO] 2020-12-28 23:39:13.950 ru.yandex.clickhouse.ClickHouseDriver:[42] - Driver registered [INFO] 2020-12-28 23:39:14.813 com.alibaba.druid.pool.DruidDataSource:[991] - {dataSource-1} inited [INFO] 2020-12-28 23:39:15.016 org.apache.dolphinscheduler.api.controller.UsersController:[299] - login user admin,get user info [INFO] 2020-12-28 23:39:15.312 org.apache.dolphinscheduler.api.controller.DataAnalysisController:[126] - count process definition , user:admin, project id:0 [INFO] 2020-12-28 23:39:15.317 org.apache.dolphinscheduler.api.controller.DataAnalysisController:[104] - count process instance state, user:admin, start date: 2020-12-28 00:00:00, end date:2020-12-28 23:39:15, project id:0 [INFO] 2020-12-28 23:39:15.317 org.apache.dolphinscheduler.api.controller.DataAnalysisController:[76] - count task state, user:admin, start date: 2020-12-28 00:00:00, end date:2020-12-28 23:39:15, project id 0 [INFO] 2020-12-28 23:39:21.060 org.apache.dolphinscheduler.api.controller.TenantController:[112] - login user admin, list paging, pageNo: 1, searchVal: , pageSize: 10 [INFO] 2020-12-28 23:39:22.521 org.apache.dolphinscheduler.api.controller.QueueController:[68] - login user admin, query queue list [INFO] 2020-12-28 23:39:30.587 org.apache.dolphinscheduler.api.controller.TenantController:[216] - login user admin, verfiy tenant code: hadoop [INFO] 2020-12-28 23:39:30.618 org.apache.dolphinscheduler.api.controller.TenantController:[83] - login user admin, create tenant, tenantCode: hadoop, tenantName: hadoop, queueId: 1, desc: [ERROR] 2020-12-28 23:39:30.777 org.apache.dolphinscheduler.common.utils.HadoopUtils:[172] - Can't get Kerberos realm java.lang.IllegalArgumentException: Can't get Kerberos realm at org.apache.hadoop.security.HadoopKerberosName.setConfiguration(HadoopKerberosName.java:65) at org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:276) at org.apache.hadoop.security.UserGroupInformation.setConfiguration(UserGroupInformation.java:312) at org.apache.dolphinscheduler.common.utils.HadoopUtils.init(HadoopUtils.java:123) at org.apache.dolphinscheduler.common.utils.HadoopUtils.(HadoopUtils.java:81) at org.apache.dolphinscheduler.common.utils.HadoopUtils.(HadoopUtils.java:54) at org.apache.dolphinscheduler.common.utils.HadoopUtils$1.load(HadoopUtils.java:71) at org.apache.dolphinscheduler.common.utils.HadoopUtils$1.load(HadoopUtils.java:68) at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3628) at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2336) at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2295) at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2208) at com.google.common.cache.LocalCache.get(LocalCache.java:4053) at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:4057) at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4986) at com.google.common.cache.LocalCache$LocalLoadingCache.getUnchecked(LocalCache.java:4992) at org.apache.dolphinscheduler.common.utils.HadoopUtils.getInstance(HadoopUtils.java:87) at org.apache.dolphinscheduler.api.service.BaseService.createTenantDirIfNotExists(BaseService.java:147) at org.apache.dolphinscheduler.api.service.TenantService.createTenant(TenantService.java:120) at org.apache.dolphinscheduler.api.service.TenantService$$FastClassBySpringCGLIB$$13bc5043.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:749) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:294) at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:98) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:688) at org.apache.dolphinscheduler.api.service.TenantService$$EnhancerBySpringCGLIB$$8b3770dc.createTenant() at org.apache.dolphinscheduler.api.controller.TenantController.createTenant(TenantController.java:85) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:189) at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:138) at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:102) at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:895) at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:800) at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87) at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1038) at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:942) at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1005) at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:908) at javax.servlet.http.HttpServlet.service(HttpServlet.java:707) at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:882) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:867) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1623) at com.github.xiaoymin.swaggerbootstrapui.filter.SecurityBasicAuthFilter.doFilter(SecurityBasicAuthFilter.java:84) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610) at com.github.xiaoymin.swaggerbootstrapui.filter.ProductionSecurityFilter.doFilter(ProductionSecurityFilter.java:53) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610) at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:99) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610) at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:92) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610) at org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:93) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610) at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:200) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:540) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1588) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1345) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1557) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1247) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at org.eclipse.jetty.server.Server.handle(Server.java:502) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:364) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103) at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.reflect.InvocationTargetException: null at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.security.authentication.util.KerberosUtil.getDefaultRealm(KerberosUtil.java:84) at org.apache.hadoop.security.HadoopKerberosName.setConfiguration(HadoopKerberosName.java:63) ... 92 common frames omitted Caused by: sun.security.krb5.KrbException: Cannot locate default realm at sun.security.krb5.Config.getDefaultRealm(Config.java:1134) ... 98 common frames omitted [ERROR] 2020-12-28 23:39:30.779 org.apache.dolphinscheduler.common.utils.HadoopUtils:[102] - null java.lang.NullPointerException: null at org.apache.dolphinscheduler.common.utils.HadoopUtils.initHdfsPath(HadoopUtils.java:98) at org.apache.dolphinscheduler.common.utils.HadoopUtils.(HadoopUtils.java:82) at org.apache.dolphinscheduler.common.utils.HadoopUtils.(HadoopUtils.java:54) at org.apache.dolphinscheduler.common.utils.HadoopUtils$1.load(HadoopUtils.java:71) at org.apache.dolphinscheduler.common.utils.HadoopUtils$1.load(HadoopUtils.java:68) at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3628) at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2336) at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2295) at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2208) at com.google.common.cache.LocalCache.get(LocalCache.java:4053) at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:4057) at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4986) at com.google.common.cache.LocalCache$LocalLoadingCache.getUnchecked(LocalCache.java:4992) at org.apache.dolphinscheduler.common.utils.HadoopUtils.getInstance(HadoopUtils.java:87) at org.apache.dolphinscheduler.api.service.BaseService.createTenantDirIfNotExists(BaseService.java:147) at org.apache.dolphinscheduler.api.service.TenantService.createTenant(TenantService.java:120) at org.apache.dolphinscheduler.api.service.TenantService$$FastClassBySpringCGLIB$$13bc5043.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:749) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:294) at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:98) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:688) at org.apache.dolphinscheduler.api.service.TenantService$$EnhancerBySpringCGLIB$$8b3770dc.createTenant() at org.apache.dolphinscheduler.api.controller.TenantController.createTenant(TenantController.java:85) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:189) at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:138) at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:102) at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:895) at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:800) at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87) at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1038) at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:942) at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1005) at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:908) at javax.servlet.http.HttpServlet.service(HttpServlet.java:707) at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:882) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:867) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1623) at com.github.xiaoymin.swaggerbootstrapui.filter.SecurityBasicAuthFilter.doFilter(SecurityBasicAuthFilter.java:84) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610) at com.github.xiaoymin.swaggerbootstrapui.filter.ProductionSecurityFilter.doFilter(ProductionSecurityFilter.java:53) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610) at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:99) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610) at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:92) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610) at org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:93) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610) at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:200) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:540) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1588) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1345) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1557) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1247) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at org.eclipse.jetty.server.Server.handle(Server.java:502) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:364) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103) at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683) at java.lang.Thread.run(Thread.java:748) [ERROR] 2020-12-28 23:39:30.788 org.apache.dolphinscheduler.api.exceptions.ApiExceptionHandler:[45] - 创建租户错误 java.lang.NullPointerException: null at org.apache.dolphinscheduler.common.utils.HadoopUtils.mkdir(HadoopUtils.java:268) at org.apache.dolphinscheduler.api.service.BaseService.createTenantDirIfNotExists(BaseService.java:147) at org.apache.dolphinscheduler.api.service.TenantService.createTenant(TenantService.java:120) at org.apache.dolphinscheduler.api.service.TenantService$$FastClassBySpringCGLIB$$13bc5043.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:749) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:294) at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:98) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:688) at org.apache.dolphinscheduler.api.service.TenantService$$EnhancerBySpringCGLIB$$8b3770dc.createTenant() at org.apache.dolphinscheduler.api.controller.TenantController.createTenant(TenantController.java:85) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:189) at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:138) at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:102) at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:895) at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:800) at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87) at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1038) at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:942) at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1005) at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:908) at javax.servlet.http.HttpServlet.service(HttpServlet.java:707) at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:882) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:867) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1623) at com.github.xiaoymin.swaggerbootstrapui.filter.SecurityBasicAuthFilter.doFilter(SecurityBasicAuthFilter.java:84) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610) at com.github.xiaoymin.swaggerbootstrapui.filter.ProductionSecurityFilter.doFilter(ProductionSecurityFilter.java:53) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610) at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:99) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610) at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:92) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610) at org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:93) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610) at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:200) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:540) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1588) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1345) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1557) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1247) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at org.eclipse.jetty.server.Server.handle(Server.java:502) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:364) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103) at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683) at java.lang.Thread.run(Thread.java:748)

Requirement or improvement In the web management page to create a tenant, the above error message appears, please help to see what the problem is? 在web管理页面创建租户，出现上面的报错信息，请帮忙看看问题出在什么地方？

felix-thinkingdata commented 3 years ago

Can't get Kerberos ，Please confirm that your configuration is correct.

davidzollo commented 3 years ago

Can't get Kerberos ，Please confirm that your configuration is correct.

yes, you should check the config in the common.properties, usually it is because your following config are not right 

# whether kerberos starts
hadoop.security.authentication.startup.state=true

# java.security.krb5.conf path
java.security.krb5.conf.path=/opt/krb5.conf

# login user from keytab username
login.user.keytab.username=hdfs-mycluster@ESZ.COM

# loginUserFromKeytab path
login.user.keytab.path=/opt/hdfs.headless.keytab
xiaot502 commented 3 years ago

my   common.properties  configuration is as follows:

resource.storage.type=HDFS

resource.storage.type=HDFS

resource store on HDFS/S3 path, resource file will store to this hadoop hdfs path, self configuration, please make sure the directory exists on hdfs and have read write permissions。"/dolphinscheduler" is recommended

resource.upload.path=/dolphinscheduler

user data local directory path, please make sure the directory exists and have read write permissions

data.basedir.path=/tmp/dolphinscheduler

whether kerberos starts

hadoop.security.authentication.startup.state=true

java.security.krb5.conf.path=/opt/dolphinscheduler/conf/krb5.conf

java.security.krb5.conf.path=/opt/dolphinscheduler/conf/krb5.conf

login user from keytab username

login.user.keytab.username=hdfs-mycluster@ESZ.COM

loginUserFromKeytab path

login.user.keytab.path=/opt/dolphinscheduler/conf/hdfs.headless.keytab

resource.view.suffixs

resource.view.suffixs=txt,log,sh,conf,cfg,py,java,sql,hql,xml,properties

if resource.storage.type=HDFS

hdfs.root.user=hadoop

if resource.storage.type=HDFS

fs.defaultFS=hdfs://sv1:8020

if resource.storage.type=HDFS

fs.s3a.endpoint=http://192.168.xx.xx:9010

if resource.storage.type=HDFS

fs.s3a.access.key=xxxxxxxxxx

if resource.storage.type=HDFS

fs.s3a.secret.key=xxxxxxxxxx

if resourcemanager HA enable, please type the HA ips ; if resourcemanager is single, make this value empty

yarn.resourcemanager.ha.rm.ids=

if resourcemanager HA enable or not use resourcemanager, please keep the default value; If resourcemanager is single, you only need to replace ds1 to actual resourcemanager hostname.

yarn.application.status.address=http://sv1:8088/ws/v1/cluster/apps/%s

system env path

dolphinscheduler.env.path=env/dolphinscheduler_env.sh

development.state=false

I don't know if the Dolphin Scheduler has to configure Kerberos when it uses the HDFS file system as a resource 我现在不明白海豚调度器 使用hdfs文件系统作为资源存储的时候是否必去配置kerberos

------------------ 原始邮件 ------------------ 发件人: "dailidong"<notifications@github.com>; 发送时间: 2020年12月29日(星期二) 晚上11:00 收件人: "apache/incubator-dolphinscheduler"<incubator-dolphinscheduler@noreply.github.com>; 抄送: "涛"<302983717@qq.com>; "Author"<author@noreply.github.com>; 主题: Re: [apache/incubator-dolphinscheduler] [Question] Create tenant to report an error (#4328)

Can't get Kerberos ，Please confirm that your configuration is correct.

yes, you should check the config in the common.properties, usually it is because your following config are not right

whether kerberos starts hadoop.security.authentication.startup.state=true # java.security.krb5.conf path java.security.krb5.conf.path=/opt/krb5.conf # login user from keytab username login.user.keytab.username=hdfs-mycluster@ESZ.COM # loginUserFromKeytab path login.user.keytab.path=/opt/hdfs.headless.keytab

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.

zhuangchong commented 3 years ago

Whether Dolphin's storage resources need to be configured with Kerberos for HDFS depends on whether your big data cluster has Kerberos authentication enabled. If Kerberos is not enabled, no configuration is required; Kerberos-related parameters are required if Kerberos authentication is enabled

—- dolphin的存储资源使用hdfs是否需要配置kerberos，主要看你的大数据集群是否开启kerberos认证。如果没有开启kerberos就不需要配置；如果开启Kerberos认证则需要填写kerberos相关参数

xiaot502 commented 3 years ago

谢谢大家，把kerberos关闭之后就可以创建租户了。但是使用资源管理中心的时候发现数据库初始化脚本问题，我会在bug里面去提交。 Thank you. Once Kerberos is shut down, tenants can be created.