apache / incubator-streampark

Make stream processing easier! Easy-to-use streaming application development framework and operation platform.
https://streampark.apache.org/
Apache License 2.0
3.9k stars 1.01k forks source link

[Bug] create application on yarn 模式, 提示 Hadoop environment initialization failed #614

Closed chuansliu closed 2 years ago

chuansliu commented 2 years ago

请在提交你的 issue 前,请回答以下问题,这有助于社区快速定位问题,谢谢! 🙏

Environment Description(运行环境描述)

Bug Description(Bug 描述)

创建 application 选择 "yarn application", 显示 "Hadoop environment initialization failed, please check the environment settings". 此时查看后端日志会看到一直卡在 “[StreamX] kerberos login starting....” 这里。 但所有的 hadoop环境变量已经 export 好了的。

How to Reproduce(如何重现这个 Bug)

  1. create new application
  2. yarn application 模式
  3. See error

Additional context(额外的上下文信息)

image

启动时控制台打印日志: [StreamX] Using APP_BASE: /opt/soft/streamx/streamx1.2.1/streamx-console-service-1.2.1 [StreamX] Using APP_HOME: /opt/soft/streamx/streamx1.2.1/streamx-console-service-1.2.1 [StreamX] Using JRE_HOME: /usr/java/jdk1.8.0_161 [StreamX] Using APP_PID: /opt/soft/streamx/streamx1.2.1/streamx-console-service-1.2.1/.pid [StreamX] Usage: properties file:application.yml [StreamX] Using HADOOP_HOME: /data/cloudera/parcels/CDH/lib/hadoop [StreamX] Using HADOOP_CONF_DIR: /etc/hadoop/conf [StreamX] StreamX start successful. pid: 8973

运行日志: 22/01/12 10:47:37 INFO util.Version: HV000001: Hibernate Validator 6.1.7.Final 2022-01-12 10:47:37 | INFO | main | com.streamxhub.streamx.console.StreamXConsole ] Starting StreamXConsole v1.2.1 on node09.data.com with PID 8973 (/opt/soft/streamx/streamx1.2.1/streamx-console-service-1.2.1/lib/streamx-console-service-1.2.1.jar started by devuser in /opt/soft/streamx/streamx1.2.1/streamx-console-service-1.2.1) 2022-01-12 10:47:37 | INFO | main | com.streamxhub.streamx.console.StreamXConsole ] No active profile set, falling back to default profiles: default 2022-01-12 10:47:38 | INFO | main | org.springframework.context.support.PostProcessorRegistrationDelegate$BeanPostProcessorChecker ] Bean 'spring.datasource.dynamic-com.baomidou.dynamic.datasource.spring.boot.autoconfigure.DynamicDataSourceProperties' of type [com.baomidou.dynamic.datasource.spring.boot.autoconfigure.DynamicDataSourceProperties] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying) 2022-01-12 10:47:38 | INFO | main | org.springframework.context.support.PostProcessorRegistrationDelegate$BeanPostProcessorChecker ] Bean 'com.baomidou.dynamic.datasource.spring.boot.autoconfigure.DynamicDataSourceAutoConfiguration' of type [com.baomidou.dynamic.datasource.spring.boot.autoconfigure.DynamicDataSourceAutoConfiguration$$EnhancerBySpringCGLIB$$6cec0684] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying) 2022-01-12 10:47:38 | INFO | main | org.springframework.context.support.PostProcessorRegistrationDelegate$BeanPostProcessorChecker ] Bean 'dsProcessor' of type [com.baomidou.dynamic.datasource.processor.DsHeaderProcessor] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying) 2022-01-12 10:47:38 | INFO | main | org.springframework.context.support.PostProcessorRegistrationDelegate$BeanPostProcessorChecker ] Bean 'dynamicDatasourceAnnotationAdvisor' of type [com.baomidou.dynamic.datasource.aop.DynamicDataSourceAnnotationAdvisor] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying) 2022-01-12 10:47:38 | INFO | main | org.springframework.context.support.PostProcessorRegistrationDelegate$BeanPostProcessorChecker ] Bean 'shiroConfig' of type [com.streamxhub.streamx.console.system.authentication.ShiroConfig$$EnhancerBySpringCGLIB$$1aa13ae3] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying) 2022-01-12 10:47:38 | INFO | main | org.springframework.context.support.PostProcessorRegistrationDelegate$BeanPostProcessorChecker ] Bean 'mybatis-plus-com.baomidou.mybatisplus.autoconfigure.MybatisPlusProperties' of type [com.baomidou.mybatisplus.autoconfigure.MybatisPlusProperties] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying) 2022-01-12 10:47:38 | INFO | main | org.springframework.context.support.PostProcessorRegistrationDelegate$BeanPostProcessorChecker ] Bean 'mybatisPlusConfig' of type [com.streamxhub.streamx.console.base.config.MybatisPlusConfig$$EnhancerBySpringCGLIB$$8ee70e27] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying) 2022-01-12 10:47:38 | INFO | main | org.springframework.context.support.PostProcessorRegistrationDelegate$BeanPostProcessorChecker ] Bean 'paginationInterceptor' of type [com.baomidou.mybatisplus.extension.plugins.PaginationInterceptor] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying) 2022-01-12 10:47:38 | INFO | main | org.springframework.context.support.PostProcessorRegistrationDelegate$BeanPostProcessorChecker ] Bean 'com.baomidou.mybatisplus.autoconfigure.MybatisPlusAutoConfiguration' of type [com.baomidou.mybatisplus.autoconfigure.MybatisPlusAutoConfiguration$$EnhancerBySpringCGLIB$$4126024d] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying) 2022-01-12 10:47:38 | INFO | main | org.springframework.context.support.PostProcessorRegistrationDelegate$BeanPostProcessorChecker ] Bean 'dynamicDataSourceCreator' of type [com.baomidou.dynamic.datasource.DynamicDataSourceCreator] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying) 2022-01-12 10:47:38 | INFO | main | org.springframework.context.support.PostProcessorRegistrationDelegate$BeanPostProcessorChecker ] Bean 'dynamicDataSourceProvider' of type [com.baomidou.dynamic.datasource.provider.YmlDynamicDataSourceProvider] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying) 2022-01-12 10:47:38 | INFO | main | com.zaxxer.hikari.HikariDataSource ] primary - Starting... 2022-01-12 10:47:39 | INFO | main | com.zaxxer.hikari.HikariDataSource ] primary - Start completed. 2022-01-12 10:47:39 | INFO | main | com.baomidou.dynamic.datasource.DynamicRoutingDataSource ] 初始共加载 1 个数据源 2022-01-12 10:47:39 | INFO | main | com.baomidou.dynamic.datasource.DynamicRoutingDataSource ] 动态数据源-加载 primary 成功 2022-01-12 10:47:39 | INFO | main | com.baomidou.dynamic.datasource.DynamicRoutingDataSource ] 当前的默认数据源是单数据源,数据源名为 primary 2022-01-12 10:47:39 | INFO | main | org.springframework.context.support.PostProcessorRegistrationDelegate$BeanPostProcessorChecker ] Bean 'dataSource' of type [com.baomidou.dynamic.datasource.DynamicRoutingDataSource] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying) 2022-01-12 10:47:39 | INFO | main | org.springframework.context.support.PostProcessorRegistrationDelegate$BeanPostProcessorChecker ] Bean 'spring.datasource-org.springframework.boot.autoconfigure.jdbc.DataSourceProperties' of type [org.springframework.boot.autoconfigure.jdbc.DataSourceProperties] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying) 2022-01-12 10:47:39 | INFO | main | org.springframework.context.support.PostProcessorRegistrationDelegate$BeanPostProcessorChecker ] Bean 'org.springframework.boot.autoconfigure.jdbc.DataSourceInitializerInvoker' of type [org.springframework.boot.autoconfigure.jdbc.DataSourceInitializerInvoker] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying) 2022-01-12 10:47:40 | WARN | main | com.baomidou.mybatisplus.core.toolkit.TableInfoHelper ] Warn: Could not find @TableId in Class: com.streamxhub.streamx.console.core.entity.Setting. 2022-01-12 10:47:40 | INFO | main | org.springframework.context.support.PostProcessorRegistrationDelegate$BeanPostProcessorChecker ] Bean 'sqlSessionFactory' of type [org.apache.ibatis.session.defaults.DefaultSqlSessionFactory] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying) 2022-01-12 10:47:40 | INFO | main | org.springframework.context.support.PostProcessorRegistrationDelegate$BeanPostProcessorChecker ] Bean 'sqlSessionTemplate' of type [org.mybatis.spring.SqlSessionTemplate] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying) 2022-01-12 10:47:40 | INFO | main | org.springframework.context.support.PostProcessorRegistrationDelegate$BeanPostProcessorChecker ] Bean 'userMapper' of type [org.mybatis.spring.mapper.MapperFactoryBean] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying) 2022-01-12 10:47:40 | INFO | main | org.springframework.context.support.PostProcessorRegistrationDelegate$BeanPostProcessorChecker ] Bean 'userMapper' of type [com.sun.proxy.$Proxy93] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying) 2022-01-12 10:47:40 | INFO | main | org.springframework.context.support.PostProcessorRegistrationDelegate$BeanPostProcessorChecker ] Bean 'userRoleMapper' of type [org.mybatis.spring.mapper.MapperFactoryBean] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying) 2022-01-12 10:47:40 | INFO | main | org.springframework.context.support.PostProcessorRegistrationDelegate$BeanPostProcessorChecker ] Bean 'userRoleMapper' of type [com.sun.proxy.$Proxy94] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying) 2022-01-12 10:47:40 | INFO | main | org.springframework.context.support.PostProcessorRegistrationDelegate$BeanPostProcessorChecker ] Bean 'userRoleServiceImpl' of type [com.streamxhub.streamx.console.system.service.impl.UserRoleServiceImpl] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying) 2022-01-12 10:47:40 | INFO | main | org.springframework.context.support.PostProcessorRegistrationDelegate$BeanPostProcessorChecker ] Bean 'roleMapper' of type [org.mybatis.spring.mapper.MapperFactoryBean] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying) 2022-01-12 10:47:40 | INFO | main | org.springframework.context.support.PostProcessorRegistrationDelegate$BeanPostProcessorChecker ] Bean 'roleMapper' of type [com.sun.proxy.$Proxy96] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying) 2022-01-12 10:47:40 | INFO | main | org.springframework.context.support.PostProcessorRegistrationDelegate$BeanPostProcessorChecker ] Bean 'roleMenuMapper' of type [org.mybatis.spring.mapper.MapperFactoryBean] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying) 2022-01-12 10:47:40 | INFO | main | org.springframework.context.support.PostProcessorRegistrationDelegate$BeanPostProcessorChecker ] Bean 'roleMenuMapper' of type [com.sun.proxy.$Proxy97] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying) 2022-01-12 10:47:40 | INFO | main | org.springframework.context.support.PostProcessorRegistrationDelegate$BeanPostProcessorChecker ] Bean 'roleMenuServiceImpl' of type [com.streamxhub.streamx.console.system.service.impl.RoleMenuServiceImpl] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying) 2022-01-12 10:47:40 | INFO | main | org.springframework.context.support.PostProcessorRegistrationDelegate$BeanPostProcessorChecker ] Bean 'roleServiceImpl' of type [com.streamxhub.streamx.console.system.service.impl.RoleServiceImpl] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying) 2022-01-12 10:47:40 | INFO | main | org.springframework.context.support.PostProcessorRegistrationDelegate$BeanPostProcessorChecker ] Bean 'menuMapper' of type [org.mybatis.spring.mapper.MapperFactoryBean] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying) 2022-01-12 10:47:40 | INFO | main | org.springframework.context.support.PostProcessorRegistrationDelegate$BeanPostProcessorChecker ] Bean 'menuMapper' of type [com.sun.proxy.$Proxy100] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying) 2022-01-12 10:47:40 | INFO | main | org.springframework.context.support.PostProcessorRegistrationDelegate$BeanPostProcessorChecker ] Bean 'menuServiceImpl' of type [com.streamxhub.streamx.console.system.service.impl.MenuServiceImpl] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying) 2022-01-12 10:47:40 | INFO | main | org.springframework.context.support.PostProcessorRegistrationDelegate$BeanPostProcessorChecker ] Bean 'userServiceImpl' of type [com.streamxhub.streamx.console.system.service.impl.UserServiceImpl] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying) 2022-01-12 10:47:40 | INFO | main | org.springframework.context.support.PostProcessorRegistrationDelegate$BeanPostProcessorChecker ] Bean 'shiroRealm' of type [com.streamxhub.streamx.console.system.authentication.ShiroRealm] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying) 2022-01-12 10:47:40 | INFO | main | org.springframework.context.support.PostProcessorRegistrationDelegate$BeanPostProcessorChecker ] Bean 'securityManager' of type [org.apache.shiro.web.mgt.DefaultWebSecurityManager] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying) 2022-01-12 10:47:40 | INFO | main | org.springframework.context.support.PostProcessorRegistrationDelegate$BeanPostProcessorChecker ] Bean 'authorizationAttributeSourceAdvisor' of type [org.apache.shiro.spring.security.interceptor.AuthorizationAttributeSourceAdvisor] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying) 22/01/12 10:47:41 WARN websockets.jsr: UT026010: Buffer pool was not set on WebSocketDeploymentInfo, the default pool will be used 22/01/12 10:47:41 INFO undertow.servlet: Initializing Spring embedded WebApplicationContext 2022-01-12 10:47:41 | INFO | main | org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext ] Root WebApplicationContext: initialization completed in 3713 ms 2022-01-12 10:47:42 | ERROR | main | com.streamxhub.streamx.console.core.service.impl.SqlCompleteServiceImpl ] FstTree require reserved word init fail,{} java.io.FileNotFoundException: class path resource [sql-rev.dict] cannot be resolved to absolute file path because it does not reside in the file system: jar:file:/opt/soft/streamx/streamx1.2.1/streamx-console-service-1.2.1/lib/streamx-console-service-1.2.1.jar!/sql-rev.dict at org.springframework.util.ResourceUtils.getFile(ResourceUtils.java:217) at org.springframework.core.io.AbstractFileResolvingResource.getFile(AbstractFileResolvingResource.java:154) at com.streamxhub.streamx.console.core.service.impl.SqlCompleteServiceImpl$FstTree.(SqlCompleteServiceImpl.java:110) at com.streamxhub.streamx.console.core.service.impl.SqlCompleteServiceImpl.initialize(SqlCompleteServiceImpl.java:58) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleElement.invoke(InitDestroyAnnotationBeanPostProcessor.java:389) at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleMetadata.invokeInitMethods(InitDestroyAnnotationBeanPostProcessor.java:333) at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessBeforeInitialization(InitDestroyAnnotationBeanPostProcessor.java:157) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyBeanPostProcessorsBeforeInitialization(AbstractAutowireCapableBeanFactory.java:415) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1786) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:594) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:516) at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:324) at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234) at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:322) at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202) at org.springframework.beans.factory.config.DependencyDescriptor.resolveCandidate(DependencyDescriptor.java:276) at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1307) at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1227) at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBeanPostProcessor.java:640) at org.springframework.beans.factory.annotation.InjectionMetadata.inject(InjectionMetadata.java:119) at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessProperties(AutowiredAnnotationBeanPostProcessor.java:399) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1420) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:593) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:516) at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:324) at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234) at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:322) at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202) at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:897) at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:879) at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:551) at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:143) at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:758) at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:750) at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:405) at org.springframework.boot.SpringApplication.run(SpringApplication.java:315) at com.streamxhub.streamx.console.StreamXConsole.main(StreamXConsole.java:78) 2022-01-12 10:47:42 | INFO | main | com.streamxhub.streamx.console.core.service.impl.SqlCompleteServiceImpl ] Error while FstTree ini that: class path resource [sql-statistics.dict] cannot be resolved to absolute file path because it does not reside in the file system: jar:file:/opt/soft/streamx/streamx1.2.1/streamx-console-service-1.2.1/lib/streamx-console-service-1.2.1.jar!/sql-statistics.dict 2022-01-12 10:47:42 | INFO | main | org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor ] Initializing ExecutorService 'taskExecutor' 2022-01-12 10:47:42 | INFO | main | org.springframework.boot.autoconfigure.web.servlet.WelcomePageHandlerMapping ] Adding welcome page: class path resource [static/index.html] 2022-01-12 10:47:42 | WARN | main | org.springframework.boot.autoconfigure.freemarker.FreeMarkerAutoConfiguration ] Cannot find template location(s): [classpath:/templates/] (please add some templates, check your FreeMarker configuration, or set spring.freemarker.checkTemplateLocation=false) 2022-01-12 10:47:43 | INFO | main | org.quartz.impl.StdSchedulerFactory ] Using default implementation for ThreadExecutor 2022-01-12 10:47:43 | INFO | main | org.quartz.core.SchedulerSignalerImpl ] Initialized Scheduler Signaller of type: class org.quartz.core.SchedulerSignalerImpl 2022-01-12 10:47:43 | INFO | main | org.quartz.core.QuartzScheduler ] Quartz Scheduler v.2.3.2 created. 2022-01-12 10:47:43 | INFO | main | org.quartz.simpl.RAMJobStore ] RAMJobStore initialized. 2022-01-12 10:47:43 | INFO | main | org.quartz.core.QuartzScheduler ] Scheduler meta-data: Quartz Scheduler (v2.3.2) 'quartzScheduler' with instanceId 'NON_CLUSTERED' Scheduler class: 'org.quartz.core.QuartzScheduler' - running locally. NOT STARTED. Currently in standby mode. Number of jobs executed: 0 Using thread pool 'org.quartz.simpl.SimpleThreadPool' - with 10 threads. Using job-store 'org.quartz.simpl.RAMJobStore' - which does not support persistence. and is not clustered.

2022-01-12 10:47:43 | INFO | main | org.quartz.impl.StdSchedulerFactory ] Quartz scheduler 'quartzScheduler' initialized from an externally provided properties instance. 2022-01-12 10:47:43 | INFO | main | org.quartz.impl.StdSchedulerFactory ] Quartz scheduler version: 2.3.2 2022-01-12 10:47:43 | INFO | main | org.quartz.core.QuartzScheduler ] JobFactory set to: org.springframework.scheduling.quartz.SpringBeanJobFactory@38e46765 2022-01-12 10:47:43 | INFO | main | org.springframework.scheduling.concurrent.ThreadPoolTaskScheduler ] Initializing ExecutorService 'taskScheduler' 22/01/12 10:47:43 INFO websockets.jsr: UT026003: Adding annotated server endpoint class com.streamxhub.streamx.console.core.websocket.WebSocketEndpoint for path /websocket/{id} 22/01/12 10:47:43 INFO io.undertow: starting server: Undertow - 2.1.5.Final 22/01/12 10:47:43 INFO org.xnio: XNIO version 3.8.0.Final 22/01/12 10:47:43 INFO xnio.nio: XNIO NIO Implementation Version 3.8.0.Final 22/01/12 10:47:43 INFO jboss.threads: JBoss Threads version 3.1.0.Final 2022-01-12 10:47:43 | INFO | main | org.springframework.boot.web.embedded.undertow.UndertowWebServer ] Undertow started on port(s) 10000 (http) 2022-01-12 10:47:43 | INFO | main | org.springframework.scheduling.quartz.SchedulerFactoryBean ] Starting Quartz Scheduler now 2022-01-12 10:47:43 | INFO | main | org.quartz.core.QuartzScheduler ] Scheduler quartzScheduler_$_NON_CLUSTERED started. 2022-01-12 10:47:43 | INFO | main | com.streamxhub.streamx.console.StreamXConsole ] Started StreamXConsole in 6.636 seconds (JVM running for 7.098) 2022-01-12 10:47:43 | INFO | main | com.streamxhub.streamx.console.core.runner.EnvInitializer ] initialize system properties: key:streamx.workspace.local, value:~/streamx_workspace 2022-01-12 10:47:43 | INFO | main | com.streamxhub.streamx.console.core.runner.EnvInitializer ] initialize system properties: key:streamx.workspace.remote, value:hdfs:///streamx 2022-01-12 10:47:43 | INFO | main | com.streamxhub.streamx.console.core.runner.EnvInitializer ] initialize system properties: key:streamx.docker.register.image-namespace, value:streamx 2022-01-12 10:47:43 | INFO | main | com.streamxhub.streamx.console.core.runner.EnvInitializer ] initialize system properties: key:HADOOP_USER_NAME, value:hdfs 2022-01-12 10:47:43 | INFO | main | com.streamxhub.streamx.console.core.runner.EnvInitializer ] load plugin:streamx-flink-sqlclient-1.2.1.jar to ~/streamx_workspace/plugins 2022-01-12 10:47:43 | INFO | main | com.streamxhub.streamx.console.core.runner.EnvInitializer ] load plugin:streamx-jvm-profiler-1.2.1.jar to ~/streamx_workspace/plugins 2022-01-12 10:47:43 | INFO | main | com.streamxhub.streamx.console.core.runner.EnvInitializer ] load shims:streamx-flink-shims_flink-1.12-1.2.1.jar to ~/streamx_workspace/shims/flink-1.12 2022-01-12 10:47:43 | INFO | main | com.streamxhub.streamx.console.core.runner.EnvInitializer ] load shims:streamx-flink-shims_flink-1.13-1.2.1.jar to ~/streamx_workspace/shims/flink-1.13 2022-01-12 10:47:43 | INFO | main | com.streamxhub.streamx.console.core.runner.EnvInitializer ] load shims:streamx-flink-shims_flink-1.14-1.2.1.jar to ~/streamx_workspace/shims/flink-1.14

             .+.                                
       _____/ /_________  ____ _____ ___  _  __     
      / ___/ __/ ___/ _ \/ __ `/ __ `__ \| |/_/   
     (__  ) /_/ /  /  __/ /_/ / / / / / />  <       
    /____/\__/_/   \___/\__,_/_/ /_/ /_/_/|_|    
                                          |/        
                                          .         

   WebSite:  http://www.streamxhub.com            
   GitHub :  https://github.com/streamxhub/streamx  
   Gitee  :  https://gitee.com/streamxhub/streamx   
   Ver    :  1.2.1                                  
   Info   :  streamx-console start successful                                 
   Time   :  2022-01-12T10:47:43.467              

22/01/12 10:48:06 INFO undertow.servlet: Initializing Spring DispatcherServlet 'dispatcherServlet' 2022-01-12 10:48:06 | INFO | XNIO-1 task-1 | org.springframework.web.servlet.DispatcherServlet ] Initializing Servlet 'dispatcherServlet' 2022-01-12 10:48:06 | INFO | XNIO-1 task-1 | org.springframework.web.servlet.DispatcherServlet ] Completed initialization in 10 ms 2022-01-12 10:49:48 | INFO | XNIO-1 task-3 | com.streamxhub.streamx.common.util.HadoopUtils ] [StreamX] kerberos login starting.... 2022-01-12 12:00:00 | INFO | scheduling-1 | com.streamxhub.streamx.console.core.task.PackerResourceGCTask ] [streamx-packer] Starting Packer Resource GC Task. 2022-01-12 14:41:36 | INFO | XNIO-1 task-4 | com.streamxhub.streamx.common.util.HadoopUtils ] [StreamX] kerberos login starting.... 2022-01-12 15:21:15 | INFO | XNIO-1 task-3 | com.streamxhub.streamx.common.util.HadoopUtils ] [StreamX] kerberos login starting.... 2022-01-12 15:32:42 | INFO | XNIO-1 task-2 | com.streamxhub.streamx.common.util.HadoopUtils ] [StreamX] kerberos login starting....

Requirement or improvement(诉求 & 改进建议)

phenixmzy commented 2 years ago

HadoopUtils.scala has 2 bug, reusableConf can't be initialization. private[this] lazy val ugi: UserGroupInformation = { ..... kerberosLogin(reusableConf) --> kerberosLogin(hadoopConf) ..... }

kerberos authentication successful~!

wolfboys commented 2 years ago

fixed. see: https://github.com/streamxhub/streamx/pull/704