search

WeBankFinTech / Exchangis

Exchangis is a lightweight,highly extensible data exchange platform that supports data transmission between structured and unstructured heterogeneous data sources
https://github.com/WeBankFinTech/Exchangis.git
Apache License 2.0
448 stars 208 forks source link

[Bug] exchangis任务可以执行也可以正常完成，但是完成后不会停止，一直是进行中 #394

Closed xixinpeng436 closed 1 year ago

xixinpeng436 commented 1 year ago

Search before asking

Exchangis Component

exchangis-job-server

What happened + What you expected to happen

Listening for transport dt_socket at address: 24587 2023-02-24 14:42:26.510 INFO [main] org.apache.linkis.manager.label.builder.factory.LabelBuilderFactoryContext 84 labelBuilderInitRegister - Succeed to register label builder: org.apache.linkis.manager.label.builder.CombinedLabelBuilder 2023-02-24 14:42:26.520 INFO [main] org.apache.linkis.engineconn.launch.EngineConnServer$ 63 main - <<---------------------EngineConnServer Start --------------------->> 2023-02-24 14:42:26.546 INFO [main] org.apache.linkis.common.conf.Configuration$ 85 getLinkisHome - linkisHome is /data/datasphere/linkis/LinkisInstall 2023-02-24 14:42:26.655 INFO [main] org.apache.linkis.engineconn.launch.EngineConnServer$ 145 init - Finished to init engineCreationContext: {"user":"linkis","ticketId":"aa444dfe-60d6-49f0-8ee5-0bd07921c808","labels":[{},{},{}],"options":{"wds.linkis.rm.yarnqueue.instance.max":"30","sqoop.setting.max.memory":"1024","sqoop.setting.max.parallelism":"1","wds.linkis.rm.yarnqueue":"default","wds.linkis.engineconn.sqoop.execution.id":"1840401728390959104","wds.linkis.rm.yarnqueue.memory.max":"100G","label.engineType":"sqoop-1.4.7","wds.linkis.rm.client.core.max":"10","wds.linkis.rm.instance":"10","wds.linkis.engineconn.java.driver.memory":"1G","label.userCreator":"linkis-exchangis","wds.linkis.rm.yarnqueue.cores.max":"150","wds.linkis.rm.client.memory.max":"10G","label.engineConnMode":"once","user":"linkis","ticketId":"aa444dfe-60d6-49f0-8ee5-0bd07921c808","onceExecutorContent":"resource_036d2f456a9-490a-4008-b00a-b8fbd3a695acv000001"},"emInstance":{"org$apache$linkis$common$ServiceInstance$$applicationName":"linkis-cg-engineconnmanager","org$apache$linkis$common$ServiceInstance$$instance":"192.168.144.9:9102"},"executorId":0,"args":["--engineconn-conf","sqoop.setting.max.memory\u003d1024","--engineconn-conf","wds.linkis.engineconn.sqoop.execution.id\u003d1840401728390959104","--engineconn-conf","wds.linkis.rm.instance\u003d10","--engineconn-conf","label.userCreator\u003dlinkis-exchangis","--engineconn-conf","ticketId\u003daa444dfe-60d6-49f0-8ee5-0bd07921c808","--engineconn-conf","sqoop.setting.max.parallelism\u003d1","--engineconn-conf","wds.linkis.engineconn.java.driver.memory\u003d1G","--engineconn-conf","wds.linkis.rm.yarnqueue.memory.max\u003d100G","--engineconn-conf","label.engineConnMode\u003donce","--engineconn-conf","label.engineType\u003dsqoop-1.4.7","--engineconn-conf","wds.linkis.rm.yarnqueue.instance.max\u003d30","--engineconn-conf","wds.linkis.rm.client.memory.max\u003d10G","--engineconn-conf","onceExecutorContent\u003dresource_036d2f456a9-490a-4008-b00a-b8fbd3a695acv000001","--engineconn-conf","wds.linkis.rm.client.core.max\u003d10","--engineconn-conf","wds.linkis.rm.yarnqueue.cores.max\u003d150","--engineconn-conf","user\u003dlinkis","--engineconn-conf","wds.linkis.rm.yarnqueue\u003ddefault","--spring-conf","eureka.client.serviceUrl.defaultZone\u003dhttp://192.168.144.9:9600/eureka/","--spring-conf","logging.config\u003dclasspath:log4j2.xml","--spring-conf","eureka.instance.prefer-ip-address\u003dtrue","--spring-conf","spring.profiles.active\u003dengineconn","--spring-conf","eureka.instance.instance-id\u003d${spring.cloud.client.ip-address}:${spring.application.name}:${server.port}","--spring-conf","server.port\u003d33821","--spring-conf","spring.application.name\u003dlinkis-cg-engineconn"]} 2023-02-24 14:42:26.656 INFO [main] org.apache.linkis.engineconn.launch.EngineConnServer$ 75 main - Finished to create EngineCreationContext, EngineCreationContext content: {"user":"linkis","ticketId":"aa444dfe-60d6-49f0-8ee5-0bd07921c808","labels":[{},{},{}],"options":{"wds.linkis.rm.yarnqueue.instance.max":"30","sqoop.setting.max.memory":"1024","sqoop.setting.max.parallelism":"1","wds.linkis.rm.yarnqueue":"default","wds.linkis.engineconn.sqoop.execution.id":"1840401728390959104","wds.linkis.rm.yarnqueue.memory.max":"100G","label.engineType":"sqoop-1.4.7","wds.linkis.rm.client.core.max":"10","wds.linkis.rm.instance":"10","wds.linkis.engineconn.java.driver.memory":"1G","label.userCreator":"linkis-exchangis","wds.linkis.rm.yarnqueue.cores.max":"150","wds.linkis.rm.client.memory.max":"10G","label.engineConnMode":"once","user":"linkis","ticketId":"aa444dfe-60d6-49f0-8ee5-0bd07921c808","onceExecutorContent":"resource_036d2f456a9-490a-4008-b00a-b8fbd3a695acv000001"},"emInstance":{"org$apache$linkis$common$ServiceInstance$$applicationName":"linkis-cg-engineconnmanager","org$apache$linkis$common$ServiceInstance$$instance":"192.168.144.9:9102"},"executorId":0,"args":["--engineconn-conf","sqoop.setting.max.memory\u003d1024","--engineconn-conf","wds.linkis.engineconn.sqoop.execution.id\u003d1840401728390959104","--engineconn-conf","wds.linkis.rm.instance\u003d10","--engineconn-conf","label.userCreator\u003dlinkis-exchangis","--engineconn-conf","ticketId\u003daa444dfe-60d6-49f0-8ee5-0bd07921c808","--engineconn-conf","sqoop.setting.max.parallelism\u003d1","--engineconn-conf","wds.linkis.engineconn.java.driver.memory\u003d1G","--engineconn-conf","wds.linkis.rm.yarnqueue.memory.max\u003d100G","--engineconn-conf","label.engineConnMode\u003donce","--engineconn-conf","label.engineType\u003dsqoop-1.4.7","--engineconn-conf","wds.linkis.rm.yarnqueue.instance.max\u003d30","--engineconn-conf","wds.linkis.rm.client.memory.max\u003d10G","--engineconn-conf","onceExecutorContent\u003dresource_036d2f456a9-490a-4008-b00a-b8fbd3a695acv000001","--engineconn-conf","wds.linkis.rm.client.core.max\u003d10","--engineconn-conf","wds.linkis.rm.yarnqueue.cores.max\u003d150","--engineconn-conf","user\u003dlinkis","--engineconn-conf","wds.linkis.rm.yarnqueue\u003ddefault","--spring-conf","eureka.client.serviceUrl.defaultZone\u003dhttp://192.168.144.9:9600/eureka/","--spring-conf","logging.config\u003dclasspath:log4j2.xml","--spring-conf","eureka.instance.prefer-ip-address\u003dtrue","--spring-conf","spring.profiles.active\u003dengineconn","--spring-conf","eureka.instance.instance-id\u003d${spring.cloud.client.ip-address}:${spring.application.name}:${server.port}","--spring-conf","server.port\u003d33821","--spring-conf","spring.application.name\u003dlinkis-cg-engineconn"]} 2023-02-24 14:42:26.662 INFO [main] org.apache.linkis.engineconn.computation.executor.hook.ComputationEngineConnHook 45 beforeCreateEngineConn - Spring is enabled, now try to start SpringBoot. 2023-02-24 14:42:26.663 INFO [main] org.apache.linkis.engineconn.computation.executor.hook.ComputationEngineConnHook 46 beforeCreateEngineConn - <--------------------Start SpringBoot App--------------------> 2023-02-24 14:42:27.776 INFO [main] org.apache.linkis.engineconn.acessible.executor.log.SendAppender 169 info - SendAppender init success

. _ _ /\ / '_ () \ \ \ \ ( ( )\ | ' | '| | ' \/ ` | \ \ \ \ \/ _)| |)| | | | | || (| | ) ) ) ) ' |__| .|| ||| |\, | / / / / =========|_|==============|__/=//// :: Spring Boot :: (v2.3.12.RELEASE)

2023-02-24 14:42:27.960 INFO [main] org.apache.linkis.DataWorkCloudApplication 652 logStartupProfileInfo - The following profiles are active: engineconn 2023-02-24 14:42:28.020 INFO [main] org.apache.linkis.DataWorkCloudApplication 94 onApplicationEvent - add config from config server... 2023-02-24 14:42:28.020 INFO [main] org.apache.linkis.DataWorkCloudApplication 99 onApplicationEvent - initialize DataWorkCloud spring application... 2023-02-24 14:42:28.023 INFO [main] org.apache.linkis.DataWorkCloudApplication 158 initDWCApplication - using ip address replace hostname,beacause eureka.instance.prefer-ip-address:true 2023-02-24 14:42:30.303 WARN [main] com.netflix.config.sources.URLConfigurationSource 126 - No URLs will be polled as dynamic configuration sources. 2023-02-24 14:42:30.304 INFO [main] com.netflix.config.sources.URLConfigurationSource 127 - To enable URLs as dynamic configuration sources, define System property archaius.configurationSource.additionalUrls or make config.properties available on classpath. 2023-02-24 14:42:30.320 INFO [main] com.netflix.config.DynamicPropertyFactory 281 getInstance - DynamicPropertyFactory is initialized with configuration sources: com.netflix.config.ConcurrentCompositeConfiguration@2dacaa7f 2023-02-24 14:42:30.563 INFO [main] org.apache.linkis.engineconn.executor.listener.EngineConnAsyncListenerBus 40 addListener - EngineConn-Asyn-Thread-ListenerBus add a new listener => class org.apache.linkis.engineconn.acessible.executor.service.DefaultExecutorHeartbeatService 2023-02-24 14:42:30.565 INFO [main] org.apache.linkis.engineconn.executor.listener.EngineConnAsyncListenerBus 40 addListener - EngineConn-Asyn-Thread-ListenerBus add a new listener => class org.apache.linkis.engineconn.acessible.executor.service.DefaultAccessibleService 2023-02-24 14:42:30.618 INFO [main] org.apache.linkis.engineconn.executor.listener.EngineConnAsyncListenerBus 40 addListener - EngineConn-Asyn-Thread-ListenerBus add a new listener => class org.apache.linkis.engineconn.acessible.executor.service.EngineConnTimedLockService 2023-02-24 14:42:30.623 INFO [main] org.apache.linkis.engineconn.executor.listener.EngineConnSyncListenerBus 40 addListener - org.apache.linkis.engineconn.executor.listener.EngineConnSyncListenerBus@b632594 add a new listener => class org.apache.linkis.engineconn.computation.executor.service.TaskExecutionServiceImpl 2023-02-24 14:42:30.774 INFO [main] org.apache.linkis.engineconn.executor.listener.EngineConnSyncListenerBus 40 addListener - org.apache.linkis.engineconn.executor.listener.EngineConnSyncListenerBus@b632594 add a new listener => class org.apache.linkis.engineconn.computation.executor.upstream.service.ECTaskEntranceMonitorService 2023-02-24 14:42:30.774 INFO [main] org.apache.linkis.engineconn.computation.executor.upstream.ECTaskEntranceMonitor 69 start - started upstream monitor 2023-02-24 14:42:31.070 INFO [main] org.apache.linkis.rpc.RPCReceiveRestful 92 initListenerBus - init all receiverChoosers in spring beans, list => List(org.apache.linkis.rpc.CommonReceiverChooser@5db05003, org.apache.linkis.rpc.MessageReceiverChooser@32200389) 2023-02-24 14:42:31.074 INFO [main] org.apache.linkis.rpc.RPCReceiveRestful 97 initListenerBus - init all receiverSenderBuilders in spring beans, list => List(org.apache.linkis.rpc.CommonReceiverSenderBuilder@5253aa21) 2023-02-24 14:42:31.076 INFO [main] org.apache.linkis.rpc.RPCReceiveRestful 106 initListenerBus - init RPCReceiverListenerBus with queueSize 5000 and consumeThreadSize 400. 2023-02-24 14:42:31.077 INFO [main] org.apache.linkis.rpc.AsynRPCMessageBus 40 addListener - RPC-Receiver-Asyn-Thread-ListenerBus add a new listener => class org.apache.linkis.rpc.RPCReceiveRestful$$anon$1 2023-02-24 14:42:32.637 INFO [main] springfox.documentation.spring.web.WebMvcPropertySourcedRequestMappingHandlerMapping 69 initHandlerMethods - Mapped URL path [/v2/api-docs] onto method [springfox.documentation.swagger2.web.Swagger2ControllerWebMvc#getDocumentation(String, HttpServletRequest)] 2023-02-24 14:42:32.676 WARN [main] com.netflix.config.sources.URLConfigurationSource 126 - No URLs will be polled as dynamic configuration sources. 2023-02-24 14:42:32.677 INFO [main] com.netflix.config.sources.URLConfigurationSource 127 - To enable URLs as dynamic configuration sources, define System property archaius.configurationSource.additionalUrls or make config.properties available on classpath. 2023-02-24 14:42:33.275 INFO [main] springfox.documentation.spring.web.plugins.DocumentationPluginsBootstrapper 93 start - Documentation plugins bootstrapped 2023-02-24 14:42:33.280 INFO [main] springfox.documentation.spring.web.plugins.DocumentationPluginsBootstrapper 79 bootstrapDocumentationPlugins - Found 1 custom documentation plugin(s) 2023-02-24 14:42:33.321 INFO [main] springfox.documentation.spring.web.scanners.ApiListingReferenceScanner 44 scan - Scanning for api listing references 2023-02-24 14:42:33.432 INFO [main] org.apache.linkis.rpc.conf.RPCSpringConfiguration$$EnhancerBySpringCGLIB$$9755108e 68 completeInitialize - DataWorkCloud RPC need register RPCReceiveRestful, now add it to configuration. 2023-02-24 14:42:33.432 INFO [main] org.apache.linkis.DataWorkCloudApplication 94 onApplicationEvent - add config from config server... 2023-02-24 14:42:33.432 INFO [main] org.apache.linkis.DataWorkCloudApplication 99 onApplicationEvent - initialize DataWorkCloud spring application... 2023-02-24 14:42:33.433 INFO [main] org.apache.linkis.DataWorkCloudApplication 158 initDWCApplication - using ip address replace hostname,beacause eureka.instance.prefer-ip-address:true 2023-02-24 14:42:33.454 INFO [main] org.apache.linkis.DataWorkCloudApplication 61 logStarted - Started DataWorkCloudApplication in 6.577 seconds (JVM running for 9.02) 2023-02-24 14:42:33.645 INFO [main] org.apache.linkis.rpc.AsynRPCMessageBus 40 addListener - RPC-Sender-Asyn-Thread-ListenerBus add a new listener => class org.apache.linkis.rpc.BaseRPCSender$$anon$1 2023-02-24 14:42:33.904 INFO [main] com.netflix.config.ChainedDynamicProperty 115 checkAndFlip - Flipping property: linkis-cg-engineconnmanager.ribbon.ActiveConnectionsLimit to use NEXT property: niws.loadbalancer.availabilityFilteringRule.activeConnectionsLimit = 2147483647 2023-02-24 14:42:33.919 INFO [main] com.netflix.util.concurrent.ShutdownEnabledTimer 58 - Shutdown hook installed for: NFLoadBalancer-PingTimer-linkis-cg-engineconnmanager 2023-02-24 14:42:33.924 INFO [main] com.netflix.loadbalancer.BaseLoadBalancer 197 initWithConfig - Client: linkis-cg-engineconnmanager instantiated a LoadBalancer: DynamicServerListLoadBalancer:{NFLoadBalancer:name=linkis-cg-engineconnmanager,current list of Servers=[],Load balancer stats=Zone stats: {},Server stats: []}ServerList:null 2023-02-24 14:42:33.928 INFO [main] com.netflix.loadbalancer.DynamicServerListLoadBalancer 222 enableAndInitLearnNewServersFeature - Using serverListUpdater PollingServerListUpdater 2023-02-24 14:42:34.012 INFO [main] com.netflix.config.ChainedDynamicProperty 115 checkAndFlip - Flipping property: linkis-cg-engineconnmanager.ribbon.ActiveConnectionsLimit to use NEXT property: niws.loadbalancer.availabilityFilteringRule.activeConnectionsLimit = 2147483647 2023-02-24 14:42:34.015 INFO [main] com.netflix.loadbalancer.DynamicServerListLoadBalancer 150 restOfInit - DynamicServerListLoadBalancer for client linkis-cg-engineconnmanager initialized: DynamicServerListLoadBalancer:{NFLoadBalancer:name=linkis-cg-engineconnmanager,current list of Servers=[192.168.144.9:9102],Load balancer stats=Zone stats: {defaultzone=[Zone:defaultzone; Instance count:1; Active connections count: 0; Circuit breaker tripped count: 0; Active connections per server: 0.0;] },Server stats: [[Server:192.168.144.9:9102; Zone:defaultZone; Total Requests:0; Successive connection failure:0; Total blackout seconds:0; Last connection made:Thu Jan 01 08:00:00 CST 1970; First connection made: Thu Jan 01 08:00:00 CST 1970; Active Connections:0; total failure count in last (1000) msecs:0; average resp time:0.0; 90 percentile resp time:0.0; 95 percentile resp time:0.0; min resp time:0.0; max resp time:0.0; stddev resp time:0.0] ]}ServerList:org.springframework.cloud.netflix.ribbon.eureka.DomainExtractingServerList@65c3afc0 2023-02-24 14:42:34.094 INFO [main] org.apache.linkis.engineconn.computation.executor.hook.ComputationEngineConnHook 64 beforeCreateEngineConn - <--------------------SpringBoot App init succeed--------------------> 2023-02-24 14:42:34.095 INFO [main] org.apache.linkis.engineconn.launch.EngineConnServer$ 80 main - Finished to execute hook of beforeCreateEngineConn. 2023-02-24 14:42:34.113 INFO [main] org.apache.linkis.engineconn.launch.EngineConnServer$ 83 main - Finished to create sqoop EngineConn. 2023-02-24 14:42:34.114 INFO [main] org.apache.linkis.engineconn.launch.EngineConnServer$ 87 main - Finished to execute all hooks of beforeExecutionExecute. 2023-02-24 14:42:34.129 INFO [main] org.apache.linkis.engineconn.core.execution.EngineConnExecution$ 30 - The list of EngineConnExecution: List(org.apache.linkis.engineconn.once.executor.execution.OnceExecutorManagerEngineConnExecution@4a6eaf83, org.apache.linkis.engineconn.computation.executor.execute.ComputationExecutorManagerEngineConnExecution@4d3f7e6a, org.apache.linkis.engineconn.acessible.executor.execution.AccessibleEngineConnExecution@19db46c3, org.apache.linkis.engineconn.once.executor.execution.OnceEngineConnExecution@4feb580, org.apache.linkis.engineconn.computation.executor.execute.ComputationEngineConnExecution@3c9c90f8) 2023-02-24 14:42:34.130 INFO [main] org.apache.linkis.engineconn.launch.EngineConnServer$ 158 apply - Ready to execute OnceExecutorManagerEngineConnExecution. 2023-02-24 14:42:34.132 INFO [main] org.apache.linkis.engineconn.launch.EngineConnServer$ 158 apply - Ready to execute ComputationExecutorManagerEngineConnExecution. 2023-02-24 14:42:34.133 INFO [main] org.apache.linkis.engineconn.launch.EngineConnServer$ 158 apply - Ready to execute AccessibleEngineConnExecution. 2023-02-24 14:42:34.134 INFO [main] org.apache.linkis.engineconn.core.executor.ExecutorManager$ 248 init - Try to use org.apache.linkis.engineconn.once.executor.creation.OnceExecutorManagerImpl to instance a ExecutorManager. 2023-02-24 14:42:34.147 INFO [main] org.apache.linkis.engineconn.once.executor.creation.OnceExecutorManagerImpl 98 tryCreateExecutor - Try to create a executor with labels List([key: userCreator, value: {"creator":"exchangis","user":"linkis"}, str: linkis-exchangis], [key: engineConnMode, value: {"engineConnMode":"once"}, str: once], [key: engineType, value: {"engineType":"sqoop","version":"1.4.7"}, str: sqoop-1.4.7]). 2023-02-24 14:42:34.150 INFO [main] org.apache.linkis.engineconn.once.executor.creation.OnceExecutorManagerImpl 113 apply - No LabelExecutorFactory matched, use DefaultExecutorFactory to create executor. 2023-02-24 14:42:34.159 INFO [main] org.apache.linkis.engineconn.once.executor.creation.OnceExecutorManagerImpl 122 tryCreateExecutor - Finished to create SqoopOnceCodeExecutor(SqoopOnceApp_0) with labels List([key: userCreator, value: {"creator":"exchangis","user":"linkis"}, str: linkis-exchangis], [key: engineConnMode, value: {"engineConnMode":"once"}, str: once], [key: engineType, value: {"engineType":"sqoop","version":"1.4.7"}, str: sqoop-1.4.7]). 2023-02-24 14:42:34.174 INFO [main] org.apache.linkis.common.conf.Configuration$ 74 getGateWayURL - gatewayUrl is http://192.168.144.9:9001 2023-02-24 14:42:34.191 INFO [main] org.apache.linkis.engineconnplugin.sqoop.executor.SqoopOnceCodeExecutor 55 transition - Waitiing lock release, to change status Starting=>Running. 2023-02-24 14:42:34.191 INFO [main] org.apache.linkis.engineconnplugin.sqoop.executor.SqoopOnceCodeExecutor 57 transition - Finished wait lock release, to change status Starting=>Running. 2023-02-24 14:42:34.191 INFO [main] org.apache.linkis.engineconnplugin.sqoop.executor.SqoopOnceCodeExecutor 60 transition - org.apache.linkis.engineconnplugin.sqoop.executor.SqoopOnceCodeExecutor@669427d changed status Starting => Running. 2023-02-24 14:42:34.192 INFO [main] org.apache.linkis.engineconn.once.executor.creation.OnceExecutorManagerImpl 134 tryCreateExecutor - Finished to init SqoopOnceCodeExecutor(SqoopOnceApp_0). 2023-02-24 14:42:34.193 INFO [main] org.apache.linkis.engineconn.acessible.executor.execution.AccessibleEngineConnExecution 66 execute - Created a report executor SqoopOnceCodeExecutor(SqoopOnceApp_0). 2023-02-24 14:42:34.194 INFO [EngineConn-Asyn-Thread-Thread-0] org.apache.linkis.engineconn.executor.listener.EngineConnAsyncListenerBus 297 run - EngineConn-Asyn-Thread-Thread-0 begin. 2023-02-24 14:42:34.265 INFO [main] com.netflix.config.ChainedDynamicProperty 115 checkAndFlip - Flipping property: linkis-cg-linkismanager.ribbon.ActiveConnectionsLimit to use NEXT property: niws.loadbalancer.availabilityFilteringRule.activeConnectionsLimit = 2147483647 2023-02-24 14:42:34.266 INFO [main] com.netflix.util.concurrent.ShutdownEnabledTimer 58 - Shutdown hook installed for: NFLoadBalancer-PingTimer-linkis-cg-linkismanager 2023-02-24 14:42:34.267 INFO [main] com.netflix.loadbalancer.BaseLoadBalancer 197 initWithConfig - Client: linkis-cg-linkismanager instantiated a LoadBalancer: DynamicServerListLoadBalancer:{NFLoadBalancer:name=linkis-cg-linkismanager,current list of Servers=[],Load balancer stats=Zone stats: {},Server stats: []}ServerList:null 2023-02-24 14:42:34.268 INFO [main] com.netflix.loadbalancer.DynamicServerListLoadBalancer 222 enableAndInitLearnNewServersFeature - Using serverListUpdater PollingServerListUpdater 2023-02-24 14:42:34.270 INFO [main] com.netflix.config.ChainedDynamicProperty 115 checkAndFlip - Flipping property: linkis-cg-linkismanager.ribbon.ActiveConnectionsLimit to use NEXT property: niws.loadbalancer.availabilityFilteringRule.activeConnectionsLimit = 2147483647 2023-02-24 14:42:34.271 INFO [main] com.netflix.loadbalancer.DynamicServerListLoadBalancer 150 restOfInit - DynamicServerListLoadBalancer for client linkis-cg-linkismanager initialized: DynamicServerListLoadBalancer:{NFLoadBalancer:name=linkis-cg-linkismanager,current list of Servers=[192.168.144.9:9101],Load balancer stats=Zone stats: {defaultzone=[Zone:defaultzone; Instance count:1; Active connections count: 0; Circuit breaker tripped count: 0; Active connections per server: 0.0;] },Server stats: [[Server:192.168.144.9:9101; Zone:defaultZone; Total Requests:0; Successive connection failure:0; Total blackout seconds:0; Last connection made:Thu Jan 01 08:00:00 CST 1970; First connection made: Thu Jan 01 08:00:00 CST 1970; Active Connections:0; total failure count in last (1000) msecs:0; average resp time:0.0; 90 percentile resp time:0.0; 95 percentile resp time:0.0; min resp time:0.0; max resp time:0.0; stddev resp time:0.0] ]}ServerList:org.springframework.cloud.netflix.ribbon.eureka.DomainExtractingServerList@2f9d540e 2023-02-24 14:42:34.304 INFO [main] org.apache.linkis.engineconn.acessible.executor.execution.AccessibleEngineConnExecution 194 reportUsedResource - In the first time, report usedResources to LinkisManager succeed. 2023-02-24 14:42:34.305 INFO [main] org.apache.linkis.engineconn.acessible.executor.service.DefaultManagerService 57 labelReport - engineType labels is empty, Not reported 2023-02-24 14:42:34.305 INFO [main] org.apache.linkis.engineconn.acessible.executor.execution.AccessibleEngineConnExecution 202 reportLabel - In the first time, report all labels to LinkisManager succeed. 2023-02-24 14:42:34.305 INFO [main] org.apache.linkis.engineconn.acessible.executor.execution.AccessibleEngineConnExecution 85 executorStatusChecker - executorStatusChecker created, maxFreeTimeMills is 1800000 2023-02-24 14:42:34.306 INFO [main] org.apache.linkis.engineconn.launch.EngineConnServer$ 154 apply - Ready to execute OnceEngineConnExecution. 2023-02-24 14:42:34.307 INFO [EngineConn-Asyn-Thread-Thread-0] org.apache.linkis.engineconn.acessible.executor.service.DefaultManagerService 78 heartbeatReport - success to send engine heartbeat report to 192.168.144.9:9101,status:Running,msg:null 2023-02-24 14:42:34.307 INFO [main] org.apache.linkis.engineconn.once.executor.execution.OnceEngineConnExecution 37 apply - EngineConnMode is once. 2023-02-24 14:42:34.307 WARN [main] org.apache.linkis.engineconn.once.executor.execution.OnceEngineConnExecution 50 execute - org.apache.linkis.engineconn.once.executor.execution.OnceEngineConnExecution is enabled, now step into it's execution. 2023-02-24 14:42:34.411 INFO [main] org.apache.linkis.httpclient.dws.DWSHttpClient 156 org$apache$linkis$httpclient$AbstractHttpClient$$addAttempt$1 - invoke http://192.168.144.9:9001/api/rest_j/v1/bml/download?resourceId=d2f456a9-490a-4008-b00a-b8fbd3a695ac&version=v000001 taken: 89 2023-02-24 14:42:34.448 INFO [Linkis-Default-Scheduler-Thread-1] org.apache.linkis.engineconnplugin.sqoop.executor.SqoopOnceCodeExecutor 68 run - Try to execute params.{sqoop.args.password=****, sqoop.args.driver=com.mysql.jdbc.Driver, sqoop.args.null.string=\N, sqoop.args.columns=id,name, sqoop.args.table=test_user, sqoop.mode=import, sqoop.args.num.mappers=1, sqoop.args.hive.table=user_test, sqoop.args.delete.target.dir=, sqoop.args.username=root, sqoop.args.hive.import=, sqoop.args.fields.terminated.by=,, sqoop.args.null.non.string=\N, sqoop.args.connect=jdbc:mysql://192.168.144.19:3306/linkis1_3_0, sqoop.args.hive.database=test} 2023-02-24 14:42:34.449 INFO [main] org.apache.linkis.engineconnplugin.sqoop.executor.SqoopOnceCodeExecutor 55 transition - Waitiing lock release, to change status Running=>Busy. 2023-02-24 14:42:34.449 INFO [main] org.apache.linkis.engineconnplugin.sqoop.executor.SqoopOnceCodeExecutor 57 transition - Finished wait lock release, to change status Running=>Busy. 2023-02-24 14:42:34.449 INFO [main] org.apache.linkis.engineconnplugin.sqoop.executor.SqoopOnceCodeExecutor 60 transition - org.apache.linkis.engineconnplugin.sqoop.executor.SqoopOnceCodeExecutor@669427d changed status Running => Busy. 2023-02-24 14:42:34.451 INFO [main] org.apache.linkis.engineconn.launch.EngineConnServer$ 98 main - Finished to execute executions. 2023-02-24 14:42:34.451 INFO [main] org.apache.linkis.engineconn.launch.EngineConnServer$ 102 main - Finished to execute hook of afterExecutionExecute 2023-02-24 14:42:34.453 INFO [main] org.apache.linkis.engineconn.callback.service.EngineConnAfterStartCallback 49 callback - protocol will send to em: EngineConnStatusCallback(ServiceInstance(linkis-cg-engineconn, 192.168.144.9:33821),aa444dfe-60d6-49f0-8ee5-0bd07921c808,Unlock,success) 2023-02-24 14:42:34.468 INFO [EngineConn-Asyn-Thread-Thread-0] org.apache.linkis.engineconn.acessible.executor.service.DefaultManagerService 78 heartbeatReport - success to send engine heartbeat report to 192.168.144.9:9101,status:Busy,msg:null 2023-02-24 14:42:34.471 WARN [main] org.apache.linkis.engineconn.computation.executor.hook.ComputationEngineConnHook 121 afterEngineServerStartSuccess - EngineConnServer start succeed! 2023-02-24 14:42:34.471 INFO [main] org.apache.linkis.engineconnplugin.sqoop.executor.SqoopOnceCodeExecutor 55 transition - Waitiing lock release, to change status Busy=>Running. 2023-02-24 14:42:34.471 INFO [main] org.apache.linkis.engineconnplugin.sqoop.executor.SqoopOnceCodeExecutor 57 transition - Finished wait lock release, to change status Busy=>Running. 2023-02-24 14:42:34.471 INFO [main] org.apache.linkis.engineconnplugin.sqoop.executor.SqoopOnceCodeExecutor 60 transition - org.apache.linkis.engineconnplugin.sqoop.executor.SqoopOnceCodeExecutor@669427d changed status Busy => Running. 2023-02-24 14:42:34.480 INFO [EngineConn-Asyn-Thread-Thread-0] org.apache.linkis.engineconn.acessible.executor.service.DefaultManagerService 78 heartbeatReport - success to send engine heartbeat report to 192.168.144.9:9101,status:Running,msg:null 2023-02-24 14:42:34.592 WARN [Linkis-Default-Scheduler-Thread-1] org.apache.sqoop.tool.SqoopTool 176 loadPluginsFromConfDir - $SQOOP_CONF_DIR has not been set in the environment. Cannot check for additional configuration. 2023-02-24 14:42:34.648 WARN [Linkis-Default-Scheduler-Thread-1] org.apache.sqoop.tool.BaseSqoopTool 1203 applyCredentialsOptions - Setting your password on the command-line is insecure. Consider using -P instead. 2023-02-24 14:42:34.688 WARN [Linkis-Default-Scheduler-Thread-1] org.apache.sqoop.ConnFactory 132 getManager - Parameter --driver is set to an explicit driver however appropriate connection manager is not being set (via --connection-manager). Sqoop is going to fall back to org.apache.sqoop.manager.GenericJdbcManager. Please specify explicitly which connection manager should be used next time. 2023-02-24 14:42:34.693 INFO [Linkis-Default-Scheduler-Thread-1] org.apache.sqoop.manager.SqlManager 97 initOptionDefaults - Using default fetchSize of 1000 2023-02-24 14:42:34.693 INFO [Linkis-Default-Scheduler-Thread-1] org.apache.sqoop.tool.CodeGenTool 91 generateORM - Beginning code generation 2023-02-24 14:42:34.944 INFO [Linkis-Default-Scheduler-Thread-1] org.apache.sqoop.manager.SqlManager 775 execute - Executing SQL statement: SELECT t. FROM test_user AS t WHERE 1=0 2023-02-24 14:42:34.965 INFO [Linkis-Default-Scheduler-Thread-1] org.apache.sqoop.orm.CompilationManager 91 findHadoopJars - $HADOOP_MAPRED_HOME is not set 2023-02-24 14:42:34.972 INFO [PollingServerListUpdater-0] com.netflix.config.ChainedDynamicProperty 115 checkAndFlip - Flipping property: linkis-cg-engineconnmanager.ribbon.ActiveConnectionsLimit to use NEXT property: niws.loadbalancer.availabilityFilteringRule.activeConnectionsLimit = 2147483647 2023-02-24 14:42:35.270 INFO [PollingServerListUpdater-1] com.netflix.config.ChainedDynamicProperty 115 checkAndFlip - Flipping property: linkis-cg-linkismanager.ribbon.ActiveConnectionsLimit to use NEXT property: niws.loadbalancer.availabilityFilteringRule.activeConnectionsLimit = 2147483647 2023-02-24 14:42:36.103 WARN [Linkis-Default-Scheduler-Thread-1] org.apache.sqoop.orm.CompilationManager 255 compile - Could not rename /tmp/sqoop-linkis/compile/0d0e254f16aafb68bad190a60ec036e3/test_user.java to /home/linkis/./test_user.java. Error: File element in parameter 'null' already exists: '/home/linkis/./test_user.java' 2023-02-24 14:42:36.103 INFO [Linkis-Default-Scheduler-Thread-1] org.apache.sqoop.orm.CompilationManager 363 jar - Writing jar file: /tmp/sqoop-linkis/compile/0d0e254f16aafb68bad190a60ec036e3/test_user.jar 2023-02-24 14:42:36.218 WARN [Linkis-Default-Scheduler-Thread-1] org.apache.hadoop.util.NativeCodeLoader 60 - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-02-24 14:42:36.731 WARN [Linkis-Default-Scheduler-Thread-1] org.apache.hadoop.hdfs.shortcircuit.DomainSocketFactory 116 - The short-circuit local reads feature cannot be used because libhadoop cannot be loaded. 2023-02-24 14:42:36.852 INFO [Linkis-Default-Scheduler-Thread-1] org.apache.sqoop.tool.ImportTool 587 deleteTargetDir - Destination directory test_user deleted. org.apache.linkis.engineconnplugin.sqoop.client.utils.JarLoader@708841bd 2023-02-24 14:42:36.896 INFO [Linkis-Default-Scheduler-Thread-1] org.apache.sqoop.mapreduce.ImportJobBase 238 runImport - Beginning import of test_user 2023-02-24 14:42:36.896 INFO [Linkis-Default-Scheduler-Thread-1] org.apache.hadoop.conf.Configuration.deprecation 1396 logDeprecation - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address 2023-02-24 14:42:36.899 INFO [Linkis-Default-Scheduler-Thread-1] org.apache.hadoop.conf.Configuration.deprecation 1396 logDeprecation - mapred.jar is deprecated. Instead, use mapreduce.job.jar 2023-02-24 14:42:36.908 INFO [Linkis-Default-Scheduler-Thread-1] org.apache.hadoop.conf.Configuration.deprecation 1396 logDeprecation - mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps 2023-02-24 14:42:36.911 INFO [Linkis-Default-Scheduler-Thread-1] org.apache.sqoop.mapreduce.JobBase 332 setJob - Customize JobBase Set The Job 2023-02-24 14:42:37.195 INFO [Linkis-Default-Scheduler-Thread-1] org.apache.hadoop.mapreduce.JobResourceUploader 883 disableErasureCodingForPath - Disabling Erasure Coding for path: /user/linkis/.staging/job_1676346470916_1382 2023-02-24 14:42:38.520 INFO [Linkis-Default-Scheduler-Thread-1] org.apache.sqoop.mapreduce.db.DBInputFormat 192 setTxIsolation - Using read commited transaction isolation 2023-02-24 14:42:38.575 INFO [Linkis-Default-Scheduler-Thread-1] org.apache.hadoop.mapreduce.JobSubmitter 202 submitJobInternal - number of splits:1 2023-02-24 14:42:38.661 INFO [Linkis-Default-Scheduler-Thread-1] org.apache.hadoop.mapreduce.JobSubmitter 298 printTokens - Submitting tokens for job: job_1676346470916_1382 2023-02-24 14:42:38.662 INFO [Linkis-Default-Scheduler-Thread-1] org.apache.hadoop.mapreduce.JobSubmitter 299 printTokens - Executing with tokens: [] 2023-02-24 14:42:38.984 INFO [Linkis-Default-Scheduler-Thread-1] org.apache.hadoop.conf.Configuration 2757 getConfResourceAsInputStream - found resource resource-types.xml at file:/etc/hadoop/3.1.5.0-152/0/resource-types.xml 2023-02-24 14:42:39.266 INFO [Linkis-Default-Scheduler-Thread-1] org.apache.hadoop.mapreduce.Job 1574 submit - The url to track the job: http://ambari01.chinatelecom.cn:8088/proxy/application_1676346470916_1382/ 2023-02-24 14:42:39.266 INFO [Linkis-Default-Scheduler-Thread-1] org.apache.hadoop.mapreduce.Job 1619 monitorAndPrintJob - Running job: job_1676346470916_1382 2023-02-24 14:42:39.449 INFO [Linkis-Default-Scheduler-Thread-3] org.apache.linkis.engineconnplugin.sqoop.executor.SqoopOnceCodeExecutor 108 run - The Sqoop Process In Running 2023-02-24 14:42:44.449 INFO [Linkis-Default-Scheduler-Thread-2] org.apache.linkis.engineconnplugin.sqoop.executor.SqoopOnceCodeExecutor 108 run - The Sqoop Process In Running 2023-02-24 14:42:45.325 INFO [Linkis-Default-Scheduler-Thread-1] org.apache.hadoop.mapreduce.Job 1640 monitorAndPrintJob - Job job_1676346470916_1382 running in uber mode : false 2023-02-24 14:42:45.327 INFO [Linkis-Default-Scheduler-Thread-1] org.apache.hadoop.mapreduce.Job 1647 monitorAndPrintJob - map 0% reduce 0% 2023-02-24 14:42:49.449 INFO [Linkis-Default-Scheduler-Thread-6] org.apache.linkis.engineconnplugin.sqoop.executor.SqoopOnceCodeExecutor 108 run - The Sqoop Process In Running 2023-02-24 14:42:50.367 INFO [Linkis-Default-Scheduler-Thread-1] org.apache.hadoop.mapreduce.Job 1647 monitorAndPrintJob - map 100% reduce 0% 2023-02-24 14:42:50.373 INFO [Linkis-Default-Scheduler-Thread-1] org.apache.hadoop.mapreduce.Job 1658 monitorAndPrintJob - Job job_1676346470916_1382 completed successfully 2023-02-24 14:42:50.440 INFO [Linkis-Default-Scheduler-Thread-1] org.apache.hadoop.mapreduce.Job 1665 monitorAndPrintJob - Counters: 32 File System Counters FILE: Number of bytes read=0 FILE: Number of bytes written=235698 FILE: Number of read operations=0 FILE: Number of large read operations=0 FILE: Number of write operations=0 HDFS: Number of bytes read=85 HDFS: Number of bytes written=216 HDFS: Number of read operations=6 HDFS: Number of large read operations=0 HDFS: Number of write operations=2 Job Counters Launched map tasks=1 Other local map tasks=1 Total time spent by all maps in occupied slots (ms)=2737 Total time spent by all reduces in occupied slots (ms)=0 Total time spent by all map tasks (ms)=2737 Total vcore-milliseconds taken by all map tasks=2737 Total megabyte-milliseconds taken by all map tasks=2802688 Map-Reduce Framework Map input records=12 Map output records=12 Input split bytes=85 Spilled Records=0 Failed Shuffles=0 Merged Map outputs=0 GC time elapsed (ms)=53 CPU time spent (ms)=1320 Physical memory (bytes) snapshot=326078464 Virtual memory (bytes) snapshot=11771236352 Total committed heap usage (bytes)=330825728 Peak Map Physical memory (bytes)=326078464 Peak Map Virtual memory (bytes)=11771236352 File Input Format Counters Bytes Read=0 File Output Format Counters Bytes Written=216 2023-02-24 14:42:50.445 INFO [Linkis-Default-Scheduler-Thread-1] org.apache.sqoop.mapreduce.ImportJobBase 187 runJob - Transferred 216 bytes in 13.5306 seconds (15.9638 bytes/sec) 2023-02-24 14:42:50.447 INFO [Linkis-Default-Scheduler-Thread-1] org.apache.sqoop.mapreduce.ImportJobBase 189 runJob - Retrieved 12 records. 2023-02-24 14:42:50.454 INFO [Linkis-Default-Scheduler-Thread-1] org.apache.sqoop.manager.SqlManager 775 execute - Executing SQL statement: SELECT t.* FROM test_user AS t WHERE 1=0 2023-02-24 14:42:50.458 INFO [Linkis-Default-Scheduler-Thread-1] org.apache.sqoop.hive.HiveImport 169 importTable - Loading uploaded data into Hive 2023-02-24 14:42:51.845 INFO [Thread-124] org.apache.sqoop.hive.HiveImport 83 run - SLF4J: Class path contains multiple SLF4J bindings. 2023-02-24 14:42:51.846 INFO [Thread-124] org.apache.sqoop.hive.HiveImport 83 run - SLF4J: Found binding in [jar:file:/usr/hdp/3.1.5.0-152/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class] 2023-02-24 14:42:51.846 INFO [Thread-124] org.apache.sqoop.hive.HiveImport 83 run - SLF4J: Found binding in [jar:file:/usr/hdp/3.1.5.0-152/hadoop/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class] 2023-02-24 14:42:51.846 INFO [Thread-124] org.apache.sqoop.hive.HiveImport 83 run - SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. 2023-02-24 14:42:51.850 INFO [Thread-124] org.apache.sqoop.hive.HiveImport 83 run - SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory] 2023-02-24 14:42:54.013 INFO [Thread-124] org.apache.sqoop.hive.HiveImport 83 run - Connecting to jdbc:hive2://ambari01.chinatelecom.cn:2181,ambari02.chinatelecom.cn:2181,ambari03.chinatelecom.cn:2181/default;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2 2023-02-24 14:42:54.449 INFO [Linkis-Default-Scheduler-Thread-3] org.apache.linkis.engineconnplugin.sqoop.executor.SqoopOnceCodeExecutor 108 run - The Sqoop Process In Running 2023-02-24 14:42:59.449 INFO [Linkis-Default-Scheduler-Thread-5] org.apache.linkis.engineconnplugin.sqoop.executor.SqoopOnceCodeExecutor 108 run - The Sqoop Process In Running 2023-02-24 14:43:01.693 INFO [qtp1937312693-31] org.apache.linkis.rpc.message.parser.DefaultServiceParser 79 getServiceMethod - public org.apache.linkis.manager.common.protocol.engine.EngineOperateResponse org.apache.linkis.engineconn.acessible.executor.service.DefaultOperateService.executeOperation(org.apache.linkis.manager.common.protocol.engine.EngineOperateRequest) parameter:[arg0] 2023-02-24 14:43:01.695 WARN [qtp1937312693-31] org.apache.linkis.rpc.message.utils.MessageUtils 47 getBean - can not get bean from spring ioc:org.apache.linkis.engineconn.acessible.executor.service.EngineConnConcurrentLockService 2023-02-24 14:43:01.695 INFO [qtp1937312693-31] org.apache.linkis.rpc.message.parser.DefaultServiceParser 79 getServiceMethod - public org.apache.linkis.manager.common.protocol.ResponseEngineLock org.apache.linkis.engineconn.acessible.executor.service.EngineConnTimedLockService.requestLock(org.apache.linkis.manager.common.protocol.RequestEngineLock) parameter:[arg0] 2023-02-24 14:43:01.696 INFO [qtp1937312693-31] org.apache.linkis.rpc.message.parser.DefaultServiceParser 79 getServiceMethod - public org.apache.linkis.manager.common.protocol.ResponseEngineUnlock org.apache.linkis.engineconn.acessible.executor.service.EngineConnTimedLockService.requestUnLock(org.apache.linkis.manager.common.protocol.RequestEngineUnlock) parameter:[arg0] 2023-02-24 14:43:01.697 INFO [qtp1937312693-31] org.apache.linkis.rpc.message.parser.DefaultServiceParser 79 getServiceMethod - public void org.apache.linkis.engineconn.acessible.executor.service.DefaultAccessibleService.dealEngineStopRequest(org.apache.linkis.manager.common.protocol.engine.EngineSuicideRequest,org.apache.linkis.rpc.Sender) parameter:[arg0, arg1] 2023-02-24 14:43:01.698 INFO [qtp1937312693-31] org.apache.linkis.rpc.message.parser.DefaultServiceParser 79 getServiceMethod - public org.apache.linkis.manager.common.protocol.node.ResponseNodeStatus org.apache.linkis.engineconn.acessible.executor.service.DefaultAccessibleService.dealRequestNodeStatus(org.apache.linkis.manager.common.protocol.node.RequestNodeStatus) parameter:[arg0] 2023-02-24 14:43:01.699 INFO [qtp1937312693-31] org.apache.linkis.rpc.message.parser.DefaultServiceParser 79 getServiceMethod - public org.apache.linkis.scheduler.executer.ExecuteResponse org.apache.linkis.engineconn.computation.executor.service.TaskExecutionServiceImpl.execute(org.apache.linkis.governance.common.protocol.task.RequestTask,org.apache.linkis.rpc.Sender) parameter:[arg0, arg1] 2023-02-24 14:43:01.699 INFO [qtp1937312693-31] org.apache.linkis.rpc.message.parser.DefaultServiceParser 79 getServiceMethod - public void org.apache.linkis.engineconn.computation.executor.service.TaskExecutionServiceImpl.dealRequestTaskPause(org.apache.linkis.governance.common.protocol.task.RequestTaskPause) parameter:[arg0] 2023-02-24 14:43:01.699 INFO [qtp1937312693-31] org.apache.linkis.rpc.message.parser.DefaultServiceParser 79 getServiceMethod - public void org.apache.linkis.engineconn.computation.executor.service.TaskExecutionServiceImpl.dealRequestTaskKill(org.apache.linkis.governance.common.protocol.task.RequestTaskKill) parameter:[arg0] 2023-02-24 14:43:01.700 INFO [qtp1937312693-31] org.apache.linkis.rpc.message.parser.DefaultServiceParser 79 getServiceMethod - public void org.apache.linkis.engineconn.computation.executor.service.TaskExecutionServiceImpl.dealRequestTaskResume(org.apache.linkis.governance.common.protocol.task.RequestTaskResume) parameter:[arg0] 2023-02-24 14:43:01.700 INFO [qtp1937312693-31] org.apache.linkis.rpc.message.parser.DefaultServiceParser 79 getServiceMethod - public org.apache.linkis.governance.common.protocol.task.ResponseTaskStatus org.apache.linkis.engineconn.computation.executor.service.TaskExecutionServiceImpl.dealRequestTaskStatus(org.apache.linkis.governance.common.protocol.task.RequestTaskStatus) parameter:[arg0] 2023-02-24 14:43:01.701 INFO [qtp1937312693-31] org.apache.linkis.rpc.message.parser.DefaultServiceParser 79 getServiceMethod - public org.apache.linkis.manager.common.protocol.node.NodeHeartbeatMsg org.apache.linkis.engineconn.acessible.executor.service.DefaultExecutorHeartbeatService.dealNodeHeartbeatRequest(org.apache.linkis.manager.common.protocol.node.NodeHeartbeatRequest) parameter:[arg0] 2023-02-24 14:43:01.719 INFO [qtp1937312693-31] org.apache.linkis.manager.common.operator.OperatorFactoryImpl 53 - Launched operators list => Map(engineConnYarnApplication -> org.apache.linkis.engineconn.acessible.executor.operator.impl.EngineConnApplicationInfoOperator@338680cd, engineConnMetrics -> org.apache.linkis.engineconn.once.executor.operator.OperableOnceEngineConnOperator@28e4038, engineConnDiagnosis -> org.apache.linkis.engineconn.once.executor.operator.OperableOnceEngineConnOperator@28e4038, engineConnProgress -> org.apache.linkis.engineconn.once.executor.operator.OperableOnceEngineConnOperator@28e4038) 2023-02-24 14:43:01.721 INFO [qtp1937312693-31] org.apache.linkis.engineconn.acessible.executor.service.DefaultOperateService 46 executeOperation - Try to execute operator OperableOnceEngineConnOperator with parameters {operator_name=engineConnMetrics}. 2023-02-24 14:43:01.721 INFO [qtp1937312693-26] org.apache.linkis.engineconn.acessible.executor.service.DefaultOperateService 46 executeOperation - Try to execute operator OperableOnceEngineConnOperator with parameters {operator_name=engineConnProgress}. 2023-02-24 14:43:01.831 INFO [qtp1937312693-31] org.apache.hadoop.mapred.ClientServiceDelegate 279 getProxy - Application state is completed. FinalApplicationStatus=SUCCEEDED. Redirecting to job history server 2023-02-24 14:43:01.891 INFO [qtp1937312693-26] org.apache.linkis.engineconnplugin.sqoop.executor.SqoopOnceCodeExecutor 160 getProgressInfo - Progress Info, id: job_1676346470916_1382, total: 1, running: 0, succeed: 1, fail: 0 2023-02-24 14:43:04.449 INFO [Linkis-Default-Scheduler-Thread-8] org.apache.linkis.engineconnplugin.sqoop.executor.SqoopOnceCodeExecutor 108 run - The Sqoop Process In Running 2023-02-24 14:43:04.958 INFO [qtp1937312693-27] org.apache.linkis.engineconn.acessible.executor.service.DefaultOperateService 46 executeOperation - Try to execute operator OperableOnceEngineConnOperator with parameters {operator_name=engineConnMetrics}. 2023-02-24 14:43:06.953 INFO [qtp1937312693-30] org.apache.linkis.engineconn.acessible.executor.service.DefaultOperateService 46 executeOperation - Try to execute operator OperableOnceEngineConnOperator with parameters {operator_name=engineConnProgress}. 2023-02-24 14:43:06.958 INFO [qtp1937312693-30] org.apache.linkis.engineconnplugin.sqoop.executor.SqoopOnceCodeExecutor 160 getProgressInfo - Progress Info, id: job_1676346470916_1382, total: 1, running: 0, succeed: 1, fail: 0 2023-02-24 14:43:07.996 INFO [qtp1937312693-164] org.apache.linkis.engineconn.acessible.executor.service.DefaultOperateService 46 executeOperation - Try to execute operator OperableOnceEngineConnOperator with parameters {operator_name=engineConnMetrics}. 2023-02-24 14:43:09.449 INFO [Linkis-Default-Scheduler-Thread-6] org.apache.linkis.engineconnplugin.sqoop.executor.SqoopOnceCodeExecutor 108 run - The Sqoop Process In Running 2023-02-24 14:43:11.037 INFO [qtp1937312693-30] org.apache.linkis.engineconn.acessible.executor.service.DefaultOperateService 46 executeOperation - Try to execute operator OperableOnceEngineConnOperator with parameters {operator_name=engineConnMetrics}. 2023-02-24 14:43:11.993 INFO [qtp1937312693-164] org.apache.linkis.engineconn.acessible.executor.service.DefaultOperateService 46 executeOperation - Try to execute operator OperableOnceEngineConnOperator with parameters {operator_name=engineConnProgress}. 2023-02-24 14:43:11.996 INFO [qtp1937312693-164] org.apache.linkis.engineconnplugin.sqoop.executor.SqoopOnceCodeExecutor 160 getProgressInfo - Progress Info, id: job_1676346470916_1382, total: 1, running: 0, succeed: 1, fail: 0 2023-02-24 14:43:14.077 INFO [qtp1937312693-30] org.apache.linkis.engineconn.acessible.executor.service.DefaultOperateService 46 executeOperation - Try to execute operator OperableOnceEngineConnOperator with parameters {operator_name=engineConnMetrics}. 2023-02-24 14:43:14.449 INFO [Linkis-Default-Scheduler-Thread-7] org.apache.linkis.engineconnplugin.sqoop.executor.SqoopOnceCodeExecutor 108 run - The Sqoop Process In Running 2023-02-24 14:43:17.037 INFO [qtp1937312693-164] org.apache.linkis.engineconn.acessible.executor.service.DefaultOperateService 46 executeOperation - Try to execute operator OperableOnceEngineConnOperator with parameters {operator_name=engineConnProgress}. 2023-02-24 14:43:17.040 INFO [qtp1937312693-164] org.apache.linkis.engineconnplugin.sqoop.executor.SqoopOnceCodeExecutor 160 getProgressInfo - Progress Info, id: job_1676346470916_1382, total: 1, running: 0, succeed: 1, fail: 0 2023-02-24 14:43:17.117 INFO [qtp1937312693-30] org.apache.linkis.engineconn.acessible.executor.service.DefaultOperateService 46 executeOperation - Try to execute operator OperableOnceEngineConnOperator with parameters {operator_name=engineConnMetrics}. 2023-02-24 14:43:19.449 INFO [Linkis-Default-Scheduler-Thread-10] org.apache.linkis.engineconnplugin.sqoop.executor.SqoopOnceCodeExecutor 108 run - The Sqoop Process In Running 2023-02-24 14:43:20.160 INFO [qtp1937312693-164] org.apache.linkis.engineconn.acessible.executor.service.DefaultOperateService 46 executeOperation - Try to execute operator OperableOnceEngineConnOperator with parameters {operator_name=engineConnMetrics}. 2023-02-24 14:43:22.073 INFO [qtp1937312693-30] org.apache.linkis.engineconn.acessible.executor.service.DefaultOperateService 46 executeOperation - Try to execute operator OperableOnceEngineConnOperator with parameters {operator_name=engineConnProgress}. 2023-02-24 14:43:22.076 INFO [qtp1937312693-30] org.apache.linkis.engineconnplugin.sqoop.executor.SqoopOnceCodeExecutor 160 getProgressInfo - Progress Info, id: job_1676346470916_1382, total: 1, running: 0, succeed: 1, fail: 0 2023-02-24 14:43:23.198 INFO [qtp1937312693-164] org.apache.linkis.engineconn.acessible.executor.service.DefaultOperateService 46 executeOperation - Try to execute operator OperableOnceEngineConnOperator with parameters {operator_name=engineConnMetrics}. 2023-02-24 14:43:24.449 INFO [Linkis-Default-Scheduler-Thread-5] org.apache.linkis.engineconnplugin.sqoop.executor.SqoopOnceCodeExecutor 108 run - The Sqoop Process In Running 2023-02-24 14:43:25.759 INFO [Linkis-Default-Scheduler-Thread-4] org.apache.linkis.engineconn.computation.executor.service.TaskExecutionServiceImpl 618 onLogUpdate - Task not ready, log will be dropped : null 2023-02-24 14:43:26.238 INFO [qtp1937312693-30] org.apache.linkis.engineconn.acessible.executor.service.DefaultOperateService 46 executeOperation - Try to execute operator OperableOnceEngineConnOperator with parameters {operator_name=engineConnMetrics}. 2023-02-24 14:43:27.110 INFO [qtp1937312693-164] org.apache.linkis.engineconn.acessible.executor.service.DefaultOperateService 46 executeOperation - Try to execute operator OperableOnceEngineConnOperator with parameters {operator_name=engineConnProgress}. 2023-02-24 14:43:27.113 INFO [qtp1937312693-164] org.apache.linkis.engineconnplugin.sqoop.executor.SqoopOnceCodeExecutor 160 getProgressInfo - Progress Info, id: job_1676346470916_1382, total: 1, running: 0, succeed: 1, fail: 0 2023-02-24 14:43:29.278 INFO [qtp1937312693-30] org.apache.linkis.engineconn.acessible.executor.service.DefaultOperateService 46 executeOperation - Try to execute operator OperableOnceEngineConnOperator with parameters {operator_name=engineConnMetrics}.

Relevent platform

exchangis1.1.1 +linkis1.3.0 +dss1.1.1+cenos7.6

Reproduction script

总进度(取消) 正在运行 ceshiyx 资源使用 1 vcores CPU使用 1024 MB 内存使用 流量情况source2.33 Records/Ssink核心指标 12 已同步 0 错误记录 0 忽略记录

Anything else

任务可以正常执行，但是没有报错的情况下 不会自动结束，一直是正在运行状态

Are you willing to submit a PR?

xixinpeng436 commented 1 year ago

hive向mysql导数正常，只是在mysql向hive导数有问题