search

apache / dolphinscheduler

Apache DolphinScheduler is the modern data orchestration platform. Agile to create high performance workflow with low-code
https://dolphinscheduler.apache.org/
Apache License 2.0
12.85k stars 4.62k forks source link

[Improvement] Remove t_ds_plugin_define table. #15879

Closed ruanwenjun closed 4 months ago

ruanwenjun commented 6 months ago

Search before asking

Description

DS will write the plugin info into t_ds_plugin_define table, this is not needed, since the plugin will be load into memory, we don't need to write the plugin info into db. Once the plugin is changed, this will cause concurrent problem when update the plugin info.

Are you willing to submit a PR?

Code of Conduct

xiongbl526 commented 6 months ago

The following error occurred when starting the API module and Alter module while deploying the upgrade from 3.1.8 to 3.2.1 using Helm: ![Uploading 9806BF61-6324-4cef-8953-8631B003D208.png…]() ![Uploading 9833A4F9-E0E2-428e-B718-C253E364AF6D.png…]()

xiongbl526 commented 6 months ago

[INFO] 2024-04-25 16:42:28.654 +0800 o.a.d.p.t.a.TaskPluginManager:[63] - Registering task plugin: JAVA - JavaTaskChannelFactory [INFO] 2024-04-25 16:42:28.724 +0800 o.a.d.p.t.a.TaskPluginManager:[68] - Registered task plugin: JAVA - JavaTaskChannelFactory [INFO] 2024-04-25 16:42:28.724 +0800 o.a.d.p.t.a.TaskPluginManager:[63] - Registering task plugin: JUPYTER - JupyterTaskChannelFactory [INFO] 2024-04-25 16:42:28.725 +0800 o.a.d.p.t.a.TaskPluginManager:[68] - Registered task plugin: JUPYTER - JupyterTaskChannelFactory [INFO] 2024-04-25 16:42:28.726 +0800 o.a.d.p.t.a.TaskPluginManager:[63] - Registering task plugin: SPARK - SparkTaskChannelFactory [INFO] 2024-04-25 16:42:28.728 +0800 o.a.d.p.t.a.TaskPluginManager:[68] - Registered task plugin: SPARK - SparkTaskChannelFactory [INFO] 2024-04-25 16:42:28.728 +0800 o.a.d.p.t.a.TaskPluginManager:[63] - Registering task plugin: FLINK_STREAM - FlinkStreamTaskChannelFactory [INFO] 2024-04-25 16:42:28.744 +0800 o.a.d.p.t.a.TaskPluginManager:[68] - Registered task plugin: FLINK_STREAM - FlinkStreamTaskChannelFactory [INFO] 2024-04-25 16:42:28.745 +0800 o.a.d.p.t.a.TaskPluginManager:[63] - Registering task plugin: PYTHON - PythonTaskChannelFactory [INFO] 2024-04-25 16:42:28.752 +0800 o.a.d.p.t.a.TaskPluginManager:[68] - Registered task plugin: PYTHON - PythonTaskChannelFactory [INFO] 2024-04-25 16:42:28.752 +0800 o.a.d.p.t.a.TaskPluginManager:[63] - Registering task plugin: DATASYNC - DatasyncTaskChannelFactory [INFO] 2024-04-25 16:42:28.753 +0800 o.a.d.p.t.a.TaskPluginManager:[68] - Registered task plugin: DATASYNC - DatasyncTaskChannelFactory [INFO] 2024-04-25 16:42:28.753 +0800 o.a.d.p.t.a.TaskPluginManager:[63] - Registering task plugin: DATA_FACTORY - DatafactoryTaskChannelFactory [INFO] 2024-04-25 16:42:28.754 +0800 o.a.d.p.t.a.TaskPluginManager:[68] - Registered task plugin: DATA_FACTORY - DatafactoryTaskChannelFactory [INFO] 2024-04-25 16:42:28.754 +0800 o.a.d.p.t.a.TaskPluginManager:[63] - Registering task plugin: CHUNJUN - ChunJunTaskChannelFactory [INFO] 2024-04-25 16:42:28.755 +0800 o.a.d.p.t.a.TaskPluginManager:[68] - Registered task plugin: CHUNJUN - ChunJunTaskChannelFactory [INFO] 2024-04-25 16:42:28.756 +0800 o.a.d.p.t.a.TaskPluginManager:[63] - Registering task plugin: REMOTESHELL - RemoteShellTaskChannelFactory [INFO] 2024-04-25 16:42:28.761 +0800 o.a.d.p.t.a.TaskPluginManager:[68] - Registered task plugin: REMOTESHELL - RemoteShellTaskChannelFactory [INFO] 2024-04-25 16:42:28.761 +0800 o.a.d.p.t.a.TaskPluginManager:[63] - Registering task plugin: PIGEON - PigeonTaskChannelFactory [INFO] 2024-04-25 16:42:28.763 +0800 o.a.d.p.t.a.TaskPluginManager:[68] - Registered task plugin: PIGEON - PigeonTaskChannelFactory [INFO] 2024-04-25 16:42:28.764 +0800 o.a.d.p.t.a.TaskPluginManager:[63] - Registering task plugin: PROCEDURE - ProcedureTaskChannelFactory [INFO] 2024-04-25 16:42:28.766 +0800 o.a.d.p.t.a.TaskPluginManager:[68] - Registered task plugin: PROCEDURE - ProcedureTaskChannelFactory [INFO] 2024-04-25 16:42:28.766 +0800 o.a.d.p.t.a.TaskPluginManager:[63] - Registering task plugin: SHELL - ShellTaskChannelFactory [INFO] 2024-04-25 16:42:28.822 +0800 o.a.d.p.t.a.TaskPluginManager:[68] - Registered task plugin: SHELL - ShellTaskChannelFactory [INFO] 2024-04-25 16:42:28.822 +0800 o.a.d.p.t.a.TaskPluginManager:[63] - Registering task plugin: PYTORCH - PytorchTaskChannelFactory [INFO] 2024-04-25 16:42:28.824 +0800 o.a.d.p.t.a.TaskPluginManager:[68] - Registered task plugin: PYTORCH - PytorchTaskChannelFactory [INFO] 2024-04-25 16:42:28.824 +0800 o.a.d.p.t.a.TaskPluginManager:[63] - Registering task plugin: SQOOP - SqoopTaskChannelFactory [INFO] 2024-04-25 16:42:28.840 +0800 o.a.d.p.t.a.TaskPluginManager:[68] - Registered task plugin: SQOOP - SqoopTaskChannelFactory [INFO] 2024-04-25 16:42:28.840 +0800 o.a.d.p.t.a.TaskPluginManager:[63] - Registering task plugin: MR - MapReduceTaskChannelFactory [INFO] 2024-04-25 16:42:28.842 +0800 o.a.d.p.t.a.TaskPluginManager:[68] - Registered task plugin: MR - MapReduceTaskChannelFactory [INFO] 2024-04-25 16:42:28.842 +0800 o.a.d.p.t.a.TaskPluginManager:[63] - Registering task plugin: K8S - K8sTaskChannelFactory [INFO] 2024-04-25 16:42:28.844 +0800 o.a.d.p.t.a.TaskPluginManager:[68] - Registered task plugin: K8S - K8sTaskChannelFactory [INFO] 2024-04-25 16:42:28.844 +0800 o.a.d.p.t.a.TaskPluginManager:[63] - Registering task plugin: SEATUNNEL - SeatunnelTaskChannelFactory [INFO] 2024-04-25 16:42:28.846 +0800 o.a.d.p.t.a.TaskPluginManager:[68] - Registered task plugin: SEATUNNEL - SeatunnelTaskChannelFactory [INFO] 2024-04-25 16:42:28.846 +0800 o.a.d.p.t.a.TaskPluginManager:[63] - Registering task plugin: SAGEMAKER - SagemakerTaskChannelFactory [INFO] 2024-04-25 16:42:28.848 +0800 o.a.d.p.t.a.TaskPluginManager:[68] - Registered task plugin: SAGEMAKER - SagemakerTaskChannelFactory [INFO] 2024-04-25 16:42:28.848 +0800 o.a.d.p.t.a.TaskPluginManager:[63] - Registering task plugin: HTTP - HttpTaskChannelFactory [INFO] 2024-04-25 16:42:28.850 +0800 o.a.d.p.t.a.TaskPluginManager:[68] - Registered task plugin: HTTP - HttpTaskChannelFactory [INFO] 2024-04-25 16:42:28.850 +0800 o.a.d.p.t.a.TaskPluginManager:[63] - Registering task plugin: DMS - DmsTaskChannelFactory [INFO] 2024-04-25 16:42:28.860 +0800 o.a.d.p.t.a.TaskPluginManager:[68] - Registered task plugin: DMS - DmsTaskChannelFactory [INFO] 2024-04-25 16:42:28.860 +0800 o.a.d.p.t.a.TaskPluginManager:[63] - Registering task plugin: EMR - EmrTaskChannelFactory [INFO] 2024-04-25 16:42:28.863 +0800 o.a.d.p.t.a.TaskPluginManager:[68] - Registered task plugin: EMR - EmrTaskChannelFactory [INFO] 2024-04-25 16:42:28.863 +0800 o.a.d.p.t.a.TaskPluginManager:[63] - Registering task plugin: DATA_QUALITY - DataQualityTaskChannelFactory [INFO] 2024-04-25 16:42:28.865 +0800 o.a.d.p.t.a.TaskPluginManager:[68] - Registered task plugin: DATA_QUALITY - DataQualityTaskChannelFactory [INFO] 2024-04-25 16:42:28.865 +0800 o.a.d.p.t.a.TaskPluginManager:[63] - Registering task plugin: KUBEFLOW - KubeflowTaskChannelFactory [INFO] 2024-04-25 16:42:28.871 +0800 o.a.d.p.t.a.TaskPluginManager:[68] - Registered task plugin: KUBEFLOW - KubeflowTaskChannelFactory [INFO] 2024-04-25 16:42:28.871 +0800 o.a.d.p.t.a.TaskPluginManager:[63] - Registering task plugin: SQL - SqlTaskChannelFactory [INFO] 2024-04-25 16:42:28.933 +0800 o.a.d.p.t.a.TaskPluginManager:[68] - Registered task plugin: SQL - SqlTaskChannelFactory [INFO] 2024-04-25 16:42:28.933 +0800 o.a.d.p.t.a.TaskPluginManager:[63] - Registering task plugin: DVC - DvcTaskChannelFactory [INFO] 2024-04-25 16:42:28.935 +0800 o.a.d.p.t.a.TaskPluginManager:[68] - Registered task plugin: DVC - DvcTaskChannelFactory [INFO] 2024-04-25 16:42:28.935 +0800 o.a.d.p.t.a.TaskPluginManager:[63] - Registering task plugin: DATAX - DataxTaskChannelFactory [INFO] 2024-04-25 16:42:28.948 +0800 o.a.d.p.t.a.TaskPluginManager:[68] - Registered task plugin: DATAX - DataxTaskChannelFactory [INFO] 2024-04-25 16:42:28.948 +0800 o.a.d.p.t.a.TaskPluginManager:[63] - Registering task plugin: ZEPPELIN - ZeppelinTaskChannelFactory [INFO] 2024-04-25 16:42:28.950 +0800 o.a.d.p.t.a.TaskPluginManager:[68] - Registered task plugin: ZEPPELIN - ZeppelinTaskChannelFactory [INFO] 2024-04-25 16:42:28.950 +0800 o.a.d.p.t.a.TaskPluginManager:[63] - Registering task plugin: DINKY - DinkyTaskChannelFactory [INFO] 2024-04-25 16:42:28.963 +0800 o.a.d.p.t.a.TaskPluginManager:[68] - Registered task plugin: DINKY - DinkyTaskChannelFactory [INFO] 2024-04-25 16:42:28.963 +0800 o.a.d.p.t.a.TaskPluginManager:[63] - Registering task plugin: MLFLOW - MlflowTaskChannelFactory [INFO] 2024-04-25 16:42:29.022 +0800 o.a.d.p.t.a.TaskPluginManager:[68] - Registered task plugin: MLFLOW - MlflowTaskChannelFactory [INFO] 2024-04-25 16:42:29.023 +0800 o.a.d.p.t.a.TaskPluginManager:[63] - Registering task plugin: OPENMLDB - OpenmldbTaskChannelFactory [INFO] 2024-04-25 16:42:29.025 +0800 o.a.d.p.t.a.TaskPluginManager:[68] - Registered task plugin: OPENMLDB - OpenmldbTaskChannelFactory [INFO] 2024-04-25 16:42:29.025 +0800 o.a.d.p.t.a.TaskPluginManager:[63] - Registering task plugin: LINKIS - LinkisTaskChannelFactory [INFO] 2024-04-25 16:42:29.026 +0800 o.a.d.p.t.a.TaskPluginManager:[68] - Registered task plugin: LINKIS - LinkisTaskChannelFactory [INFO] 2024-04-25 16:42:29.026 +0800 o.a.d.p.t.a.TaskPluginManager:[63] - Registering task plugin: HIVECLI - HiveCliTaskChannelFactory [INFO] 2024-04-25 16:42:29.038 +0800 o.a.d.p.t.a.TaskPluginManager:[68] - Registered task plugin: HIVECLI - HiveCliTaskChannelFactory [INFO] 2024-04-25 16:42:29.038 +0800 o.a.d.p.t.a.TaskPluginManager:[63] - Registering task plugin: FLINK - FlinkTaskChannelFactory [INFO] 2024-04-25 16:42:29.039 +0800 o.a.d.p.t.a.TaskPluginManager:[68] - Registered task plugin: FLINK - FlinkTaskChannelFactory [INFO] 2024-04-25 16:42:29.048 +0800 o.a.d.c.u.JSONUtils:[72] - init timezone: sun.util.calendar.ZoneInfo[id="Asia/Shanghai",offset=28800000,dstSavings=0,useDaylight=false,transitions=31,lastRule=null] [ERROR] 2024-04-25 16:42:29.544 +0800 o.a.d.d.PluginDao:[73] - Insert plugin definition error, there may already exist a plugin org.springframework.dao.DuplicateKeyException:

Error updating database. Cause: java.sql.SQLIntegrityConstraintViolationException: Duplicate entry '34' for key 't_ds_plugin_define.PRIMARY'

The error may exist in org/apache/dolphinscheduler/dao/mapper/PluginDefineMapper.java (best guess)

The error may involve org.apache.dolphinscheduler.dao.mapper.PluginDefineMapper.insert-Inline

The error occurred while setting parameters

SQL: INSERT INTO t_ds_plugin_define ( plugin_name, plugin_type, plugin_params, create_time, update_time ) VALUES ( ?, ?, ?, ?, ? )

Cause: java.sql.SQLIntegrityConstraintViolationException: Duplicate entry '34' for key 't_ds_plugin_define.PRIMARY'

; Duplicate entry '34' for key 't_ds_plugin_define.PRIMARY'; nested exception is java.sql.SQLIntegrityConstraintViolationException: Duplicate entry '34' for key 't_ds_plugin_define.PRIMARY' at org.springframework.jdbc.support.SQLErrorCodeSQLExceptionTranslator.doTranslate(SQLErrorCodeSQLExceptionTranslator.java:247) at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:70) at org.mybatis.spring.MyBatisExceptionTranslator.translateExceptionIfPossible(MyBatisExceptionTranslator.java:91) at org.mybatis.spring.SqlSessionTemplate$SqlSessionInterceptor.invoke(SqlSessionTemplate.java:441) at com.sun.proxy.$Proxy151.insert(Unknown Source) at org.mybatis.spring.SqlSessionTemplate.insert(SqlSessionTemplate.java:272) at com.baomidou.mybatisplus.core.override.MybatisMapperMethod.execute(MybatisMapperMethod.java:59) at com.baomidou.mybatisplus.core.override.MybatisMapperProxy$PlainMethodInvoker.invoke(MybatisMapperProxy.java:148) at com.baomidou.mybatisplus.core.override.MybatisMapperProxy.invoke(MybatisMapperProxy.java:89) at com.sun.proxy.$Proxy152.insert(Unknown Source) at org.apache.dolphinscheduler.dao.PluginDao.addOrUpdatePluginDefine(PluginDao.java:64) at org.apache.dolphinscheduler.api.ApiApplicationServer.run(ApiApplicationServer.java:73) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.springframework.context.event.ApplicationListenerMethodAdapter.doInvoke(ApplicationListenerMethodAdapter.java:344) at org.springframework.context.event.ApplicationListenerMethodAdapter.processEvent(ApplicationListenerMethodAdapter.java:229) at org.springframework.context.event.ApplicationListenerMethodAdapter.onApplicationEvent(ApplicationListenerMethodAdapter.java:166) at org.springframework.context.event.SimpleApplicationEventMulticaster.doInvokeListener(SimpleApplicationEventMulticaster.java:176) at org.springframework.context.event.SimpleApplicationEventMulticaster.invokeListener(SimpleApplicationEventMulticaster.java:169) at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:143) at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:421) at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:378) at org.springframework.boot.context.event.EventPublishingRunListener.ready(EventPublishingRunListener.java:114) at org.springframework.boot.SpringApplicationRunListeners.lambda$ready$6(SpringApplicationRunListeners.java:82) at java.util.ArrayList.forEach(ArrayList.java:1259) at org.springframework.boot.SpringApplicationRunListeners.doWithListeners(SpringApplicationRunListeners.java:120) at org.springframework.boot.SpringApplicationRunListeners.doWithListeners(SpringApplicationRunListeners.java:114) at org.springframework.boot.SpringApplicationRunListeners.ready(SpringApplicationRunListeners.java:82) at org.springframework.boot.SpringApplication.run(SpringApplication.java:323) at org.springframework.boot.SpringApplication.run(SpringApplication.java:1306) at org.springframework.boot.SpringApplication.run(SpringApplication.java:1295) at org.apache.dolphinscheduler.api.ApiApplicationServer.main(ApiApplicationServer.java:58) Caused by: java.sql.SQLIntegrityConstraintViolationException: Duplicate entry '34' for key 't_ds_plugindefine.PRIMARY' at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:118) at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:122) at com.mysql.cj.jdbc.ClientPreparedStatement.executeInternal(ClientPreparedStatement.java:916) at com.mysql.cj.jdbc.ClientPreparedStatement.execute(ClientPreparedStatement.java:354) at com.zaxxer.hikari.pool.ProxyPreparedStatement.execute(ProxyPreparedStatement.java:44) at com.zaxxer.hikari.pool.HikariProxyPreparedStatement.execute(HikariProxyPreparedStatement.java) at org.apache.ibatis.executor.statement.PreparedStatementHandler.update(PreparedStatementHandler.java:47) at org.apache.ibatis.executor.statement.RoutingStatementHandler.update(RoutingStatementHandler.java:74) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.ibatis.plugin.Plugin.invoke(Plugin.java:64) at com.sun.proxy.$Proxy818.update(Unknown Source) at org.apache.ibatis.executor.SimpleExecutor.doUpdate(SimpleExecutor.java:50) at org.apache.ibatis.executor.BaseExecutor.update(BaseExecutor.java:117) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.ibatis.plugin.Invocation.proceed(Invocation.java:49) at com.baomidou.mybatisplus.extension.plugins.MybatisPlusInterceptor.intercept(MybatisPlusInterceptor.java:106) at org.apache.ibatis.plugin.Plugin.invoke(Plugin.java:62) at com.sun.proxy.$Proxy817.update(Unknown Source) at org.apache.ibatis.session.defaults.DefaultSqlSession.update(DefaultSqlSession.java:194) at org.apache.ibatis.session.defaults.DefaultSqlSession.insert(DefaultSqlSession.java:181) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.mybatis.spring.SqlSessionTemplate$SqlSessionInterceptor.invoke(SqlSessionTemplate.java:427) ... 30 common frames omitted [ERROR] 2024-04-25 16:42:29.742 +0800 o.s.b.SpringApplication:[824] - Application run failed org.apache.dolphinscheduler.plugin.task.api.TaskPluginException: Failed to insert plugin definition, pluginName: JAVA, pluginType: task at org.apache.dolphinscheduler.dao.PluginDao.addOrUpdatePluginDefine(PluginDao.java:78) at org.apache.dolphinscheduler.api.ApiApplicationServer.run(ApiApplicationServer.java:73) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.springframework.context.event.ApplicationListenerMethodAdapter.doInvoke(ApplicationListenerMethodAdapter.java:344) at org.springframework.context.event.ApplicationListenerMethodAdapter.processEvent(ApplicationListenerMethodAdapter.java:229) at org.springframework.context.event.ApplicationListenerMethodAdapter.onApplicationEvent(ApplicationListenerMethodAdapter.java:166) at org.springframework.context.event.SimpleApplicationEventMulticaster.doInvokeListener(SimpleApplicationEventMulticaster.java:176) at org.springframework.context.event.SimpleApplicationEventMulticaster.invokeListener(SimpleApplicationEventMulticaster.java:169) at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:143) at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:421) at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:378) at org.springframework.boot.context.event.EventPublishingRunListener.ready(EventPublishingRunListener.java:114) at org.springframework.boot.SpringApplicationRunListeners.lambda$ready$6(SpringApplicationRunListeners.java:82) at java.util.ArrayList.forEach(ArrayList.java:1259) at org.springframework.boot.SpringApplicationRunListeners.doWithListeners(SpringApplicationRunListeners.java:120) at org.springframework.boot.SpringApplicationRunListeners.doWithListeners(SpringApplicationRunListeners.java:114) at org.springframework.boot.SpringApplicationRunListeners.ready(SpringApplicationRunListeners.java:82) at org.springframework.boot.SpringApplication.run(SpringApplication.java:323) at org.springframework.boot.SpringApplication.run(SpringApplication.java:1306) at org.springframework.boot.SpringApplication.run(SpringApplication.java:1295) at org.apache.dolphinscheduler.api.ApiApplicationServer.main(ApiApplicationServer.java:58) [INFO] 2024-04-25 16:42:29.830 +0800 o.e.j.s.AbstractConnector:[383] - Stopped ServerConnector@15f11bfb{HTTP/1.1, (http/1.1)}{0.0.0.0:12345} [INFO] 2024-04-25 16:42:29.830 +0800 o.e.j.s.session:[149] - node0 Stopped scavenging [INFO] 2024-04-25 16:42:29.831 +0800 o.e.j.s.h.C.application:[2368] - Destroying Spring FrameworkServlet 'dispatcherServlet' [INFO] 2024-04-25 16:42:29.832 +0800 o.e.j.s.h.ContextHandler:[1159] - Stopped o.s.b.w.e.j.JettyEmbeddedWebAppContext@37393dab{application,/dolphinscheduler,[file:///tmp/jetty-docbase.12345.4442059716409633288/, jar:file:/opt/dolphinscheduler/libs/swagger-ui-4.11.1.jar!/META-INF/resources],STOPPED} [INFO] 2024-04-25 16:42:29.854 +0800 i.k.c.i.c.ReflectorRunnable:[162] - class io.kubernetes.client.openapi.models.V1ConfigMap#Read timeout retry list and watch [ERROR] 2024-04-25 16:42:29.857 +0800 i.k.c.i.c.Controller:[164] - DefaultController#processLoop get interrupted null java.lang.InterruptedException: null at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2048) at io.kubernetes.client.informer.cache.DeltaFIFO.pop(DeltaFIFO.java:318) at io.kubernetes.client.informer.cache.Controller.processLoop(Controller.java:162) at io.kubernetes.client.informer.cache.Controller.run(Controller.java:130) at java.lang.Thread.run(Thread.java:750) [ERROR] 2024-04-25 16:42:29.858 +0800 i.k.c.i.c.ProcessorListener:[96] - processor interrupted: {} java.lang.InterruptedException: null at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2048) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at io.kubernetes.client.informer.cache.ProcessorListener.run(ProcessorListener.java:58) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) [INFO] 2024-04-25 16:42:29.936 +0800 o.q.c.QuartzScheduler:[666] - Scheduler DolphinScheduler$dolphinscheduler321bmp-api-5c69c5dd7-xnc4g1714034522449 shutting down. [INFO] 2024-04-25 16:42:29.936 +0800 o.q.c.QuartzScheduler:[585] - Scheduler DolphinScheduler$dolphinscheduler321bmp-api-5c69c5dd7-xnc4g1714034522449 paused. [INFO] 2024-04-25 16:42:29.937 +0800 o.q.c.QuartzScheduler:[740] - Scheduler DolphinScheduler$_dolphinscheduler321bmp-api-5c69c5dd7-xnc4g1714034522449 shutdown complete. [INFO] 2024-04-25 16:42:29.937 +0800 o.s.s.q.SchedulerFactoryBean:[847] - Shutting down Quartz Scheduler [INFO] 2024-04-25 16:42:29.961 +0800 o.a.c.f.i.CuratorFrameworkImpl:[998] - backgroundOperationsLoop exiting [WARN] 2024-04-25 16:42:30.034 +0800 o.a.z.ClientCnxn:[1286] - An exception was thrown while closing send thread for session 0x3047a08329e003e. org.apache.zookeeper.ClientCnxn$EndOfStreamException: Unable to read additional data from server sessionid 0x3047a08329e003e, likely server has closed socket at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:77) at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1282) [INFO] 2024-04-25 16:42:30.255 +0800 c.z.h.HikariDataSource:[350] - DolphinScheduler - Shutdown initiated... [INFO] 2024-04-25 16:42:30.454 +0800 c.z.h.HikariDataSource:[352] - DolphinScheduler - Shutdown completed.

github-actions[bot] commented 5 months ago

This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.

ruanwenjun commented 5 months ago

@xiongbl526 Could you please provide the whole log? It's seems caused by the plugin upsert concurrently.

github-actions[bot] commented 4 months ago

This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.

github-actions[bot] commented 4 months ago

This issue has been closed because it has not received response for too long time. You could reopen it if you encountered similar problems in the future.