[Bug] [API] Create process response error

gt199899 commented 2 years ago

Search before asking

[X] I had searched in the issues and found no similar issues.

What happened

Hi, there

I follow https://dolphinscheduler.apache.org/en-us/docs/latest/user_doc/guide/installation/docker.html, and build dolphinscheduler with docker-compose, everything is fine until create process.

When i save process, then response error.

API: /dolphinscheduler/projects/4096931718784/process-definition Error resposne:

{"code":10105,"msg":"create process definition error","data":null,"failed":true,"success":false}

And error log in API:

[INFO] 2022-01-06 12:01:13.203 org.apache.dolphinscheduler.api.aspect.AccessLogAspect:[76] - REQUEST TRANCE_ID:1e4ad417-d6b1-4fe1-81eb-74d08f8bcf4c, LOGIN_USER:admin, URI:/dolphinscheduler/projects/4096931718784/process-definition/verify-name, METHOD:GET, HANDLER:org.apache.dolphinscheduler.api.controller.ProcessDefinitionController.verifyProcessDefinitionName, ARGS:{projectCode=4096931718784, name=DAG-1}
[INFO] 2022-01-06 12:01:13.233 org.apache.dolphinscheduler.api.aspect.AccessLogAspect:[76] - REQUEST TRANCE_ID:2c3e916f-30a9-40c8-937f-6e83422260ba, LOGIN_USER:admin, URI:/dolphinscheduler/projects/4096931718784/process-definition, METHOD:POST, HANDLER:org.apache.dolphinscheduler.api.controller.ProcessDefinitionController.createProcessDefinition, ARGS:{projectCode=4096931718784, taskDefinitionJson=[{"code":4097401467904,"name":"S-1","description":"","taskType":"SHELL","taskParams":{"resourceList":[],"localParams":[],"rawScript":"echo \"hi\"","dependence":{},"conditionResult":{"successNode":[],"failedNode":[]},"waitStartTimeout":{},"switchResult":{}},"flag":"YES","taskPriority":"MEDIUM","workerGroup":"default","failRetryTimes":"0","failRetryInterval":"1","timeoutFlag":"CLOSE","timeoutNotifyStrategy":"WARN","timeout":0,"delayTime":"0","environmentCode":-1}], taskRelationJson=[{"name":"","preTaskCode":0,"preTaskVersion":0,"postTaskCode":4097401467904,"postTaskVersion":0,"conditionType":0,"conditionParams":{}}], name=DAG-1, description=, locations=[{"taskCode":4097401467904,"x":423,"y":-185}], globalParams=[], tenantCode=default, timeout=0}
[ERROR] 2022-01-06 12:01:13.248 org.apache.dolphinscheduler.api.exceptions.ApiExceptionHandler:[46] - create process definition error
org.springframework.dao.DataIntegrityViolationException:
### Error updating database.  Cause: org.postgresql.util.PSQLException: ERROR: null value in column "id" violates not-null constraint
  Detail: Failing row contains (null, 4097401467904, S-1, 1, , 4096931718784, 1, SHELL, {"resourceList":[],"localParams":[],"rawScript":"echo \"hi\"","d..., 1, 2, default, -1, 0, 1, 0, 0, 0, 0, , 2022-01-06 12:01:13.243, 2022-01-06 12:01:13.243).
### The error may exist in class path resource [org/apache/dolphinscheduler/dao/mapper/TaskDefinitionMapper.xml]
### The error may involve defaultParameterMap
### The error occurred while setting parameters
### SQL: insert into t_ds_task_definition (code, name, version, description, project_code, user_id,         task_type, task_params, flag, task_priority, worker_group, environment_code, fail_retry_times, fail_retry_interval,         timeout_flag, timeout_notify_strategy, timeout, delay_time, resource_ids, create_time, update_time)         values                        (?,?,?,?,             ?,?,?,?,?,             ?,?,?,?,             ?,?,?,?,             ?,?,?,?)
### Cause: org.postgresql.util.PSQLException: ERROR: null value in column "id" violates not-null constraint
  Detail: Failing row contains (null, 4097401467904, S-1, 1, , 4096931718784, 1, SHELL, {"resourceList":[],"localParams":[],"rawScript":"echo \"hi\"","d..., 1, 2, default, -1, 0, 1, 0, 0, 0, 0, , 2022-01-06 12:01:13.243, 2022-01-06 12:01:13.243).
; ERROR: null value in column "id" violates not-null constraint
  Detail: Failing row contains (null, 4097401467904, S-1, 1, , 4096931718784, 1, SHELL, {"resourceList":[],"localParams":[],"rawScript":"echo \"hi\"","d..., 1, 2, default, -1, 0, 1, 0, 0, 0, 0, , 2022-01-06 12:01:13.243, 2022-01-06 12:01:13.243).; nested exception is org.postgresql.util.PSQLException: ERROR: null value in column "id" violates not-null constraint
  Detail: Failing row contains (null, 4097401467904, S-1, 1, , 4096931718784, 1, SHELL, {"resourceList":[],"localParams":[],"rawScript":"echo \"hi\"","d..., 1, 2, default, -1, 0, 1, 0, 0, 0, 0, , 2022-01-06 12:01:13.243, 2022-01-06 12:01:13.243).
    at org.springframework.jdbc.support.SQLErrorCodeSQLExceptionTranslator.doTranslate(SQLErrorCodeSQLExceptionTranslator.java:251)
    at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:70)
    at org.mybatis.spring.MyBatisExceptionTranslator.translateExceptionIfPossible(MyBatisExceptionTranslator.java:74)
    at org.mybatis.spring.SqlSessionTemplate$SqlSessionInterceptor.invoke(SqlSessionTemplate.java:440)
    at com.sun.proxy.$Proxy94.insert(Unknown Source)
    at org.mybatis.spring.SqlSessionTemplate.insert(SqlSessionTemplate.java:271)
    at com.baomidou.mybatisplus.core.override.MybatisMapperMethod.execute(MybatisMapperMethod.java:58)
    at com.baomidou.mybatisplus.core.override.MybatisMapperProxy.invoke(MybatisMapperProxy.java:61)
    at com.sun.proxy.$Proxy110.batchInsert(Unknown Source)
    at org.apache.dolphinscheduler.service.process.ProcessService.saveTaskDefine(ProcessService.java:2246)
    at org.apache.dolphinscheduler.service.process.ProcessService$$FastClassBySpringCGLIB$$ed138739.invoke(<generated>)
    at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
    at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:689)
    at org.apache.dolphinscheduler.service.process.ProcessService$$EnhancerBySpringCGLIB$$d0008a0f.saveTaskDefine(<generated>)
    at org.apache.dolphinscheduler.api.service.impl.ProcessDefinitionServiceImpl.createDagDefine(ProcessDefinitionServiceImpl.java:244)
    at org.apache.dolphinscheduler.api.service.impl.ProcessDefinitionServiceImpl.createProcessDefinition(ProcessDefinitionServiceImpl.java:236)
    at org.apache.dolphinscheduler.api.service.impl.ProcessDefinitionServiceImpl$$FastClassBySpringCGLIB$$e8e34ed9.invoke(<generated>)
    at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
    at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
    at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753)
    at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:123)
    at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:388)
    at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:119)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
    at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753)
    at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698)
    at org.apache.dolphinscheduler.api.service.impl.ProcessDefinitionServiceImpl$$EnhancerBySpringCGLIB$$4539a4ef.createProcessDefinition(<generated>)
    at org.apache.dolphinscheduler.api.controller.ProcessDefinitionController.createProcessDefinition(ProcessDefinitionController.java:130)
    at org.apache.dolphinscheduler.api.controller.ProcessDefinitionController$$FastClassBySpringCGLIB$$dc9bf5db.invoke(<generated>)
    at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
    at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
    at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753)
    at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:89)
    at org.apache.dolphinscheduler.api.aspect.AccessLogAspect.doAround(AccessLogAspect.java:87)
    at sun.reflect.GeneratedMethodAccessor138.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:634)
    at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:624)
    at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:72)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
    at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753)
    at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
    at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753)
    at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698)
    at org.apache.dolphinscheduler.api.controller.ProcessDefinitionController$$EnhancerBySpringCGLIB$$e55d5844.createProcessDefinition(<generated>)
    at sun.reflect.GeneratedMethodAccessor236.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205)
    at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:150)
    at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:117)
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:895)
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:808)
    at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87)
    at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1067)
    at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:963)
    at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006)
    at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:909)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:517)
    at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:584)
    at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799)
    at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631)
    at org.springframework.web.filter.CorsFilter.doFilterInternal(CorsFilter.java:91)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
    at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
    at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
    at com.github.xiaoymin.swaggerbootstrapui.filter.SecurityBasicAuthFilter.doFilter(SecurityBasicAuthFilter.java:84)
    at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
    at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
    at com.github.xiaoymin.swaggerbootstrapui.filter.ProductionSecurityFilter.doFilter(ProductionSecurityFilter.java:53)
    at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
    at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
    at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
    at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
    at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
    at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
    at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
    at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
    at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
    at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
    at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
    at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
    at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600)
    at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
    at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235)
    at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624)
    at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
    at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1434)
    at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
    at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501)
    at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594)
    at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
    at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1349)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
    at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:763)
    at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
    at org.eclipse.jetty.server.Server.handle(Server.java:516)
    at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:400)
    at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:645)
    at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:392)
    at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277)
    at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
    at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
    at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)
    at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338)
    at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315)
    at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173)
    at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)
    at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409)
    at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883)
    at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034)
    at java.lang.Thread.run(Thread.java:748)
Caused by: org.postgresql.util.PSQLException: ERROR: null value in column "id" violates not-null constraint
  Detail: Failing row contains (null, 4097401467904, S-1, 1, , 4096931718784, 1, SHELL, {"resourceList":[],"localParams":[],"rawScript":"echo \"hi\"","d..., 1, 2, default, -1, 0, 1, 0, 0, 0, 0, , 2022-01-06 12:01:13.243, 2022-01-06 12:01:13.243).
    at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2440)
    at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2183)
    at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:308)
    at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:441)
    at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:365)
    at org.postgresql.jdbc.PgPreparedStatement.executeWithFlags(PgPreparedStatement.java:143)
    at org.postgresql.jdbc.PgPreparedStatement.execute(PgPreparedStatement.java:132)
    at com.zaxxer.hikari.pool.ProxyPreparedStatement.execute(ProxyPreparedStatement.java:44)
    at com.zaxxer.hikari.pool.HikariProxyPreparedStatement.execute(HikariProxyPreparedStatement.java)
    at org.apache.ibatis.executor.statement.PreparedStatementHandler.update(PreparedStatementHandler.java:47)
    at org.apache.ibatis.executor.statement.RoutingStatementHandler.update(RoutingStatementHandler.java:74)
    at sun.reflect.GeneratedMethodAccessor234.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.apache.ibatis.plugin.Plugin.invoke(Plugin.java:63)
    at com.sun.proxy.$Proxy175.update(Unknown Source)
    at com.baomidou.mybatisplus.core.executor.MybatisSimpleExecutor.doUpdate(MybatisSimpleExecutor.java:54)
    at org.apache.ibatis.executor.BaseExecutor.update(BaseExecutor.java:117)
    at org.apache.ibatis.session.defaults.DefaultSqlSession.update(DefaultSqlSession.java:197)
    at org.apache.ibatis.session.defaults.DefaultSqlSession.insert(DefaultSqlSession.java:184)
    at sun.reflect.GeneratedMethodAccessor233.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.mybatis.spring.SqlSessionTemplate$SqlSessionInterceptor.invoke(SqlSessionTemplate.java:426)
    ... 117 common frames omitted

And i have found same issue https://github.com/apache/dolphinscheduler/issues/4650, but it looks like this problem reappeared in version 2.0.1.

What you expected to happen

create process success.

How to reproduce

Just follow https://dolphinscheduler.apache.org/en-us/docs/latest/user_doc/guide/installation/docker.html, and build dolphinscheduler with docker-compose.

Anything else

No response

Version

2.0.1

Are you willing to submit PR?

[ ] Yes I am willing to submit a PR!

Code of Conduct

[X] I agree to follow this project's Code of Conduct

github-actions[bot] commented 2 years ago

Hi:

Thank you for your feedback, we have received your issue, Please wait patiently for a reply.
In order for us to understand your request as soon as possible, please provide detailed information、version or pictures.
If you haven't received a reply for a long time, you can subscribe to the developer's email，Mail subscription steps reference https://dolphinscheduler.apache.org/en-us/community/development/subscribe.html ,Then write the issue URL in the email content and send question to dev@dolphinscheduler.apache.org.

gt199899 commented 2 years ago

For now, after add some seq, it work

create sequence t_ds_task_definition_id_sequence increment by 1 minvalue 1 no maxvalue start with 1;
alter table t_ds_task_definition alter column id set default nextval('t_ds_task_definition_id_sequence');

create sequence t_ds_task_definition_log_id_sequence increment by 1 minvalue 1 no maxvalue start with 1;
alter table t_ds_task_definition_log alter column id set default nextval('t_ds_task_definition_log_id_sequence');

create sequence t_ds_process_definition_log_id_sequence increment by 1 minvalue 1 no maxvalue start with 1;
alter table t_ds_process_definition_log alter column id set default nextval('t_ds_process_definition_log_id_sequence');

create sequence t_ds_process_task_relation_id_sequence increment by 1 minvalue 1 no maxvalue start with 1;
alter table t_ds_process_task_relation alter column id set default nextval('t_ds_process_task_relation_id_sequence');

create sequence t_ds_process_task_relation_log_id_sequence increment by 1 minvalue 1 no maxvalue start with 1;
alter table t_ds_process_task_relation_log alter column id set default nextval('t_ds_process_task_relation_log_id_sequence');

Is the creation of seq missing when the database is initialized?

apache / dolphinscheduler

[Bug] [API] Create process response error #7843