search

avniproject / avni-etl

GNU Affero General Public License v3.0
0 stars 6 forks source link

Unable to Enable ETL for OrgGroups with large number of orgs #105

Closed himeshr closed 2 months ago

himeshr commented 2 months ago

Unable to Enable ETL for OrgGroups with large number of orgs, as we exceed the limit for JobDetail.description column of length 250 characters.

Root cause exception stack trace

Jul 18 05:39:02 ip-10-100-1-136 start.sh[336794]: 2024-07-18T05:39:02.369Z ERROR 336794 --- [nio-8022-exec-1] o.a.c.c.C.[.[.[/].[dispatcherServlet]    : Servlet.service() for servlet [dispatcherServlet] in context>
Jul 18 05:39:02 ip-10-100-1-136 start.sh[336794]: org.postgresql.util.PSQLException: ERROR: value too long for type character varying(250)
Jul 18 05:39:02 ip-10-100-1-136 start.sh[336794]:         at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2713) ~[postgresql-42.6.0.jar!/:42.6.0]
Jul 18 05:39:02 ip-10-100-1-136 start.sh[336794]:         at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2401) ~[postgresql-42.6.0.jar!/:42.6.0]
Jul 18 05:39:02 ip-10-100-1-136 start.sh[336794]:         at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:368) ~[postgresql-42.6.0.jar!/:42.6.0]
Jul 18 05:39:02 ip-10-100-1-136 start.sh[336794]:         at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:498) ~[postgresql-42.6.0.jar!/:42.6.0]
Jul 18 05:39:02 ip-10-100-1-136 start.sh[336794]:         at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:415) ~[postgresql-42.6.0.jar!/:42.6.0]
Jul 18 05:39:02 ip-10-100-1-136 start.sh[336794]:         at org.postgresql.jdbc.PgPreparedStatement.executeWithFlags(PgPreparedStatement.java:190) ~[postgresql-42.6.0.jar!/:42.6.0]
Jul 18 05:39:02 ip-10-100-1-136 start.sh[336794]:         at org.postgresql.jdbc.PgPreparedStatement.executeUpdate(PgPreparedStatement.java:152) ~[postgresql-42.6.0.jar!/:42.6.0]
Jul 18 05:39:02 ip-10-100-1-136 start.sh[336794]:         at com.zaxxer.hikari.pool.ProxyPreparedStatement.executeUpdate(ProxyPreparedStatement.java:61) ~[HikariCP-5.0.1.jar!/:na]
Jul 18 05:39:02 ip-10-100-1-136 start.sh[336794]:         at com.zaxxer.hikari.pool.HikariProxyPreparedStatement.executeUpdate(HikariProxyPreparedStatement.java) ~[HikariCP-5.0.1.jar!/:na]
Jul 18 05:39:02 ip-10-100-1-136 start.sh[336794]:         at org.quartz.impl.jdbcjobstore.StdJDBCDelegate.insertJobDetail(StdJDBCDelegate.java:624) ~[quartz-2.3.2.jar!/:na]
Jul 18 05:39:02 ip-10-100-1-136 start.sh[336794]:         at org.quartz.impl.jdbcjobstore.JobStoreSupport.storeJob(JobStoreSupport.java:1117) ~[quartz-2.3.2.jar!/:na]
Jul 18 05:39:02 ip-10-100-1-136 start.sh[336794]:         at org.quartz.impl.jdbcjobstore.JobStoreSupport$3.executeVoid(JobStoreSupport.java:1095) ~[quartz-2.3.2.jar!/:na]
Jul 18 05:39:02 ip-10-100-1-136 start.sh[336794]:         at org.quartz.impl.jdbcjobstore.JobStoreSupport$VoidTransactionCallback.execute(JobStoreSupport.java:3780) ~[quartz-2.3.2.jar!/:na]
Jul 18 05:39:02 ip-10-100-1-136 start.sh[336794]:         at org.quartz.impl.jdbcjobstore.JobStoreSupport$VoidTransactionCallback.execute(JobStoreSupport.java:3778) ~[quartz-2.3.2.jar!/:na]
Jul 18 05:39:02 ip-10-100-1-136 start.sh[336794]:         at org.quartz.impl.jdbcjobstore.JobStoreCMT.executeInLock(JobStoreCMT.java:245) ~[quartz-2.3.2.jar!/:na]
Jul 18 05:39:02 ip-10-100-1-136 start.sh[336794]:         at org.quartz.impl.jdbcjobstore.JobStoreSupport.storeJob(JobStoreSupport.java:1091) ~[quartz-2.3.2.jar!/:na]
Jul 18 05:39:02 ip-10-100-1-136 start.sh[336794]:         at org.quartz.core.QuartzScheduler.addJob(QuartzScheduler.java:938) ~[quartz-2.3.2.jar!/:na]
Jul 18 05:39:02 ip-10-100-1-136 start.sh[336794]:         at org.quartz.core.QuartzScheduler.addJob(QuartzScheduler.java:927) ~[quartz-2.3.2.jar!/:na]
Jul 18 05:39:02 ip-10-100-1-136 start.sh[336794]:         at org.quartz.impl.StdScheduler.addJob(StdScheduler.java:268) ~[quartz-2.3.2.jar!/:na]
Jul 18 05:39:02 ip-10-100-1-136 start.sh[336794]:         at org.avniproject.etl.controller.backgroundJob.EtlJobController.createJob(EtlJobController.java:90) ~[classes!/:1.0.0-SNAPSHOT]
Jul 18 05:39:02 ip-10-100-1-136 start.sh[336794]:         at jdk.internal.reflect.GeneratedMethodAccessor554.invoke(Unknown Source) ~[na:na]
Jul 18 05:39:02 ip-10-100-1-136 start.sh[336794]:         at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na]
Jul 18 05:39:02 ip-10-100-1-136 start.sh[336794]:         at java.base/java.lang.reflect.Method.invoke(Method.java:568) ~[na:na]
Jul 18 05:39:02 ip-10-100-1-136 start.sh[336794]:         at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:343) ~[spring-aop-6.0.9.jar!/:6.0.9]
Jul 18 05:39:02 ip-10-100-1-136 start.sh[336794]:         at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:196) ~[spring-aop-6.0.9.jar!/:6.0.9]
Jul 18 05:39:02 ip-10-100-1-136 start.sh[336794]:         at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) ~[spring-aop-6.0.9.jar!/:6.0.9]
Jul 18 05:39:02 ip-10-100-1-136 start.sh[336794]:         at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:750) ~[spring-aop-6.0.9.jar!/:6.0.9]
Jul 18 05:39:02 ip-10-100-1-136 start.sh[336794]:         at org.springframework.security.authorization.method.AuthorizationManagerBeforeMethodInterceptor.invoke(AuthorizationManagerBeforeMethodInterceptor.java:19>
Jul 18 05:39:02 ip-10-100-1-136 start.sh[336794]:         at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184) ~[spring-aop-6.0.9.jar!/:6.0.9]
Jul 18 05:39:02 ip-10-100-1-136 start.sh[336794]:         at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:750) ~[spring-aop-6.0.9.jar!/:6.0.9]
Jul 18 05:39:02 ip-10-100-1-136 start.sh[336794]:         at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:702) ~[spring-aop-6.0.9.jar!/:6.0.9]
Jul 18 05:39:02 ip-10-100-1-136 start.sh[336794]:         at org.avniproject.etl.controller.backgroundJob.EtlJobController$$SpringCGLIB$$0.createJob(<generated>) ~[classes!/:1.0.0-SNAPSHOT]
Jul 18 05:39:02 ip-10-100-1-136 start.sh[336794]:         at jdk.internal.reflect.GeneratedMethodAccessor554.invoke(Unknown Source) ~[na:na]
Jul 18 05:39:02 ip-10-100-1-136 start.sh[336794]:         at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na]
Jul 18 05:39:02 ip-10-100-1-136 start.sh[336794]:         at java.base/java.lang.reflect.Method.invoke(Method.java:568) ~[na:na]
himeshr commented 2 months ago

Code change done as part of commit https://github.com/avniproject/avni-etl/commit/b917f31d9bd70159c1b5482dd900ac2ca3594ffc