[Bug] [CDCSOURCE] MysqlCDC to paimon with hive catalog error

zhuxt2015 commented 3 weeks ago

Search before asking

[X] I had searched in the issues and found no similar issues.

What happened

SQL

SET 'table.exec.sink.upsert-materialize' = 'NONE'; -- 消除无用的SinkMaterialize算子。

EXECUTE CDCSOURCE dinky_paimon_auto_create_table
WITH
  (
    'connector' = 'mysql-cdc',
    'hostname' = 'mysql-new',
    'port' = '3306',
    'username' = 'root',
    'password' = 'root',
    'checkpoint' = '10000',
    'parallelism' = '1',
    'jdbc.properties.useSSL' = 'false',
    'jdbc.properties.tinyInt1isBit' = 'false',
    'scan.startup.mode' = 'initial',
    'table-name' = 'dinky_dev\.dinky_history',
'sink.connector' = 'sql-catalog',
'sink.catalog.name' = 'paimon_test',
'sink.catalog.type' = 'paimon',
'sink.catalog.metastore' = 'hive',
'sink.catalog.uri' = 'thrift://cdhtestdn02:9083',
'sink.catalog.warehouse'='hdfs:///user/hive/warehouse',
'sink.catalog.hive-conf-dir' = '/etc/hive/conf',
'sink.catalog.hadoop-conf-dir' = '/etc/hadoop/conf'
  );

error log:

Start Process:FlinkSubmit/12
Start Process Step:SUBMIT_PRECHECK
2024-08-20 07:55:33.205 INFO  org.dinky.service.impl.TaskServiceImpl(188): Start check and config task, task:test-3
Process Step SUBMIT_PRECHECK exit with status:FINISHED
Start Process Step:SUBMIT_EXECUTE
Start Process Step:SUBMIT_BUILD_CONFIG
2024-08-20 07:55:33.211 INFO  org.dinky.service.impl.TaskServiceImpl(300): Start initialize FlinkSQLEnv:
2024-08-20 07:55:33.212 INFO  org.dinky.service.impl.TaskServiceImpl(320): Initializing data permissions...
2024-08-20 07:55:33.220 INFO  org.dinky.service.impl.TaskServiceImpl(322): Finish initialize FlinkSQLEnv.
2024-08-20 07:55:33.221 INFO  org.dinky.service.impl.TaskServiceImpl(250): Init remote cluster
Process Step SUBMIT_BUILD_CONFIG exit with status:FINISHED
2024-08-20 07:55:33.263 INFO  org.dinky.service.task.FlinkSqlTask(67): Initializing Flink job config...
2024-08-20 07:55:33.277 INFO  org.dinky.job.builder.JobUDFBuilder(125): A total of 0 UDF have been Init.
2024-08-20 07:55:33.278 INFO  org.dinky.job.builder.JobUDFBuilder(126): Initializing Flink UDF...Finish
2024-08-20 07:55:33.279 INFO  org.dinky.trans.AbstractOperation(77): Start build CDCSOURCE Task...
2024-08-20 07:55:33.303 INFO  org.dinky.trans.AbstractOperation(174): A total of 1 tables were detected...
2024-08-20 07:55:33.304 INFO  org.dinky.trans.AbstractOperation(176): 1: dinky_dev.dinky_history
2024-08-20 07:55:33.304 INFO  org.dinky.trans.AbstractOperation(185): Set parallelism: 1
2024-08-20 07:55:33.305 INFO  org.dinky.trans.AbstractOperation(189): Set checkpoint: 10000
2024-08-20 07:55:33.349 INFO  org.dinky.trans.AbstractOperation(192): Build mysql-cdc successful...
2024-08-20 07:55:33.350 INFO  org.dinky.cdc.AbstractSinkBuilder(90): Build catalog successful...
2024-08-20 07:55:33.497 INFO  org.apache.paimon.hive.HiveCatalog(654): Setting hive conf dir as /etc/hive/conf
2024-08-20 07:55:33.508 INFO  org.apache.hadoop.hive.conf.HiveConf(187): Found configuration file file:/etc/hive/conf/hive-site.xml
2024-08-20 07:55:34.495 INFO  hive.metastore(405): Trying to connect to metastore with URI thrift://cdhtestdn02.yili.com:9083
2024-08-20 07:55:34.583 INFO  hive.metastore(479): Opened a connection to metastore, current connections: 1
2024-08-20 07:55:34.659 INFO  hive.metastore(532): Connected to metastore.
2024-08-20 07:55:35.063 INFO  org.dinky.cdc.AbstractSinkBuilder(220): Build deserialize successful...
2024-08-20 07:55:35.070 INFO  org.dinky.cdc.AbstractSinkBuilder(146): Build dinky_dev.dinky_history shunt successful...
2024-08-20 07:55:35.072 INFO  org.dinky.cdc.AbstractSinkBuilder(153): Build dinky_dev.dinky_history flatMap successful...
2024-08-20 07:55:35.073 INFO  org.dinky.cdc.AbstractSinkBuilder(154): Start build dinky_dev.dinky_history sink...
2024-08-20 07:55:35.075 INFO  org.dinky.cdc.AbstractSinkBuilder(74): Create VIEW_dinky_dev_dinky_history temporaryView successful...
2024-08-20 07:55:35.076 INFO  org.dinky.cdc.AbstractSinkBuilder(248): INSERT INTO `paimon_test.dinky_dev.dinky_history` SELECT
    `id` 
    ,`tenant_id` 
    ,`cluster_id` 
    ,`cluster_configuration_id` 
    ,`session` 
    ,`job_id` 
    ,`job_name` 
    ,`job_manager_address` 
    ,`status` 
    ,`batch_model` 
    ,`type` 
    ,`statement` 
    ,`error` 
    ,`result` 
    ,`config_json` 
    ,`start_time` 
    ,`end_time` 
    ,`task_id` 
 FROM `VIEW_dinky_dev_dinky_history`
2024-08-20 07:55:35.083 ERROR org.dinky.cdc.AbstractSinkBuilder(158): Build dinky_dev.dinky_history cdc sync failed...
2024-08-20 07:55:35.084 ERROR org.dinky.cdc.AbstractSinkBuilder(159): org.apache.flink.table.api.TableException: Cannot find table '`default_catalog`.`default_database`.`paimon_test.dinky_dev.dinky_history`' in any of the catalogs [paimon_test, default_catalog], nor as a temporary table.
    at org.apache.flink.table.catalog.CatalogManager.lambda$getTableOrError$4(CatalogManager.java:622)
    at java.util.Optional.orElseThrow(Optional.java:290)
    at org.apache.flink.table.catalog.CatalogManager.getTableOrError(CatalogManager.java:619)
    at org.apache.flink.table.planner.operations.SqlNodeToOperationConversion.convertSqlInsert(SqlNodeToOperationConversion.java:741)
    at org.apache.flink.table.planner.operations.SqlNodeToOperationConversion.convertValidatedSqlNode(SqlNodeToOperationConversion.java:353)
    at org.apache.flink.table.planner.operations.SqlNodeToOperationConversion.convert(SqlNodeToOperationConversion.java:262)
    at org.apache.flink.table.planner.delegation.ParserImpl.parse(ParserImpl.java:106)
    at org.dinky.executor.ParserWrapper.parse(ParserWrapper.java:47)
    at org.dinky.cdc.AbstractSinkBuilder.createInsertOperations(AbstractSinkBuilder.java:250)
    at org.dinky.cdc.sql.catalog.SQLCatalogSinkBuilder.addTableSink(SQLCatalogSinkBuilder.java:76)
    at org.dinky.cdc.sql.AbstractSqlSinkBuilder.lambda$addTableSinkForTags$0(AbstractSqlSinkBuilder.java:156)
    at java.util.LinkedHashMap.forEach(LinkedHashMap.java:684)
    at org.dinky.cdc.sql.AbstractSqlSinkBuilder.addTableSinkForTags(AbstractSqlSinkBuilder.java:142)
    at org.dinky.cdc.sql.AbstractSqlSinkBuilder.build(AbstractSqlSinkBuilder.java:244)
    at org.dinky.trans.ddl.CreateCDCSourceOperation.execute(CreateCDCSourceOperation.java:193)
    at org.dinky.interceptor.FlinkInterceptor.build(FlinkInterceptor.java:53)
    at org.dinky.executor.Executor.pretreatExecute(Executor.java:186)
    at org.dinky.executor.Executor.executeSql(Executor.java:199)
    at org.dinky.job.builder.JobExecuteBuilder.run(JobExecuteBuilder.java:96)
    at org.dinky.job.JobManager.executeSql(JobManager.java:301)
    at org.dinky.service.task.FlinkSqlTask.execute(FlinkSqlTask.java:68)
    at org.dinky.service.impl.TaskServiceImpl.executeJob(TaskServiceImpl.java:202)
    at org.dinky.service.impl.TaskServiceImpl$$FastClassBySpringCGLIB$$22087f7c.invoke(<generated>)
    at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
    at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:792)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
    at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:762)
    at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:89)
    at org.dinky.aop.ProcessAspect.processStepAround(ProcessAspect.java:110)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:634)
    at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:624)
    at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:72)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
    at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:762)
    at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
    at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:762)
    at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:707)
    at org.dinky.service.impl.TaskServiceImpl$$EnhancerBySpringCGLIB$$5d452f2f.executeJob(<generated>)
    at org.dinky.service.impl.TaskServiceImpl.submitTask(TaskServiceImpl.java:333)
    at org.dinky.service.impl.TaskServiceImpl$$FastClassBySpringCGLIB$$22087f7c.invoke(<generated>)
    at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
    at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:792)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
    at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:762)
    at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:89)
    at org.dinky.aop.UdfClassLoaderAspect.round(UdfClassLoaderAspect.java:55)
    at sun.reflect.GeneratedMethodAccessor284.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:634)
    at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:624)
    at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:72)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
    at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:762)
    at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
    at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:762)
    at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:707)
    at org.dinky.service.impl.TaskServiceImpl$$EnhancerBySpringCGLIB$$5d452f2f.submitTask(<generated>)
    at org.dinky.controller.TaskController.submitTask(TaskController.java:88)
    at org.dinky.controller.TaskController$$FastClassBySpringCGLIB$$5b68bd32.invoke(<generated>)
    at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
    at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:792)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
    at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:762)
    at org.springframework.aop.framework.adapter.AfterReturningAdviceInterceptor.invoke(AfterReturningAdviceInterceptor.java:57)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
    at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:762)
    at org.springframework.aop.aspectj.AspectJAfterThrowingAdvice.invoke(AspectJAfterThrowingAdvice.java:64)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
    at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:762)
    at org.springframework.aop.framework.adapter.AfterReturningAdviceInterceptor.invoke(AfterReturningAdviceInterceptor.java:57)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
    at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:762)
    at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:89)
    at org.dinky.aop.TaskOperationPermissionAspect.processAround(TaskOperationPermissionAspect.java:82)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:634)
    at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:624)
    at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:72)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
    at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:762)
    at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:89)
    at org.dinky.aop.ProcessAspect.processAround(ProcessAspect.java:73)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:634)
    at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:624)
    at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:72)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
    at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:762)
    at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
    at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:762)
    at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:707)
    at org.dinky.controller.TaskController$$EnhancerBySpringCGLIB$$71bcfe79.submitTask(<generated>)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205)
    at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:150)
    at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:117)
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:895)
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:808)
    at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87)
    at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1072)
    at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:965)
    at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006)
    at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:898)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:529)
    at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:623)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:209)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:153)
    at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:51)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:178)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:153)
    at org.springframework.web.filter.CorsFilter.doFilterInternal(CorsFilter.java:91)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:178)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:153)
    at com.alibaba.druid.support.http.WebStatFilter.doFilter(WebStatFilter.java:114)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:178)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:153)
    at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:178)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:153)
    at cn.dev33.satoken.filter.SaPathCheckFilterForServlet.doFilter(SaPathCheckFilterForServlet.java:55)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:178)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:153)
    at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:178)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:153)
    at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.doFilterInternal(WebMvcMetricsFilter.java:96)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:178)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:153)
    at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:178)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:153)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:168)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:90)
    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:481)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:130)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:93)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74)
    at org.apache.catalina.valves.RemoteIpValve.invoke(RemoteIpValve.java:765)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:342)
    at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:390)
    at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:63)
    at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:928)
    at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1794)
    at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:52)
    at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191)
    at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)
    at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
    at java.lang.Thread.run(Thread.java:750)

2024-08-20 07:55:35.086 INFO  org.dinky.cdc.AbstractSinkBuilder(250): A total of 0 table cdc sync were build successful...
2024-08-20 07:55:35.086 INFO  org.dinky.trans.AbstractOperation(195): Build CDCSOURCE Task successful!
2024-08-20 07:55:35.159 INFO  org.apache.flink.client.program.rest.RestClusterClient(410): Submitting job 'test-3' (97833154b6a4650869f688a478c929b4).
2024-08-20 07:55:35.249 INFO  org.apache.flink.client.program.rest.RestClusterClient(429): Successfully submitted job 'test-3' (97833154b6a4650869f688a478c929b4) to 'http://flink-session-rest:8081'.
2024-08-20 07:55:35.470 INFO  org.dinky.service.impl.TaskServiceImpl(203): execute job finished,status is SUCCESS
Process Step SUBMIT_EXECUTE exit with status:FINISHED
2024-08-20 07:55:35.471 INFO  org.dinky.service.impl.TaskServiceImpl(337): Job Submit success
Process FlinkSubmit/12 exit with status:FINISHED

What you expected to happen

create paimon table with hive catalog automate, can query paimon table in hive CLI

How to reproduce

use example SQL

Anything else

No response

Version

dev

Are you willing to submit PR?

[ ] Yes I am willing to submit a PR!

Code of Conduct

[X] I agree to follow this project's Code of Conduct

github-actions[bot] commented 3 weeks ago

Hello @zhuxt2015, this issue is about web, so I assign it to @Zzm0809. If you have any questions, you can comment and reply.

你好 @zhuxt2015, 这个 issue 是关于 web 的，所以我把它分配给了 @Zzm0809。如有任何问题，可以评论回复。

github-actions[bot] commented 3 weeks ago

Hello @zhuxt2015, this issue is about CDC/CDCSOURCE, so I assign it to @aiwenmo. If you have any questions, you can comment and reply.

你好 @zhuxt2015, 这个 issue 是关于 CDC/CDCSOURCE 的，所以我把它分配给了 @aiwenmo。如有任何问题，可以评论回复。

Zzm0809 commented 3 weeks ago

看一下你的报错信息: 2024-08-20 07:55:35.084 ERROR org.dinky.cdc.AbstractSinkBuilder(159): org.apache.flink.table.api.TableException: Cannot find table 'default_catalog.default_database.paimon_test.dinky_dev.dinky_history' in any of the catalogs [paimon_test, default_catalog], nor as a temporary table.

你的表层级: default_catalog.default_database.paimon_test.dinky_dev.dinky_history 这肯定是不对的

zhuxt2015 commented 3 weeks ago

看一下你的报错信息:

2024-08-20 07:55:35.084 ERROR org.dinky.cdc.AbstractSinkBuilder(159): org.apache.flink.table.api.TableException: Cannot find table 'default_catalog.default_database.paimon_test.dinky_dev.dinky_history' in any of the catalogs [paimon_test, default_catalog], nor as a temporary table.

你的表层级: default_catalog.default_database.paimon_test.dinky_dev.dinky_history 这肯定是不对的

这个表表层级是dinky生成的，我想基于hive catalog创建一个default.db/dinky_history paimon表，应该怎么做呢？

diaozhonghao commented 1 week ago

看一下你的报错信息: 2024-08-20 07:55:35.084 ERROR org.dinky.cdc.AbstractSinkBuilder(159): org.apache.flink.table.api.TableException: Cannot find table 'default_catalog.default_database.paimon_test.dinky_dev.dinky_history' in any of the catalogs [paimon_test, default_catalog], nor as a temporary table. 你的表层级: default_catalog.default_database.paimon_test.dinky_dev.dinky_history 这肯定是不对的

这个表表层级是dinky生成的，我想基于hive catalog创建一个default.db/dinky_history paimon表，应该怎么做呢？

解决了吗哥们,我也sink到hive也遇到了同样的问题

zhuxt2015 commented 1 week ago

看一下你的报错信息:

2024-08-20 07:55:35.084 ERROR org.dinky.cdc.AbstractSinkBuilder(159): org.apache.flink.table.api.TableException: Cannot find table 'default_catalog.default_database.paimon_test.dinky_dev.dinky_history' in any of the catalogs [paimon_test, default_catalog], nor as a temporary table.

你的表层级: default_catalog.default_database.paimon_test.dinky_dev.dinky_history 这肯定是不对的

这个表表层级是dinky生成的，我想基于hive catalog创建一个default.db/dinky_history paimon表，应该怎么做呢？

解决了吗哥们,我也sink到hive也遇到了同样的问题

我现在不用整库同步了，改用单表同步

diaozhonghao commented 1 week ago

看一下你的报错信息:

2024-08-20 07:55:35.084 ERROR org.dinky.cdc.AbstractSinkBuilder(159): org.apache.flink.table.api.TableException: Cannot find table 'default_catalog.default_database.paimon_test.dinky_dev.dinky_history' in any of the catalogs [paimon_test, default_catalog], nor as a temporary table.

你的表层级: default_catalog.default_database.paimon_test.dinky_dev.dinky_history 这肯定是不对的

这个表表层级是dinky生成的，我想基于hive catalog创建一个default.db/dinky_history paimon表，应该怎么做呢？

解决了吗哥们,我也sink到hive也遇到了同样的问题

我现在不用整库同步了，改用单表同步

okok 等我研究出来了我给你反馈奥哥们

DataLinkDC / dinky