search

apache / gravitino

World's most powerful open data catalog for building a high-performance, geo-distributed and federated metadata lake.
https://gravitino.apache.org
Apache License 2.0
1.03k stars 319 forks source link

[Bug report] Trino: RCBinary is not a support format for hive table #5401

Open danhuawang opened 2 hours ago

danhuawang commented 2 hours ago

Version

main branch

Describe what's wrong

In Trino connector doc: https://gravitino.apache.org/docs/0.6.1-incubating/trino-connector/catalog-hive

image

In Hive catalog doc: https://gravitino.apache.org/docs/0.6.1-incubating/apache-hive-catalog

image

Actually in Trino:

image

Error message and/or stacktrace

Query 20241031_082337_00378_sfnt2 failed: Failed to operate table(s) [storage_formats_compatibility_data_types_rcbinary] operation [CREATE] under schema [default], reason [Invalid value: 'RCBINARY' for property: 'format']
java.lang.IllegalArgumentException: Invalid value: 'RCBINARY' for property: 'format'
    at org.apache.gravitino.catalog.PropertiesMetadataHelpers.checkValueFormat(PropertiesMetadataHelpers.java:40)
    at org.apache.gravitino.catalog.PropertiesMetadataHelpers.validatePropertyForCreate(PropertiesMetadataHelpers.java:75)
    at org.apache.gravitino.catalog.TableOperationDispatcher.lambda$internalCreateTable$19(TableOperationDispatcher.java:436)
    at org.apache.gravitino.catalog.CatalogManager$CatalogWrapper.lambda$doWithPropertiesMeta$6(CatalogManager.java:176)
    at org.apache.gravitino.utils.IsolatedClassLoader.withClassLoader(IsolatedClassLoader.java:86)
    at org.apache.gravitino.catalog.CatalogManager$CatalogWrapper.doWithPropertiesMeta(CatalogManager.java:176)
    at org.apache.gravitino.catalog.TableOperationDispatcher.lambda$internalCreateTable$20(TableOperationDispatcher.java:434)
    at org.apache.gravitino.catalog.OperationDispatcher.doWithCatalog(OperationDispatcher.java:97)
    at org.apache.gravitino.catalog.TableOperationDispatcher.internalCreateTable(TableOperationDispatcher.java:431)
    at org.apache.gravitino.catalog.TableOperationDispatcher.lambda$createTable$4(TableOperationDispatcher.java:156)
    at org.apache.gravitino.lock.TreeLockUtils.doWithTreeLock(TreeLockUtils.java:49)
    at org.apache.gravitino.catalog.TableOperationDispatcher.createTable(TableOperationDispatcher.java:152)
    at org.apache.gravitino.hook.TableHookDispatcher.createTable(TableHookDispatcher.java:76)
    at org.apache.gravitino.catalog.TableNormalizeDispatcher.createTable(TableNormalizeDispatcher.java:78)
    at org.apache.gravitino.listener.TableEventDispatcher.createTable(TableEventDispatcher.java:114)
    at org.apache.gravitino.server.web.rest.TableOperations.lambda$createTable$2(TableOperations.java:124)
    at java.base/java.security.AccessController.doPrivileged(AccessController.java:712)
    at java.base/javax.security.auth.Subject.doAs(Subject.java:439)
    at org.apache.gravitino.utils.PrincipalUtils.doAs(PrincipalUtils.java:39)
    at org.apache.gravitino.server.web.Utils.doAs(Utils.java:155)
    at org.apache.gravitino.server.web.rest.TableOperations.createTable(TableOperations.java:116)
    at jdk.internal.reflect.GeneratedMethodAccessor53.invoke(Unknown Source)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:568)
    at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52)
    at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:146)
    at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:189)
    at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176)
    at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:93)
    at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:478)
    at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:400)
    at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81)
    at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:256)
    at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248)
    at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244)
    at org.glassfish.jersey.internal.Errors.process(Errors.java:292)
    at org.glassfish.jersey.internal.Errors.process(Errors.java:274)
    at org.glassfish.jersey.internal.Errors.process(Errors.java:244)
    at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265)
    at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:235)
    at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:684)
    at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394)
    at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346)
    at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:358)
    at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:311)
    at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205)
    at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799)
    at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1656)
    at org.apache.gravitino.server.authentication.AuthenticationFilter.doFilter(AuthenticationFilter.java:86)
    at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
    at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1626)
    at org.apache.gravitino.server.web.VersioningFilter.doFilter(VersioningFilter.java:111)
    at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
    at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1626)
    at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:552)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
    at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600)
    at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
    at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235)
    at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624)
    at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
    at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440)
    at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
    at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:505)
    at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594)
    at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
    at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
    at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146)
    at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
    at org.eclipse.jetty.server.Server.handle(Server.java:516)
    at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487)
    at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732)
    at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479)
    at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277)
    at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
    at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
    at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)
    at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338)
    at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315)
    at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173)
    at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)
    at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409)
    at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883)
    at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034)
    at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: java.lang.IllegalArgumentException: Invalid value [RCBINARY]. Valid values: [sequencefile, textfile, rcfile, orc, parquet, avro, json, csv, regex]
    at org.apache.gravitino.connector.PropertyEntry.lambda$enumPropertyEntry$1(PropertyEntry.java:360)
    at org.apache.gravitino.connector.PropertyEntry.decode(PropertyEntry.java:170)
    at org.apache.gravitino.catalog.PropertiesMetadataHelpers.checkValueFormat(PropertiesMetadataHelpers.java:37)
    ... 85 more
Caused by: java.lang.IllegalArgumentException: No enum constant org.apache.gravitino.catalog.hive.StorageFormat.RCBINARY

How to reproduce

create hive table in Trino:

CREATE TABLE storage_formats_compatibility_data_types_rcbinary
 (
C_boolean boolean,
C_tinyint tinyint,
C_smallint smallint,
C_int  integer,
C_bigint bigint,
C_real real,
C_double double,
C_decimal_10_0 decimal(10,0),
C_decimal_10_2 decimal(10,2),
C_decimal_38_5 decimal(38,5),
C_char char(10),
C_varchar varchar(10),
C_string varchar,
C_binary varbinary,
C_date date,
C_timestamp timestamp,
C_array array(integer),
C_map map(varchar, varchar),
C_row row(f1 integer, f2 varchar)
) WITH (format='rcbinary');

Additional context

No response

danhuawang commented 2 hours ago

@diqiu50 Can you help check this issue?