apache / gravitino

World's most powerful open data catalog for building a high-performance, geo-distributed and federated metadata lake.
https://gravitino.apache.org
Apache License 2.0
1.1k stars 345 forks source link

[Bug report]Case sensitive issue when list schema in iceberg catalog #1853

Closed danhuawang closed 3 months ago

danhuawang commented 9 months ago

Version

main branch

Describe what's wrong

截屏2024-02-01 18 40 28 截屏2024-02-01 18 40 58

Error message and/or stacktrace

N/A

How to reproduce

  1. create schema with upper case name in api
  2. create table under the schema in trino, fill in the api path parameter with the schema lower case name
  3. list schema in api
  4. show schemas under catalog in trino
  5. the result are different

Additional context

No response

yuqi1129 commented 9 months ago

create schema with upper case name create table under the schema, fill in the api path parameter with the schema lower case name list schema in api show schemas under catalog in trino the result are different

@danhuawang Can you give detailed information and steps so that I can reproduce it locally, I have checked the steps you mentioned, but everything seems fine for me.

danhuawang commented 9 months ago
  • create schema with upper case name in api
  • create table under the schema in trino, fill in the api path parameter with the schema lower case name

create schema with upper case name in api create table under the schema in trino, using the schema lower case name

yuqi1129 commented 9 months ago

@danhuawang This is a bug caused by Trino and Iceberg.

  1. Create an upper case schema (database) through Gravitino server.
  2. Using the database in Trino, Trino is case-insensitive, so it will display the name as lowercase.
  3. Creating a table under this schema through Trino, then
  4. The Gravitino server will Iceberg API to create table, the database passed to the Gravitino server is lowercase, due to the logic image and MySQL is case-insensitive in like condition, so it will call the following code to create table
image

Then Iceberg will create a new schema in lowercase.

yuqi1129 commented 9 months ago

@FANNG1 We should work together to resolve this issue.

danhuawang commented 9 months ago

Another scenerio: rename catalog name from lower case to upper case in UI, then go to trino to create schema ,encounter catalog not exist issue.

trino> show catalogs; Catalog

gravitino
jmx
memory
metalake_demo.CATALOG_HIVE metalake_demo.ICE1
system
tpcds
tpch
(8 rows)

Query 20240204_111303_00013_h2i2r, FINISHED, 1 node Splits: 19 total, 19 done (100.00%) 0.05 [0 rows, 0B] [0 rows/s, 0B/s]

trino> CREATE SCHEMA "metalake_demo.CATALOG_HIVE".database_01; Query 20240204_111759_00014_h2i2r failed: line 1:1: Catalog 'metalake_demo.catalog_hive' does not exist CREATE SCHEMA "metalake_demo.CATALOG_HIVE".database_01

trino> CREATE SCHEMA "metalake_demo.catalog_hive".database_01; Query 20240204_111848_00015_h2i2r failed: line 1:1: Catalog 'metalake_demo.catalog_hive' does not exist CREATE SCHEMA "metalake_demo.catalog_hive".database_01

yuqi1129 commented 7 months ago

This issue is because Trino will convert all schema names to lower case and there will be case-related issues between Gravitino and Trino. Since it's Trino's internal logic, I'm afraid it can't be fixed in a short time.

What's your opinion? @diqiu50

diqiu50 commented 3 months ago

We don't need to fix it. When users are using Trino, they should be aware of this issue when naming tables or schemas