apache / iceberg

Apache Iceberg
https://iceberg.apache.org/
Apache License 2.0
6.49k stars 2.24k forks source link

Iceberg metadata table is not created if using MYSQL JDBC catalog and there are existing iceberg meta tables in another database. #11423

Open FANNG1 opened 3 weeks ago

FANNG1 commented 3 weeks ago

Apache Iceberg version

1.6.1 (latest release)

Query engine

Other

Please describe the bug 🐞

If I setup a Jdbc catalog with MYSQL URI like jdbc:mysql://127.0.0.1:3307/db1, it works fines. but when I create another jdbc catalog with different database like jdbc:mysql://127.0.0.1:3307/db2, the query was failed because Iceberg metadata table is not setup in db2. INIT_CATALOG_TABLES_PROPERTY are both enabled.

environment:

  1. MYSQL server: 5.7.44
  2. MYSQL JDBC driver: mysql-connector-java-8.0.15.jar

Willingness to contribute

FANNG1 commented 3 weeks ago

The main problem is we may get table object from other databases for MYSQL, PG works well, and not sure whether other RMDB works well.

            DatabaseMetaData dbMeta = conn.getMetaData();
            ResultSet tableExists =
                dbMeta.getTables(
                    null /* catalog name */,
                    null /* schemaPattern */,
                    JdbcUtil.CATALOG_TABLE_VIEW_NAME /* tableNamePattern */,
                    null /* types */);
            if (tableExists.next()) {
              return true;
            }

There may be two options:

  1. passing database information when get table from connection
  2. not check the existing of the table , use CREATE TABLE IF NOT EXISTS to create table directly.

I prefer to use Option2 for it is more extensible for other RDMS and the performance decrease may minor