tabular-io / iceberg-rest-image

Simple project to expose a catalog over REST using a Java catalog backend
Apache License 2.0
93 stars 39 forks source link

No suitable driver found for jdbc:mysql #85

Open dongsupkim-onepredict opened 4 months ago

dongsupkim-onepredict commented 4 months ago

Hi. I'm attempting to use a MySQL database instead of SQLite, which is the default for the catalog database. However, it appears that there is no MySQL driver class available, as shown below in the Apache Iceberg documentation:

Class.forName("com.mysql.cj.jdbc.Driver"); // ensure JDBC driver is at runtime classpath
Map<String, String> properties = new HashMap<>();
properties.put(CatalogProperties.CATALOG_IMPL, JdbcCatalog.class.getName());
properties.put(CatalogProperties.URI, "jdbc:mysql://localhost:3306/test");
properties.put(JdbcCatalog.PROPERTY_PREFIX + "user", "admin");
properties.put(JdbcCatalog.PROPERTY_PREFIX + "password", "pass");
properties.put(CatalogProperties.WAREHOUSE_LOCATION, "s3://warehouse/path");
Configuration hadoopConf = new Configuration(); // configs if you use HadoopFileIO
JdbcCatalog catalog = CatalogUtil.buildIcebergCatalog("test_jdbc_catalog", properties, hadoopConf);

When I changed the environment like this: "CATALOG_URI: "jdbc:mysql://10.10.50.182/lakehouse", the logs output is as follows:

Exception in thread "main" org.apache.iceberg.jdbc.UncheckedSQLException: Failed to connect: 
jdbc:mysql://10.10.50.182/lakehouse
    at org.apache.iceberg.jdbc.JdbcClientPool.newClient(JdbcClientPool.java:57)
    at org.apache.iceberg.jdbc.JdbcClientPool.newClient(JdbcClientPool.java:30)
    at org.apache.iceberg.ClientPoolImpl.get(ClientPoolImpl.java:125)
    at org.apache.iceberg.ClientPoolImpl.run(ClientPoolImpl.java:56)
    at org.apache.iceberg.ClientPoolImpl.run(ClientPoolImpl.java:51)
    at org.apache.iceberg.jdbc.JdbcCatalog.initializeCatalogTables(JdbcCatalog.java:155)
    at org.apache.iceberg.jdbc.JdbcCatalog.initialize(JdbcCatalog.java:141)
    at org.apache.iceberg.CatalogUtil.loadCatalog(CatalogUtil.java:255)
    at org.apache.iceberg.CatalogUtil.buildIcebergCatalog(CatalogUtil.java:309)
    at org.apache.iceberg.rest.RESTCatalogServer.backendCatalog(RESTCatalogServer.java:86)
    at org.apache.iceberg.rest.RESTCatalogServer.main(RESTCatalogServer.java:90)
Caused by: java.sql.SQLException: No suitable driver found for jdbc:mysql://10.10.50.182/lakehouse
    at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:706)
    at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:190)
    at org.apache.iceberg.jdbc.JdbcClientPool.newClient(JdbcClientPool.java:55)
    ... 10 more

Do you have any plans to address this issue? Thank you.

nastra commented 2 months ago

@dongsupkim-onepredict you would have to build your own Docker image and include the MySQL JDBC Driver as a dependency.