apache / shardingsphere

Distributed SQL transaction & query engine for data sharding, scaling, encryption, and more - on any database.
Apache License 2.0
19.8k stars 6.7k forks source link

5.0.0-RC1 The Springboot project failed to start:java.lang.IllegalArgumentException: Cannot support database type 'MySQL' #12345

Closed AdelaHong closed 1 year ago

AdelaHong commented 3 years ago

java.lang.IllegalArgumentException: Cannot support database type 'MySQL'

RaigorJiang commented 3 years ago

Hi @AdelaHong Please provide more detail info.

AdelaHong commented 3 years ago

021-09-10 17:35:25.466 [main] INFO com.im.pps.account.biz.PpsAccountApplication - ========== StartApplication启动成功 ========== 2021-09-10 17:35:26.085 [RMI TCP Connection(5)-192.168.88.101] INFO o.a.c.core.ContainerBase.[Tomcat].[localhost].[/] - Initializing Spring FrameworkServlet 'dispatcherServlet' 2021-09-10 17:35:26.086 [RMI TCP Connection(5)-192.168.88.101] INFO org.springframework.web.servlet.DispatcherServlet - FrameworkServlet 'dispatcherServlet': initialization started 2021-09-10 17:35:26.126 [RMI TCP Connection(5)-192.168.88.101] INFO org.springframework.web.servlet.DispatcherServlet - FrameworkServlet 'dispatcherServlet': initialization completed in 40 ms 2021-09-10 17:35:46.495 [RMI TCP Connection(4)-192.168.88.101] WARN o.s.boot.actuate.jdbc.DataSourceHealthIndicator - DataSource health check failed java.lang.IllegalArgumentException: Cannot support database type 'MySQL' at com.google.common.base.Preconditions.checkArgument(Preconditions.java:216) at org.apache.shardingsphere.sql.parser.core.database.parser.DatabaseTypedSQLParserFacadeRegistry.getFacade(DatabaseTypedSQLParserFacadeRegistry.java:50) at org.apache.shardingsphere.sql.parser.core.database.parser.SQLParserExecutor.twoPhaseParse(SQLParserExecutor.java:55) at org.apache.shardingsphere.sql.parser.core.database.parser.SQLParserExecutor.parse(SQLParserExecutor.java:47) at org.apache.shardingsphere.sql.parser.api.SQLParserEngine.parse(SQLParserEngine.java:51) at org.apache.shardingsphere.infra.parser.sql.SQLStatementParserExecutor.parse(SQLStatementParserExecutor.java:47) at org.apache.shardingsphere.infra.parser.sql.SQLStatementParserEngine.parse(SQLStatementParserEngine.java:48) at org.apache.shardingsphere.infra.parser.ShardingSphereSQLParserEngine.parse0(ShardingSphereSQLParserEngine.java:67) at org.apache.shardingsphere.infra.parser.ShardingSphereSQLParserEngine.parse(ShardingSphereSQLParserEngine.java:56) at org.apache.shardingsphere.driver.jdbc.core.statement.ShardingSphereStatement.createLogicSQL(ShardingSphereStatement.java:364) at org.apache.shardingsphere.driver.jdbc.core.statement.ShardingSphereStatement.createExecutionContext(ShardingSphereStatement.java:372) at org.apache.shardingsphere.driver.jdbc.core.statement.ShardingSphereStatement.executeQuery(ShardingSphereStatement.java:146) at org.springframework.jdbc.core.JdbcTemplate$1QueryStatementCallback.doInStatement(JdbcTemplate.java:433) at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:376) at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:446) at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:456) at org.springframework.boot.actuate.jdbc.DataSourceHealthIndicator.doDataSourceHealthCheck(DataSourceHealthIndicator.java:114) at org.springframework.boot.actuate.jdbc.DataSourceHealthIndicator.doHealthCheck(DataSourceHealthIndicator.java:104) at org.springframework.boot.actuate.health.AbstractHealthIndicator.health(AbstractHealthIndicator.java:84) at org.springframework.boot.actuate.health.CompositeHealthIndicator.health(CompositeHealthIndicator.java:68) at org.springframework.boot.actuate.health.CompositeHealthIndicator.health(CompositeHealthIndicator.java:68) at org.springframework.boot.actuate.health.HealthEndpoint.health(HealthEndpoint.java:47) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:483) at org.springframework.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:223) at org.springframework.boot.actuate.endpoint.invoke.reflect.ReflectiveOperationInvoker.invoke(ReflectiveOperationInvoker.java:76) at org.springframework.boot.actuate.endpoint.annotation.AbstractDiscoveredOperation.invoke(AbstractDiscoveredOperation.java:61) at org.springframework.boot.actuate.endpoint.jmx.EndpointMBean.invoke(EndpointMBean.java:102) at org.springframework.boot.actuate.endpoint.jmx.EndpointMBean.invoke(EndpointMBean.java:93) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:819) at com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:801) at javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1466) at javax.management.remote.rmi.RMIConnectionImpl.access$300(RMIConnectionImpl.java:76) at javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1307) at javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1399) at javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:828) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:483) at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:323) at sun.rmi.transport.Transport$1.run(Transport.java:178) at sun.rmi.transport.Transport$1.run(Transport.java:175) at java.security.AccessController.doPrivileged(Native Method) at sun.rmi.transport.Transport.serviceCall(Transport.java:174) at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:557) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:812) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:671) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 2021-09-10 17:35:46.626 [RMI TCP Connection(4)-192.168.88.101] INFO io.lettuce.core.EpollProvider - Starting without optional epoll library 2021-09-10 17:35:46.631 [RMI TCP Connection(4)-192.168.88.101] INFO io.lettuce.core.KqueueProvider - Starting without optional kqueue library

AdelaHong commented 3 years ago

ServiceLoader.load(DatabaseTypedSQLParserFacade.class)
the content obtained by this code is null
`@NoArgsConstructor(access = AccessLevel.PRIVATE) public final class DatabaseTypedSQLParserFacadeRegistry {

private static final Map<String, DatabaseTypedSQLParserFacade> FACADES = new HashMap<>();

static {
    for (DatabaseTypedSQLParserFacade each : ServiceLoader.load(DatabaseTypedSQLParserFacade.class)) {
        FACADES.put(each.getDatabaseType(), each);
    }
}

/**
 * Get database type based SQL parser facade.
 * 
 * @param databaseType database type
 * @return database type based SQL parser facade
 */
public static DatabaseTypedSQLParserFacade getFacade(final String databaseType) {
    Preconditions.checkArgument(FACADES.containsKey(databaseType), "Cannot support database type '%s'", databaseType);
    return FACADES.get(databaseType);
}

}`

RaigorJiang commented 3 years ago

@AdelaHong

AdelaHong commented 3 years ago
org.apache.shardingsphere shardingsphere-jdbc-core 5.0.0-RC1-SNAPSHO
AdelaHong commented 3 years ago

`

org.apache.shardingsphere
        <artifactId>shardingsphere-jdbc-core</artifactId>
        <version>5.0.0-RC1-SNAPSHO</version>
    </dependency>`
AdelaHong commented 3 years ago

When I first used version 5.0.1-beta, it appeared: Cannot find strategy for generate keys; After downloading the master version, there appeared: java.lang.IllegalArgumentException: Cannot support database type'MySQL'

RaigorJiang commented 3 years ago

<dependency> <groupId>org.apache.shardingsphere</groupId> <artifactId>shardingsphere-jdbc-core</artifactId> <version>5.0.0-RC1-SNAPSHO</version> </dependency>

SNAPSHO? Is it a spelling mistake? Since you are using springboot, you should consider using shardingsphere springboot starter. Please refer to the document and example

AdelaHong commented 3 years ago

Download the shardingsphere master branch, and the packaged version number is: 5.0.0-RC1-SNAPSHOT. After the spring boot project is imported `

org.apache.shardingsphere
        <artifactId>shardingsphere-jdbc-core-spring-boot-starter</artifactId>
        <version>5.0.0-RC1-SNAPSHOT</version>
    </dependency>
    <dependency>
        <groupId>org.mybatis.spring.boot</groupId>
        <artifactId>mybatis-spring-boot-starter</artifactId>
    </dependency>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-test</artifactId>
    </dependency>`

, it still reports an error after the project starts successfully: java.lang.IllegalArgumentException: Cannot support database type 'MySQL'

Specify the database type as MySQL `public DataSource shardDataSource(DataSource writeDataSource, DataSource readDataSource) throws Exception {

    Map<String,DataSource> targetDataSources = new HashMap<>();
    targetDataSources.put("ds_write", writeDataSource);
    targetDataSources.put("ds_read", readDataSource);

    return ShardingSphereDataSourceFactory.createDataSource(
            targetDataSources,
            Arrays.asList(
                    shardingConfigurationBuilder.createShardingRuleConfiguration(),
                    createReadwriteSplittingConfiguration(),
                    shardingConfigurationBuilder.createEncryptRuleConfiguration())
            , new Properties());
}`

image

RaigorJiang commented 3 years ago

Can you run the example project so that you can confirm if there is a problem with your environment. https://github.com/apache/shardingsphere/tree/master/examples/shardingsphere-jdbc-example/sharding-example

AdelaHong commented 3 years ago

There is no problem with the environment, the 5.0.1-beta version starts normally, but the switch to 5.0.0-RC1-SNAPSHOT version starts an error

RaigorJiang commented 3 years ago

There is no problem with the environment, the 5.0.1-beta version starts normally, but the switch to 5.0.0-RC1-SNAPSHOT version starts an error

OK, so could you provide your sharding configuration?

AdelaHong commented 3 years ago

datasource configuration:

`datasource.c3p0.write.jdbcUrl = jdbc:mysql://127.0.0.1:3306/test?useUnicode=true&characterEncoding=utf8&autoReconnect=true&failOverReadOnly=false&serverTimezone=GMT%2B8 datasource.c3p0.write.user = root datasource.c3p0.write.password = 123456 datasource.c3p0.write.driverClass = com.mysql.jdbc.Driver datasource.c3p0.write.minPoolSize = 3 datasource.c3p0.write.maxPoolSize = 100 datasource.c3p0.write.maxIdleTime = 3600 datasource.c3p0.write.acquireIncrement = 3 datasource.c3p0.write.maxStatements = 1000 datasource.c3p0.write.initialPoolSize = 3 datasource.c3p0.write.idleConnectionTestPeriod = 60 datasource.c3p0.write.acquireRetryAttempts = 3 datasource.c3p0.write.acquireRetryDelay = 1000 datasource.c3p0.write.breakAfterAcquireFailure = false datasource.c3p0.write.testConnectionOnCheckout = false datasource.c3p0.write.testConnectionOnCheckin = true

增加C3P0数据源配置

datasource.c3p0.read.jdbcUrl = jdbc:mysql://127.0.0.1:3306/test?useUnicode=true&characterEncoding=utf8&autoReconnect=true&failOverReadOnly=false&serverTimezone=GMT%2B8 datasource.c3p0.read.user = root datasource.c3p0.read.password = 123456 datasource.c3p0.read.driverClass = com.mysql.jdbc.Driver datasource.c3p0.read.minPoolSize = 3 datasource.c3p0.read.maxPoolSize = 100 datasource.c3p0.read.maxIdleTime = 3600 datasource.c3p0.read.acquireIncrement = 3 datasource.c3p0.read.maxStatements = 1000 datasource.c3p0.read.initialPoolSize = 3 datasource.c3p0.read.idleConnectionTestPeriod = 60 datasource.c3p0.read.acquireRetryAttempts = 3 datasource.c3p0.read.acquireRetryDelay = 1000 datasource.c3p0.read.breakAfterAcquireFailure = false datasource.c3p0.read.testConnectionOnCheckout = false datasource.c3p0.read.testConnectionOnCheckin = true`

sharding configuration:

`@Configuration public class ShardingConfiguration {

@Bean
@ConfigurationProperties("datasource.c3p0.write")
public DataSource writeDataSource(){
    return DataSourceBuilder.create().type(ComboPooledDataSource.class).build();
}

@Bean
@ConfigurationProperties("datasource.c3p0.read")
public DataSource readDataSource(){
    return DataSourceBuilder.create().type(ComboPooledDataSource.class).build();
}

/**
 * C3P0-0.9.1.2
 * Shard-JDBC 分库配置
 */
@Bean
@DependsOn({"writeDataSource", "readDataSource"})
@Primary
public DataSource shardDataSource(DataSource writeDataSource, DataSource readDataSource) throws Exception {

    Map<String,DataSource> targetDataSources = new HashMap<>();
    targetDataSources.put("ds_write", writeDataSource);
    targetDataSources.put("ds_read", readDataSource);

    return ShardingSphereDataSourceFactory.createDataSource(
            targetDataSources,
            Arrays.asList(
                    createShardingRuleConfiguration(),
                    createReadwriteSplittingConfiguration(),
                    createEncryptRuleConfiguration())
            , new Properties());
}

protected ReadwriteSplittingRuleConfiguration createReadwriteSplittingConfiguration() {
    ReadwriteSplittingDataSourceRuleConfiguration dataSourceConfiguration = new ReadwriteSplittingDataSourceRuleConfiguration(
            "ds_0", "", "ds_write", Arrays.asList("ds_read"), null, false);
    return new ReadwriteSplittingRuleConfiguration(Arrays.asList(dataSourceConfiguration), Collections.emptyMap());
}

protected ShardingRuleConfiguration createShardingRuleConfiguration() {
    ShardingRuleConfiguration shardingRuleConfig = new ShardingRuleConfiguration();

    String actualDataNodes = "ds_0.tf_f_finance_bill_${0.."+ (32 - 1) +"}";

    ShardingTableRuleConfiguration configuration =
            new ShardingTableRuleConfiguration("tf_f_finance_bill", actualDataNodes);
    configuration.setTableShardingStrategy(new StandardShardingStrategyConfiguration("CUSTOMER_ID", MOD_32));
    shardingRuleConfig.getTables().add(configuration);

    return shardingRuleConfig;
}

public EncryptRuleConfiguration createEncryptRuleConfiguration() {
    Map<String, ShardingSphereAlgorithmConfiguration> encryptAlgorithmConfigs = new LinkedHashMap<>();
    Properties encryptProperties = new Properties();
    encryptProperties.setProperty("aes-key-value", "123456");
    encryptAlgorithmConfigs.put("aes", new ShardingSphereAlgorithmConfiguration("AES", encryptProperties));

    Collection<EncryptTableRuleConfiguration> tables = new LinkedList<>();

    Collection<EncryptColumnRuleConfiguration> columns = new LinkedList<>();
    EncryptColumnRuleConfiguration columnRuleConfiguration = new EncryptColumnRuleConfiguration("NUMBER","NUMBER","","","aes");
    columns.add(columnRuleConfiguration);
    EncryptTableRuleConfiguration configuration = new EncryptTableRuleConfiguration("tf_b_bill", columns);
    tables.add(configuration);

    EncryptRuleConfiguration encryptRuleConfiguration = new EncryptRuleConfiguration(tables,encryptAlgorithmConfigs);
    return encryptRuleConfiguration;
}

} `

RaigorJiang commented 3 years ago

Hi @AdelaHong Please do a check: Is shardingsphere-sql-parser-mysql in the dependency list of your project?

RaigorJiang commented 3 years ago

image

AdelaHong commented 3 years ago

shardingsphere-sql-parser-mysql is already dependent image

org.apache.shardingsphere shardingsphere-jdbc-core 5.0.0-RC1-SNAPSHOT
    <dependency>
        <groupId>org.apache.shardingsphere</groupId>
        <artifactId>shardingsphere-sql-parser-mysql</artifactId>
        <version>5.0.0-RC1-SNAPSHOT</version>
    </dependency>

DatabaseTypedSQLParserFacadeRegistry.class

In the following code static { for (DatabaseTypedSQLParserFacade each : ServiceLoader.load(DatabaseTypedSQLParserFacade.class)) { FACADES.put(each.getDatabaseType(), each); } } ServiceLoader.load(DatabaseTypedSQLParserFacade.class) Nothing is loaded, so the FACADES field is empty

RaigorJiang commented 3 years ago

In the following code static { for (DatabaseTypedSQLParserFacade each : ServiceLoader.load(DatabaseTypedSQLParserFacade.class)) { FACADES.put(each.getDatabaseType(), each); } } ServiceLoader.load(DatabaseTypedSQLParserFacade.class) Nothing is loaded, so the FACADES field is empty

Yes, the SPI should not be empty, so I suggest you check the dependencies. How did you build ShardingSphere?

RaigorJiang commented 3 years ago

The name of your dependent packages is not standardized. Did you make any changes?

AdelaHong commented 3 years ago

Because the Cannot find strategy for generate keys problem occurred in the 5.0.0-beta version, check the GitHub that the master branch has fixed this problem, so I downloaded the master branch to the local and renamed it

RaigorJiang commented 3 years ago

Can you rebuild it without making any changes?

AdelaHong commented 3 years ago

Has been rebuilt without any changes, just changed the maven artifactId

RaigorJiang commented 3 years ago

Why you change artifactId?

AdelaHong commented 3 years ago

Just checked, after rebuilding, just repackage the dependencies and use

RaigorJiang commented 3 years ago

Can it work?

Just checked, after rebuilding, just repackage the dependencies and use

AdelaHong commented 3 years ago

No, what I mean is: just rebuild the master branch code, re-type the jar package for dependency reference, and make no other changes

RaigorJiang commented 3 years ago

image

So what is your dependent package name now?

AdelaHong commented 3 years ago

`

org.apache.shardingsphere
        <artifactId>shardingsphere-sql-parser-mysql</artifactId>
        <version>5.0.0-RC1-SNAPSHOT</version>
    </dependency>`
RaigorJiang commented 3 years ago

<dependency> <groupId>org.apache.shardingsphere</groupId> <artifactId>shardingsphere-sql-parser-mysql</artifactId> <version>5.0.0-RC1-SNAPSHOT</version> </dependency>

What about the jar file name.

RaigorJiang commented 3 years ago

Because according to your previous screenshot, the jar package name is not standardized, so maven cannot find the right dependent package.

AdelaHong commented 3 years ago

This is our partial reliance information: OSY)4 N_HX0XXT2N9D(M(K

HoasZhang commented 3 years ago

Why you change artifactId?

We changed the artifactid because we used version 4.0.0-rc1 before

RaigorJiang commented 3 years ago

If it is determined that there is no problem with the environment, it would be strange. Can you compare with the example project and see if there are any differences? Or please upload a demo project to github to help us reproduce the problem.

HoasZhang commented 3 years ago

This is my exception information. I guess the springboot DB health check uses JDBC template, but we use other instead. Similar problems link:https://www.cnblogs.com/maohuidong/p/15006724.html

java.lang.IllegalArgumentException: Cannot support database type 'MySQL' at com.google.common.base.Preconditions.checkArgument(Preconditions.java:216) at org.apache.shardingsphere.sql.parser.core.database.parser.DatabaseTypedSQLParserFacadeRegistry.getFacade(DatabaseTypedSQLParserFacadeRegistry.java:50) at org.apache.shardingsphere.sql.parser.core.database.parser.SQLParserExecutor.twoPhaseParse(SQLParserExecutor.java:55) at org.apache.shardingsphere.sql.parser.core.database.parser.SQLParserExecutor.parse(SQLParserExecutor.java:47) at org.apache.shardingsphere.sql.parser.api.SQLParserEngine.parse(SQLParserEngine.java:51) at org.apache.shardingsphere.infra.parser.sql.SQLStatementParserExecutor.parse(SQLStatementParserExecutor.java:47) at org.apache.shardingsphere.infra.parser.sql.SQLStatementParserEngine.parse(SQLStatementParserEngine.java:48) at org.apache.shardingsphere.infra.parser.ShardingSphereSQLParserEngine.parse0(ShardingSphereSQLParserEngine.java:67) at org.apache.shardingsphere.infra.parser.ShardingSphereSQLParserEngine.parse(ShardingSphereSQLParserEngine.java:56) at org.apache.shardingsphere.driver.jdbc.core.statement.ShardingSphereStatement.createLogicSQL(ShardingSphereStatement.java:364) at org.apache.shardingsphere.driver.jdbc.core.statement.ShardingSphereStatement.createExecutionContext(ShardingSphereStatement.java:372) at org.apache.shardingsphere.driver.jdbc.core.statement.ShardingSphereStatement.executeQuery(ShardingSphereStatement.java:146) at org.springframework.jdbc.core.JdbcTemplate$1QueryStatementCallback.doInStatement(JdbcTemplate.java:433) at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:376) at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:446) at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:456)

RaigorJiang commented 3 years ago

I think It does not look similar. Your exception is because the SPI is not loaded. Since the health check is in the same process, the dependent packages can also be found.

RaigorJiang commented 3 years ago

If it is determined that there is no problem with the environment, it would be strange. Can you compare with the example project and see if there are any differences? Or please upload a demo project to github to help us reproduce the problem.

Could you provide a test project which can reproduce the problem.

HoasZhang commented 3 years ago

We use the following configuration to temporarily solve this problem. Is there a better solution here? We don't want to turn off the health check of DB,and test insert one record。 I think it's because the springboot JDBC template doesn't come under shardingjdbc's supervision

management.health.db.enabled=false

RaigorJiang commented 3 years ago

This is the first time that this problem has been mentioned. Welcome to study and propose a solution together.

We use the following configuration to temporarily solve this problem. Is there a better solution here? We don't want to turn off the health check of DB,and test insert one record。

I think it's because the springboot JDBC template doesn't come under shardingjdbc's supervision management.health.db.enabled=false

HoasZhang commented 3 years ago

We tried to extract a small item, but it could not be reproduced, but our error mainly occurred when the datasourcehealthindicator called and executed select 1. org.springframework.boot.actuate.jdbc.DataSourceHealthIndicator

RaigorJiang commented 3 years ago

Thanks for your feedback, I have logged this issue and will continue to investigate, but it may take a long time.

github-actions[bot] commented 1 year ago

Hello , this issue has not received a reply for several days. This issue is supposed to be closed.

TeslaCN commented 1 year ago

Please use the official release.