An error occurs after use shadingsphere-jdbc

lcjing commented 5 months ago

Question

For English only, other languages will not accept.

Before asking a question, make sure you have:

Googled your question.
Searched open and closed GitHub issues.
Read documentation: ShardingSphere Doc.

Please pay attention on issues you submitted, because we maybe need more details. If no response anymore and we cannot reproduce it on current information, we will close it.

Using ShardingJDBC for table partitioning，2 datasources：db_test，db_demo.

The tables that need to be split are under db_test

Environment

ShardingJDBC version 5.2.1 Spring 4.3.16 Mybatis 3.2.7 MySQL

Problem

the SQL is：

<select id="queryTaskListByTaskStatus" resultMap="BaseResultMap" parameterType="java.util.Map">
    SELECT
    <include refid="Base_Column_List"/>
    FROM db_demo.t_act_notify_task
    WHERE task_status = #{task_status}
</select>

when execute this SQL，An error occur，error message details：

org.mybatis.spring.MyBatisSystemException: nested exception is org.apache.ibatis.executor.ExecutorException: Error preparing statement.  Cause: org.apache.shardingsphere.dialect.exception.syntax.database.UnknownDatabaseException
    at org.mybatis.spring.MyBatisExceptionTranslator.translateExceptionIfPossible(MyBatisExceptionTranslator.java:75)
    at org.mybatis.spring.SqlSessionTemplate$SqlSessionInterceptor.invoke(SqlSessionTemplate.java:371)
    at com.sun.proxy.$Proxy45.selectList(Unknown Source)
    at org.mybatis.spring.SqlSessionTemplate.selectList(SqlSessionTemplate.java:198)
    at com.user.dao.AcctNotifyTaskDao.queryTaskListByTaskStatus(AcctNotifyTaskDao.java:133)
    at com.user.facade.impl.AcctNotifyTaskFacadeImpl.queryTaskListByTaskStatus(AcctNotifyTaskFacadeImpl.java:86)
    at org.apache.dubbo.common.bytecode.Wrapper102.invokeMethod(Wrapper102.java)
    at org.apache.dubbo.rpc.proxy.javassist.JavassistProxyFactory$1.doInvoke(JavassistProxyFactory.java:47)
    at org.apache.dubbo.rpc.proxy.AbstractProxyInvoker.invoke(AbstractProxyInvoker.java:84)
    at org.apache.dubbo.config.invoker.DelegateProviderMetaDataInvoker.invoke(DelegateProviderMetaDataInvoker.java:56)
    at org.apache.dubbo.rpc.protocol.InvokerWrapper.invoke(InvokerWrapper.java:56)
    at org.apache.dubbo.rpc.protocol.FilterNode.invoke(FilterNode.java:61)
    at org.apache.dubbo.rpc.protocol.FilterNode.invoke(FilterNode.java:61)
    at org.apache.dubbo.rpc.protocol.dubbo.filter.TraceFilter.invoke(TraceFilter.java:77)
    at org.apache.dubbo.rpc.protocol.FilterNode.invoke(FilterNode.java:61)
    at org.apache.dubbo.rpc.filter.TimeoutFilter.invoke(TimeoutFilter.java:46)
    at org.apache.dubbo.rpc.protocol.FilterNode.invoke(FilterNode.java:61)
    at org.apache.dubbo.monitor.support.MonitorFilter.invoke(MonitorFilter.java:91)
    at org.apache.dubbo.rpc.protocol.FilterNode.invoke(FilterNode.java:61)
    at com.user.framework.core.dubbo.filter.ExceptionFilter.invoke(ExceptionFilter.java:36)
    at org.apache.dubbo.rpc.protocol.FilterNode.invoke(FilterNode.java:61)
    at org.apache.dubbo.rpc.protocol.FilterNode.invoke(FilterNode.java:61)
    at org.apache.dubbo.rpc.filter.GenericFilter.invoke(GenericFilter.java:192)
    at org.apache.dubbo.rpc.protocol.FilterNode.invoke(FilterNode.java:61)
    at org.apache.dubbo.rpc.filter.ClassLoaderFilter.invoke(ClassLoaderFilter.java:38)
    at org.apache.dubbo.rpc.protocol.FilterNode.invoke(FilterNode.java:61)
    at org.apache.dubbo.rpc.filter.EchoFilter.invoke(EchoFilter.java:41)
    at org.apache.dubbo.rpc.protocol.FilterNode.invoke(FilterNode.java:61)
    at com.alibaba.dubbo.rpc.Invoker$CompatibleInvoker.invoke(Invoker.java:55)
    at com.dynatrace.oneagent.sdk.dubbo.DynatraceProviderFilter.invoke(DynatraceProviderFilter.java:74)
    at com.alibaba.dubbo.rpc.Filter.invoke(Filter.java:29)
    at org.apache.dubbo.rpc.protocol.FilterNode.invoke(FilterNode.java:61)
    at org.apache.dubbo.rpc.filter.ContextFilter.invoke(ContextFilter.java:129)
    at org.apache.dubbo.rpc.protocol.FilterNode.invoke(FilterNode.java:61)
    at org.apache.dubbo.rpc.protocol.dubbo.DubboProtocol$1.reply(DubboProtocol.java:148)
    at org.apache.dubbo.remoting.exchange.support.header.HeaderExchangeHandler.handleRequest(HeaderExchangeHandler.java:100)
    at org.apache.dubbo.remoting.exchange.support.header.HeaderExchangeHandler.received(HeaderExchangeHandler.java:175)
    at org.apache.dubbo.remoting.transport.DecodeHandler.received(DecodeHandler.java:51)
    at org.apache.dubbo.remoting.transport.dispatcher.ChannelEventRunnable.run(ChannelEventRunnable.java:57)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at org.apache.dubbo.common.threadlocal.InternalRunnable.run(InternalRunnable.java:41)
    at java.lang.Thread.run(Thread.java:750)
Caused by: org.apache.ibatis.executor.ExecutorException: Error preparing statement.  Cause: org.apache.shardingsphere.dialect.exception.syntax.database.UnknownDatabaseException
    at org.apache.ibatis.executor.statement.BaseStatementHandler.prepare(BaseStatementHandler.java:94)
    at org.apache.ibatis.executor.statement.RoutingStatementHandler.prepare(RoutingStatementHandler.java:57)
    at org.apache.ibatis.executor.BatchExecutor.doQuery(BatchExecutor.java:85)
    at org.apache.ibatis.executor.BaseExecutor.queryFromDatabase(BaseExecutor.java:267)
    at org.apache.ibatis.executor.BaseExecutor.query(BaseExecutor.java:137)
    at org.apache.ibatis.executor.CachingExecutor.query(CachingExecutor.java:96)
    at org.apache.ibatis.executor.CachingExecutor.query(CachingExecutor.java:77)
    at sun.reflect.GeneratedMethodAccessor337.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.apache.ibatis.plugin.Invocation.proceed(Invocation.java:49)
    at com.core.ibatis.PageHelper.intercept(PageHelper.java:92)
    at org.apache.ibatis.plugin.Plugin.invoke(Plugin.java:60)
    at com.sun.proxy.$Proxy58.query(Unknown Source)
    at org.apache.ibatis.session.defaults.DefaultSqlSession.selectList(DefaultSqlSession.java:108)
    at org.apache.ibatis.session.defaults.DefaultSqlSession.selectList(DefaultSqlSession.java:102)
    at sun.reflect.GeneratedMethodAccessor338.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.mybatis.spring.SqlSessionTemplate$SqlSessionInterceptor.invoke(SqlSessionTemplate.java:358)
    ... 44 common frames omitted
Caused by: org.apache.shardingsphere.dialect.exception.syntax.database.UnknownDatabaseException: null
    at org.apache.shardingsphere.infra.binder.statement.dml.SelectStatementContext.lambda$getSchemas$0(SelectStatementContext.java:138)
    at org.apache.shardingsphere.infra.util.exception.ShardingSpherePreconditions.checkNotNull(ShardingSpherePreconditions.java:55)
    at org.apache.shardingsphere.infra.binder.statement.dml.SelectStatementContext.getSchemas(SelectStatementContext.java:138)
    at org.apache.shardingsphere.infra.binder.statement.dml.SelectStatementContext.<init>(SelectStatementContext.java:116)
    at org.apache.shardingsphere.infra.binder.SQLStatementContextFactory.getDMLStatementContext(SQLStatementContextFactory.java:163)
    at org.apache.shardingsphere.infra.binder.SQLStatementContextFactory.newInstance(SQLStatementContextFactory.java:146)
    at org.apache.shardingsphere.infra.binder.SQLStatementContextFactory.newInstance(SQLStatementContextFactory.java:131)
    at org.apache.shardingsphere.driver.jdbc.core.statement.ShardingSpherePreparedStatement.<init>(ShardingSpherePreparedStatement.java:196)
    at org.apache.shardingsphere.driver.jdbc.core.statement.ShardingSpherePreparedStatement.<init>(ShardingSpherePreparedStatement.java:160)
    at org.apache.shardingsphere.driver.jdbc.core.connection.ShardingSphereConnection.prepareStatement(ShardingSphereConnection.java:91)
    at org.apache.ibatis.executor.statement.PreparedStatementHandler.instantiateStatement(PreparedStatementHandler.java:75)
    at org.apache.ibatis.executor.statement.BaseStatementHandler.prepare(BaseStatementHandler.java:85)
    ... 63 common frames omitted

but when I remove the database name db_demo，the SQL can execute normally, such as:

<select id="queryTaskListByTaskStatus" resultMap="BaseResultMap" parameterType="java.util.Map">
    SELECT
    <include refid="Base_Column_List"/>
    FROM t_act_notify_task
    WHERE task_status = #{task_status}
</select>

Because the SQL writing in the project specifies the database name, it is not convenient to modify it. Is there any other solution

Configuration

<sharding:sharding-algorithm id="userLogShardingAlgorithm" type="complex">
    <props>
        <prop key="algorithm-class-name">com.user.sharding.algorithm.UserLogShardingAlgorithm</prop>
    </props>
</sharding:sharding-algorithm>

<sharding:complex-strategy id="userLogSharding" sharding-columns="create_time,login_no" algorithm-ref="userLogShardingAlgorithm"/>

<sharding:rule id="shardingRule">
    <sharding:table-rules>
        <sharding:table-rule logic-table="t_user_log" actual-data-nodes="db_test.t_use_log_${2024..2030}" table-strategy-ref="userLogSharding"/>
    </sharding:table-rules>
</sharding:rule>

<shardingsphere:data-source id="shardingDataSourceMaster" data-source-names="testDataSource, demoDataSource" rule-refs="shardingRule">
    <props>
        <prop key="sql-show">true</prop>
        <prop key="check.table.metadata.enabled">false</prop>
    </props>
</shardingsphere:data-source>

jdbc.username=dev
jdbc.password=
jdbc.driver=com.p6spy.engine.spy.P6SpyDriver
jdbc.url=jdbc:p6spy:mysql://127.0.0.1:3306/db_test?autoReconnect=true&useUnicode=true&characterEncoding=UTF-8

task.jdbc.username=dev
task.jdbc.password=
task.jdbc.driver=com.p6spy.engine.spy.P6SpyDriver
task.jdbc.url=jdbc:p6spy:mysql://127.0.0.1:3306/db_demo?autoReconnect=true&useUnicode=true&characterEncoding=UTF-8

zzyReal666 commented 5 months ago

Perhaps you can try the new version, mysql support has been greatly improved.

lcjing commented 5 months ago

Do you mean upgrading to the ShardingJDBC version?

zzyReal666 commented 5 months ago

Do you mean upgrading to the ShardingJDBC version?

yeah.You can try version 5.5.0 or 5.5.1

lcjing commented 5 months ago

I switched to version 5.5.0, but encountered another issue where the data source driver failed to load. I tried to troubleshoot the issue and found the following，org.apache.shardingsphere.infra.datasource.pool.props.creator.DataSourcePoolPropertiesCreator#create(javax.sql.DataSource) this dataSourceClassName is null，does not support this driver or is there a configuration issue，configuartion such as: dataSources: db_test: dataSourceClassName: org.apache.tomcat.jdbc.pool.DataSource driverClassName: com.p6spy.engine.spy.P6SpyDriver url: jdbc:p6spy:mysql://127.0.0.1:3306?autoReconnect=true&useUnicode=true&characterEncoding=UTF-8 username: password:

lcjing commented 5 months ago

IMG20240607-165840264 IMG20240607-170420548

zzyReal666 commented 5 months ago

I'm not sure what the impact of using the P6Spy framework will be, but you can change it to real driver test SQL first.

lcjing commented 5 months ago

It's not a configuration issue. I checked the source code and found the problem. Why is the driverClassName property that I configured ignored here? Is it a bug?

public final class DataSourcePoolReflection {
    static {
            GENERAL_CLASS_TYPES = new HashSet<>(
                    Arrays.asList(boolean.class, Boolean.class, int.class, Integer.class, long.class, Long.class, String.class, Collection.class, List.class, Properties.class));
            SKIPPED_PROPERTY_KEYS = new HashSet<>(Arrays.asList("loginTimeout", "driverClassName"));
        }

    ...   
}

linghengqian commented 5 months ago

This is not a bug. Currently, all implementations of the org.apache.shardingsphere.infra.database.core.type.DatabaseType class in ShardingSphere only determine the prefix of the jdbcUrl to search for implementations that parse SQL dialects and query Database metadata.
What you actually need to do is to implement a org.apache.shardingsphere.infra.database.core.type.DatabaseType class that matches jdbc:p6spy:mysql:, and then implement the following methods to let ShardingSphere know that the SQL dialect needs to be parsed and MySQL Database metadata needs to be queried.
```
import org.apache.shardingsphere.infra.database.core.type.DatabaseType;
import org.apache.shardingsphere.infra.spi.type.typed.TypedSPILoader;
```

import java.util.Collection; import java.util.Collections; import java.util.Optional;

/**

Database type of MySQL under P6Spy. */ public final class P6spyMySQLDatabaseType implements DatabaseType {

@Override public Collection getJdbcUrlPrefixes() { return Collections.singletonList("jdbc:p6spy:mysql:"); }

@Override public Optional getTrunkDatabaseType() { return Optional.of(TypedSPILoader.getService(DatabaseType.class, "MySQL")); }

@Override public String getType() { return "P6spyMySQL"; } }


- You also need to register with SPI in `org.apache.shardingsphere.infra.database.core.type.DatabaseType` under `classpath`(`src/main/resources/META-INF/services`). 
- You might actually consider submitting a PR for this, since you seem to be the first person to use both P6Spy and ShardingSphere together. The design looks similar to the optional module `org.apache.shardingsphere:shardingsphere-infra-database-testcontainers`.

linghengqian commented 5 months ago

Despite this, since you did not provide any git repository with unit tests, the bug in this issue seems rather mysterious. You can create a MySQL instance using testcontainers-java in your unit tests.

lcjing commented 5 months ago

You don't seem to understand what I mean, it's not because of the issue with driver p6spy, it may be due to connection pooling. The connection pool used in my project is org.apache.tomcat.jdbc.pool.DataSource， The root cause is that when setting a property in the DataSourcePoolReflectionclass, the driverClassName property will be skipped, resulting in an invalid driverClassName configured in yaml. Subsequently, A NullPointerException occurred when creating a connection in org.apache.tomcat.jdbc.pool.PooledConnection, as follows:

protected void connectUsingDriver() throws SQLException {

    try {
        if (driver==null) {
            if (log.isDebugEnabled()) {
                log.debug("Instantiating driver using class: "+poolProperties.getDriverClassName()+" [url="+poolProperties.getUrl()+"]");
            }
            driver = (java.sql.Driver) Class.forName(poolProperties.getDriverClassName(),
                                                     true, PooledConnection.class.getClassLoader()
                                                    ).newInstance();
        }
    } catch (java.lang.Exception cn) {
        if (log.isDebugEnabled()) {
            log.debug("Unable to instantiate JDBC driver.", cn);
        }
        SQLException ex = new SQLException(cn.getMessage());
        ex.initCause(cn);
        throw ex;
    }
    ......
}

I don't know why ShardingJDBC configured to ignore the driverClassName property when setting properties:

SKIPPED_PROPERTY_KEYS = new HashSet<>(Arrays.asList("loginTimeout", "driverClassName"));

my yaml configuration is sharding-master-datasource.yaml

dataSources:
  db_test:
    dataSourceClassName: org.apache.tomcat.jdbc.pool.DataSource
    driverClassName: com.mysql.jdbc.Driver
    url: jdbc:mysql://localhost:3306?autoReconnect=true&useUnicode=true&characterEncoding=UTF-8
    username: u_dev
    password: 123456

rules:
  - !SHARDING
    tables:
      t_user_log:
        actualDataNodes: db_user.t_user_log_$->{2024..2030}
        tableStrategy:
          complex:
            shardingColumns: create_time, user_log_no
            shardingAlgorithmName: user-log-complex
    shardingAlgorithms:
      user-log-complex:
        type: complex
        props:
          algorithm-class-name: com.user.sharding.algorithm.UserLogShardingAlgorithm
  - !SINGLE
    tables:
      - "*.*" # 加载全部单表
props:
  spl-show: true
  check:
    table:
      metadata:
        enabled: false

spring configuration

<bean id="shardingDataSourceMaster" class="org.springframework.jdbc.datasource.SimpleDriverDataSource">
    <property name="driverClass" value="org.apache.shardingsphere.driver.ShardingSphereDriver" />
    <property name="url" value="jdbc:shardingsphere:classpath:sharding-master-datasource.yaml" />
</bean>

linghengqian commented 5 months ago

If you don't provide an example, it will be impossible to reproduce this completely because it seems that you have already implemented the org.apache.shardingsphere.infra.datasource.pool.metadata.DataSourcePoolFieldMetaData and org.apache.shardingsphere.infra.datasource.pool.metadata.DataSourcePoolMetaData of the tomcat jdbc in ShardingSphere Infra. Refer to https://github.com/apache/shardingsphere-plugin/tree/main/infra%2Fdata-source-pool%2Fdbcp .

lcjing commented 5 months ago

Oh, I didn't implemented the org.apache.shardingsphere.infra.datasource.pool.metadata.DataSourcePoolFieldMetaData and org.apache.shardingsphere.infra.datasource.pool.metadata.DataSourcePoolMetaData of the tomcat jdbc in ShardingSphere Infra。

The above provides all the configuration information

ok, I can implement it and try again

lcjing commented 5 months ago

I try it again and the result is still the same. I don't think you need my example to reproduce it, you can see org.apache.shardingsphere.infra.datasource.pool.creator.DataSourcePoolCreator

/**
 * Create data source.
 *
 * @param props data source pool properties
 * @return created data source
 */
public static DataSource create(final DataSourcePoolProperties props) {
    DataSource result = create(props.getPoolClassName());
    Optional<DataSourcePoolMetaData> poolMetaData = TypedSPILoader.findService(DataSourcePoolMetaData.class, props.getPoolClassName());
    DataSourcePoolReflection dataSourcePoolReflection = new DataSourcePoolReflection(result);
    if (poolMetaData.isPresent()) {
        setDefaultFields(dataSourcePoolReflection, poolMetaData.get());
        setConfiguredFields(props, dataSourcePoolReflection, poolMetaData.get());
        appendJdbcUrlProperties(props.getCustomProperties(), result, poolMetaData.get(), dataSourcePoolReflection);
        dataSourcePoolReflection.addDefaultDataSourcePoolProperties(poolMetaData.get());
    } else {
        setConfiguredFields(props, dataSourcePoolReflection);
    }
    return result;
}

Eventually, it will be called org.apache.shardingsphere.infra.datasource.pool.creator.DataSourcePoolReflection.setField(final String fieldName, final Object fieldValue)

static {
    SKIPPED_PROPERTY_KEYS = new HashSet<>(Arrays.asList("loginTimeout", "driverClassName"));
}

private boolean isSkippedProperty(final String key) {
    return SKIPPED_PROPERTY_KEYS.contains(key);
}

public void setField(final String fieldName, final Object fieldValue) {
    if (null != fieldValue && !isSkippedProperty(fieldName)) {
        findSetterMethod(fieldName).ifPresent(optional -> setField(optional, fieldValue));
    }
}

From the above code, it can be seen that no matter how the driverClassName is configured, the property value will not be set through reflection, and the driverClassName will always be null

linghengqian commented 5 months ago

For DataSourcePoolReflection, I also performed integration tests for specific properties of Alibaba Druid in #30543, and I didn't see any abnormal exceptions thrown. How do you write unit tests?

lcjing commented 5 months ago

What I mean is that the driverClassName property will not be assigned a value! The condition isSkippedProperty (fieldName) is false, so it will not be assigned，the unit tests is simple:

@Test
public void test() {
    DataSource source = new HikariDataSource();
    DataSourcePoolReflection dataSourcePoolReflection = new DataSourcePoolReflection(source);

    dataSourcePoolReflection.setField("jdbcUrl", "jdbc:mysql://localhost:3306/test");
    dataSourcePoolReflection.setField("driverClassName", "com.mysql.cj.jdbc.Driver");

    System.out.println(dataSourcePoolReflection);
}

Breakpoint observation, you can see that jdbcUrl can be assigned normally, but driverClassName does not

linghengqian commented 5 months ago

I would say that this array looks like a old-fashioned design, it seems to have existed before I became a committer. Let me look into it.

lcjing commented 5 months ago

ok, That's great

linghengqian commented 5 months ago

I see it was initially reported by #14613 and then related PR was tagged for the 5.1.1 milestone. This still appears odd because essentially, the issue revolves around the integrated test utilizing MySQL JDBC Driver version 5.6, whereas in #14613, an attempt was made to connect to a MySQL Server using MySQL JDBC Driver version 8.0. I would argue that this seems to be tied to the history of MySQL versions. However, even if we were to revert #15409, there is no logic in org.apache.shardingsphere.infra.database.core.type.DatabaseType to determine the DriverName, simply reverting the PR seems futile.
What do you think? What was your initial error log in ShardingSphere 5.5.0?

lcjing commented 5 months ago

I still don't understand why the design is like this, which means that the driverClassName property I configured in yaml is not useful at all. The detailed error log is as follows:
in fact, the creation of the connection failed due to the driverClassName being null

Caused by: java.lang.RuntimeException: java.sql.SQLException
    at org.apache.tomcat.jdbc.pool.DataSourceProxy.getWaitCount(DataSourceProxy.java:699)
    at sun.reflect.GeneratedMethodAccessor293.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.apache.shardingsphere.infra.datasource.pool.creator.DataSourcePoolReflection.convertToProperties(DataSourcePoolReflection.java:94)
    at org.apache.shardingsphere.infra.datasource.pool.creator.DataSourcePoolReflection.convertToProperties(DataSourcePoolReflection.java:79)
    at org.apache.shardingsphere.infra.datasource.pool.props.creator.DataSourcePoolPropertiesCreator.createProperties(DataSourcePoolPropertiesCreator.java:90)
    at org.apache.shardingsphere.infra.datasource.pool.props.creator.DataSourcePoolPropertiesCreator.create(DataSourcePoolPropertiesCreator.java:65)
    at org.apache.shardingsphere.infra.config.database.impl.DataSourceProvidedDatabaseConfiguration.lambda$createDataSourcePoolPropertiesMap$2(DataSourceProvidedDatabaseConfiguration.java:81)
    at java.util.stream.Collectors.lambda$toMap$58(Collectors.java:1321)
    at java.util.stream.ReduceOps$3ReducingSink.accept(ReduceOps.java:169)
    at java.util.Iterator.forEachRemaining(Iterator.java:116)
    at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
    at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
    at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
    at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
    at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
    at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
    at org.apache.shardingsphere.infra.config.database.impl.DataSourceProvidedDatabaseConfiguration.createDataSourcePoolPropertiesMap(DataSourceProvidedDatabaseConfiguration.java:81)
    at org.apache.shardingsphere.infra.config.database.impl.DataSourceProvidedDatabaseConfiguration.<init>(DataSourceProvidedDatabaseConfiguration.java:54)
    at org.apache.shardingsphere.driver.jdbc.core.datasource.ShardingSphereDataSource.createContextManager(ShardingSphereDataSource.java:77)
    at org.apache.shardingsphere.driver.jdbc.core.datasource.ShardingSphereDataSource.<init>(ShardingSphereDataSource.java:67)
    at org.apache.shardingsphere.driver.api.ShardingSphereDataSourceFactory.createDataSource(ShardingSphereDataSourceFactory.java:95)
    at org.apache.shardingsphere.driver.api.yaml.YamlShardingSphereDataSourceFactory.createDataSource(YamlShardingSphereDataSourceFactory.java:135)
    at org.apache.shardingsphere.driver.api.yaml.YamlShardingSphereDataSourceFactory.createDataSource(YamlShardingSphereDataSourceFactory.java:70)
    at org.apache.shardingsphere.driver.jdbc.core.driver.DriverDataSourceCache.createDataSource(DriverDataSourceCache.java:55)
    at org.apache.shardingsphere.driver.jdbc.core.driver.DriverDataSourceCache.lambda$get$0(DriverDataSourceCache.java:48)
    at java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1660)
    at org.apache.shardingsphere.driver.jdbc.core.driver.DriverDataSourceCache.get(DriverDataSourceCache.java:48)
    at org.apache.shardingsphere.driver.ShardingSphereDriver.connect(ShardingSphereDriver.java:56)
    at org.springframework.jdbc.datasource.SimpleDriverDataSource.getConnectionFromDriver(SimpleDriverDataSource.java:140)
    at org.springframework.jdbc.datasource.AbstractDriverBasedDataSource.getConnectionFromDriver(AbstractDriverBasedDataSource.java:196)
    at org.springframework.jdbc.datasource.AbstractDriverBasedDataSource.getConnection(AbstractDriverBasedDataSource.java:159)
    at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111)
    at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77)
    at org.mybatis.spring.transaction.SpringManagedTransaction.openConnection(SpringManagedTransaction.java:81)
    at org.mybatis.spring.transaction.SpringManagedTransaction.getConnection(SpringManagedTransaction.java:67)
    at org.apache.ibatis.executor.BaseExecutor.getConnection(BaseExecutor.java:279)
    at org.apache.ibatis.executor.BatchExecutor.doQuery(BatchExecutor.java:84)
    at org.apache.ibatis.executor.BaseExecutor.queryFromDatabase(BaseExecutor.java:267)
    at org.apache.ibatis.executor.BaseExecutor.query(BaseExecutor.java:137)
    at org.apache.ibatis.executor.CachingExecutor.query(CachingExecutor.java:96)
    at org.apache.ibatis.executor.CachingExecutor.query(CachingExecutor.java:77)
    at sun.reflect.GeneratedMethodAccessor295.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.apache.ibatis.plugin.Invocation.proceed(Invocation.java:49)
    at com.pay1pay.framework.core.ibatis.PageHelper.intercept(PageHelper.java:92)
    at org.apache.ibatis.plugin.Plugin.invoke(Plugin.java:60)
    at com.sun.proxy.$Proxy51.query(Unknown Source)
    at org.apache.ibatis.session.defaults.DefaultSqlSession.selectList(DefaultSqlSession.java:108)
    ... 48 common frames omitted
Caused by: java.sql.SQLException: null
    at org.apache.tomcat.jdbc.pool.PooledConnection.connectUsingDriver(PooledConnection.java:253)
    at org.apache.tomcat.jdbc.pool.PooledConnection.connect(PooledConnection.java:181)
    at org.apache.tomcat.jdbc.pool.ConnectionPool.createConnection(ConnectionPool.java:699)
    at org.apache.tomcat.jdbc.pool.ConnectionPool.borrowConnection(ConnectionPool.java:633)
    at org.apache.tomcat.jdbc.pool.ConnectionPool.init(ConnectionPool.java:484)
    at org.apache.tomcat.jdbc.pool.ConnectionPool.<init>(ConnectionPool.java:142)
    at org.apache.tomcat.jdbc.pool.DataSourceProxy.pCreatePool(DataSourceProxy.java:115)
    at org.apache.tomcat.jdbc.pool.DataSourceProxy.createPool(DataSourceProxy.java:102)
    at org.apache.tomcat.jdbc.pool.DataSourceProxy.getWaitCount(DataSourceProxy.java:697)
    ... 98 common frames omitted
Caused by: java.lang.NullPointerException: null
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:348)
    at org.apache.tomcat.jdbc.pool.PooledConnection.connectUsingDriver(PooledConnection.java:245)
    ... 106 common frames omitted

linghengqian commented 5 months ago

@lcjing Since you didn't provide unit tests, you need to manually compile the master branch to test it.

sdk install java 22.0.1-graalce
sdk use java 22.0.1-graalce

git clone git@github.com:apache/shardingsphere.git
cd ./shardingsphere/
git reset --hard 6ec8e4290be8db69087b87a5d7e9476a46c84931
./mvnw clean install -Prelease -T1C -DskipTests -Djacoco.skip=true -Dcheckstyle.skip=true -Drat.skip=true -Dmaven.javadoc.skip=true

Then use it directly in your private project.

<dependency>
    <groupId>org.apache.shardingsphere</groupId>
    <artifactId>shardingsphere-jdbc</artifactId>
    <version>5.5.1-SNAPSHOT</version>
</dependency>

lcjing commented 5 months ago

ok， I'll got it try

lcjing commented 5 months ago

Hello, Thank you for addressing this issue, but now I am returning to my initial question. Does ShardingJDBC not support specifying database names with SQL statements? For example:

select * from db_user.t_user_log where user_id = "123";

Executing this SQL statement will result in an error：

Error preparing statement.  Cause: org.apache.shardingsphere.infra.exception.kernel.metadata.TableNotFoundException: Table or view 't_user_log' does not exist.

linghengqian commented 5 months ago

I would say this was intentional, as access to the real tables and real database is deliberately blocked. You can only call SQL on the logical tables and logical database, although the implementation classes of the algorithm SPI can operate on real tables and real databases.
There is a typical example of a logical database in the case of read-write separation, where a logical database is created, but I have not verified whether it can be called upon.

lcjing commented 5 months ago

Oh, Does it mean that my SQL statement cannot specify a database name? Even if I don't divide the database, I only divide the table scene

linghengqian commented 5 months ago

The answer is affirmative. It is only possible to obtain a List of target database names through a custom algorithm implemented via SPI.

lcjing commented 5 months ago

Okay, thank you, but all the SQL statements in my project specify database names, so removing them all poses a high risk of modification

linghengqian commented 5 months ago

There's really nothing that can be done in the current issue.

lcjing commented 4 months ago

ok，I'll close the issue

apache / shardingsphere