apache / shardingsphere

Distributed SQL transaction & query engine for data sharding, scaling, encryption, and more - on any database.
Apache License 2.0
19.89k stars 6.73k forks source link

Multiple metadata OR All tables must be in the same datasource #17890

Closed peng-ay closed 3 months ago

peng-ay commented 2 years ago

version:

<dependency>
    <groupId>org.apache.shardingsphere</groupId>
    <artifactId>shardingsphere-jdbc-core</artifactId>
    <version>5.1.1</version>
</dependency>

question: All tables must be in the same datasource.

This problem is caused by the two tables in different databases

demand: I have multiple different logic libraries,I want to query in the same SQL,But now Sharding loads multiple data sources into a single metadata,Is there a way to get multiple data sources into different metadata. image

TeslaCN commented 2 years ago

Could you should us your rule configurations?

peng-ay commented 2 years ago

Could you should us your rule configurations?

        Map<String, DataSource> dataSourceMap = new HashMap<>();
        dataSourceMap.put("ds", dataSource);
        dataSourceMap.put("dsweixin", dataSourceWeixin);
        dataSourceMap.put("dspay", dataSourcePay);

        //TableRuleConfiguration testTableRuleConfig = new TableRuleConfiguration("t_sharding_jdbc","ds.t_sharding_jdbc_${0..1}");

        List<ShardingTableRuleConfiguration> tableRuleConfigs = getTableRuleConfigs();

        // 配置分片规则
        ShardingRuleConfiguration shardingRuleConfiguration = new ShardingRuleConfiguration();

        for (ShardingTableRuleConfiguration tableRuleConfig : tableRuleConfigs) {
            shardingRuleConfiguration.getTables().add(tableRuleConfig);

        }
        Properties datePreciseShardingAlgorithmProps = new Properties();
        datePreciseShardingAlgorithmProps.setProperty("strategy", "STANDARD");
        datePreciseShardingAlgorithmProps.setProperty("algorithmClassName", monthDateStandardShardingAlgorithm.getClass().getName());
        shardingRuleConfiguration.getShardingAlgorithms().put("monthDateStandardShardingAlgorithm", new ShardingSphereAlgorithmConfiguration(monthDateStandardShardingAlgorithm.getType(), datePreciseShardingAlgorithmProps));

        Properties properties = new Properties();
        properties.setProperty("sql-show", "true");
        properties.setProperty("check-duplicate-table-enabled", "true");
        properties.setProperty("check-table-metadata-enabled ", "true");
        properties.setProperty("max.connections.size.per.query", "200");
        properties.setProperty("sql-federation-enabled", "true");
        // 获取数据源对象
        try {
            //创建sharding数据源
            logger.info("sharding开始创建数据源");
            long time = System.currentTimeMillis();
            DataSource dataSource = ShardingSphereDataSourceFactory.createDataSource(dataSourceMap, Lists.newArrayList(shardingRuleConfiguration), properties);
            logger.info("sharding创建数据源结束,time:{}", (System.currentTimeMillis() - time));
            return dataSource;
        } catch (SQLException throwables) {
            logger.info("ShardingSphereConfig 创建ShardingDataSource失败", throwables);

        }
        return dataSource;
peng-ay commented 2 years ago

@TeslaCN Now it is added “sql-federation-enabled”, there is no “All tables must be in the same datasource”, But appeared ClassCastException, because the two tables of the associated query are in different libraries, so I still want to know how to create multiple metadata, which was supported in 4.1.1 before

TeslaCN commented 2 years ago

@q3056917665 Query tables in different databases required SQL federation. Please show us your SQL and the error after SQL federation enabled. Hi @strongduanmu, could you please take a look at this?

strongduanmu commented 2 years ago

Ok, I will check it. @q3056917665 Can you show the exception stack of sql federation?

peng-ay commented 2 years ago

@q3056917665 Query tables in different databases required SQL federation. Please show us your SQL and the error after SQL federation enabled. Hi @strongduanmu, could you please take a look at this? @strongduanmu

SELECT oc.record_date, SUM(oc.total_count),SUM(current_teacher_active_count) FROMcorp_operate_org_count cooc INNER JOIN system_event_org_count oc ON oc.orgcode = cooc.orgcode AND oc.record_date = cooc.record_date  WHERE cooc.providerid IN ?1 AND cooc.record_date between ?2 AND ?3 AND cooc.is_apply = 0 AND oc.event_configid = 36 GROUP BY oc.record_date 

image

duWubiao commented 2 years ago

Is there a solution to this problem?

github-actions[bot] commented 2 years ago

Hello , this issue has not received a reply for several days. This issue is supposed to be closed.

terrymanu commented 3 months ago

I don't think "library" is a concept related to ShardingSphere. Due to the high cost of understanding, please allow me to close this issue.