apache / shardingsphere

Empowering Data Intelligence with Distributed SQL for Sharding, Scalability, and Security Across All Databases.
Apache License 2.0
19.94k stars 6.74k forks source link

关于在自定义分片代码中使用springBean注入的问题 #31660

Closed evangoe closed 4 months ago

evangoe commented 5 months ago

环境说明

<java.version>17</java.version>
<spring-boot.version>3.0.2</spring-boot.version>
<mybatis.plus.version>3.5.3</mybatis.plus.version>
<mysql.version>8.0.27</mysql.version>

shardingsphere-jdbc:

<dependency>
    <groupId>org.apache.shardingsphere</groupId>
    <artifactId>shardingsphere-jdbc</artifactId>
    <version>5.5.0</version>
    <exclusions>
        <exclusion>
            <groupId>org.apache.shardingsphere</groupId>
            <artifactId>shardingsphere-test-util</artifactId>
        </exclusion>
    </exclusions>
</dependency>

场景、问题: 一些配置

SHARDING.YAML部分内容


rules:
- !SINGLE
  tables:
    - ds_0.*
  defaultDataSource: ds_0
- !SHARDING
  tables:
    saas_file:
      actualDataNodes: ds_0.saas_files_10001_202404,ds_0.saas_files_10001_202405,ds_0.saas_files_10002_202404,ds_0.saas_files_10002_202405
      tableStrategy:
        complex: # 用于多分片键的复合分片场景
          shardingColumns: file_id,tenant_id,file_upload_time # 分片列名称,多个列以逗号分隔
          shardingAlgorithmName: sharding-altorithm # 分片算法名称
  shardingAlgorithms:
    sharding-altorithm:
      type: CLASS_BASED # 分片算法类型
      props:
        strategy: COMPLEX
        algorithmClassName: com.micolor.saas.storage.sharding.ComplexModTableShardAlgorithm
props:
  sql-show: true

COMPLEXMODTABLESHARDALGORITHM一些内容


@Slf4j
@Component
public class ComplexModTableShardAlgorithm implements ComplexKeysShardingAlgorithm {
    @Override
    public Collection<String> doSharding(Collection collection, ComplexKeysShardingValue complexKeysShardingValue) {
        //处理精确查询
        Map<String, List<Object>> mapSharding = complexKeysShardingValue.getColumnNameAndShardingValuesMap();
        //精确查询
        List<Integer> tenantIds = new ArrayList<>();
        List<String> fileIds = new ArrayList<>();
        if (!mapSharding.isEmpty()) {
            for (Map.Entry<String, List<Object>> entry : mapSharding.entrySet()) {
                String key = entry.getKey();
                List<Object> value = entry.getValue();
                if (key.equals("tenant_id")) {
                    tenantIds = value.stream().map(o -> Integer.parseInt(o.toString())).collect(Collectors.toList());
                }
                if (key.equals("file_id")) {
                    fileIds = value.stream().map(Object::toString).collect(Collectors.toList());
                }
            }
        }
        log.info("tenantIds=>{}",tenantIds);
        log.info("fileIds=>{}",fileIds);
        List<String> list = getTableName(tenantIds,fileIds);
        if(list.isEmpty()){
            return collection;
        }else{
            return list;
        }

    }
    @Autowired
    private FileShardingIndexService fileShardingIndexService;
    private List<String> getTableName(List<Integer> tenantIds,List<String> fileIds){
        List<String> tables = new ArrayList<>();
        try{
            List list = fileShardingIndexService.queryFileShardingIndex(tenantIds,fileIds);
            log.info("list==>{}",list);
            for(int i=0;i<list.size();i++){
                //TODO 搞事情....
            }
        }catch (Exception e){
            e.printStackTrace();
        }
        return tables;
    }
}

场景 有个文件表(saas_file),对这个表按照tenant_id,file_upload_time2个字段进行了分表。 同时也有个索引表(saas_file_index)表,保存着fileId(文件编号),tenant_id(商户编号),file_upload_year_month(文件上传年月)3个字段。

实现逻辑:

1当用tenant_id查询,就自动拼,没问题。 2 当用fileuploadtime(范围)查询,也自动拼,没问题。 3(重点) 当用 =fileid 或者 in(fileids) 的时候,就想根据这些fileid先去索引表中查询(使用fileShardingIndexService查询)出对应的tenant_id和file_upload_year_month,然后把结果拼装起来返回去。

问题 如实现逻辑3中描述,势必需要在ComplexModTableShardAlgorithm类中注入fileShardingIndexService,然后查询数据库。但是通过@Autowired注解无法注入。 请教如何在ComplexModTableShardAlgorithm或者类似的类中注入Spring的Bean.

已进行操作: 已经对fileShardingIndexService单独测试(使用spring controller调用),均能正常运行。 也写了个SpringUtils,尝试,依旧无用,代码如下:


@Component
public class SpringUtils implements ApplicationContextAware  {

    private static ApplicationContext applicationContext;

    @Override
    public void setApplicationContext(ApplicationContext applicationContext) throws BeansException {
        SpringUtils.applicationContext = applicationContext;
    }

    public static <T> T getBean(String beanName) {
        if(applicationContext.containsBean(beanName)){
            return (T) applicationContext.getBean(beanName);
        }else{
            return null;
        }
    }

    public static <T> Map<String, T> getBeansOfType(Class<T> baseType) {
        return applicationContext.getBeansOfType(baseType);
    }
}

现状: 当 ComplexModTableShardAlgorithm中需要使用fileShardingIndexService类时,报null,报错信息如下:

java.lang.NullPointerException: Cannot invoke "com.micolor.saas.storage.serivce.FileShardingIndexService.queryFileShardingIndex(java.util.List, java.util.List)" because "this.fileShardingIndexService" is null
    at com.micolor.saas.storage.sharding.ComplexModTableShardAlgorithm.getTableName(ComplexModTableShardAlgorithm.java:58)
    at com.micolor.saas.storage.sharding.ComplexModTableShardAlgorithm.doSharding(ComplexModTableShardAlgorithm.java:45)
    at org.apache.shardingsphere.sharding.algorithm.sharding.classbased.ClassBasedShardingAlgorithm.doSharding(ClassBasedShardingAlgorithm.java:110)
    at org.apache.shardingsphere.sharding.route.strategy.type.complex.ComplexShardingStrategy.doSharding(ComplexShardingStrategy.java:72)
    at org.apache.shardingsphere.sharding.route.engine.type.standard.ShardingStandardRoutingEngine.routeTables(ShardingStandardRoutingEngine.java:274)
    at org.apache.shardingsphere.sharding.route.engine.type.standard.ShardingStandardRoutingEngine.route0(ShardingStandardRoutingEngine.java:253)
    at org.apache.shardingsphere.sharding.route.engine.type.standard.ShardingStandardRoutingEngine.routeByShardingConditionsWithCondition(ShardingStandardRoutingEngine.java:143)
    at org.apache.shardingsphere.sharding.route.engine.type.standard.ShardingStandardRoutingEngine.routeByShardingConditions(ShardingStandardRoutingEngine.java:136)
    at org.apache.shardingsphere.sharding.route.engine.type.standard.ShardingStandardRoutingEngine.getDataNodes(ShardingStandardRoutingEngine.java:103)
    at org.apache.shardingsphere.sharding.route.engine.type.standard.ShardingStandardRoutingEngine.route(ShardingStandardRoutingEngine.java:85)
    at org.apache.shardingsphere.sharding.route.engine.ShardingSQLRouter.createRouteContext0(ShardingSQLRouter.java:73)
    at org.apache.shardingsphere.sharding.route.engine.ShardingSQLRouter.createRouteContext(ShardingSQLRouter.java:61)
    at org.apache.shardingsphere.sharding.route.engine.ShardingSQLRouter.createRouteContext(ShardingSQLRouter.java:48)
    at org.apache.shardingsphere.infra.route.engine.impl.PartialSQLRouteExecutor.route(PartialSQLRouteExecutor.java:71)
    at org.apache.shardingsphere.infra.route.engine.SQLRouteEngine.route(SQLRouteEngine.java:59)
    at org.apache.shardingsphere.infra.connection.kernel.KernelProcessor.route(KernelProcessor.java:60)
    at org.apache.shardingsphere.infra.connection.kernel.KernelProcessor.generateExecutionContext(KernelProcessor.java:51)
    at org.apache.shardingsphere.driver.jdbc.core.statement.ShardingSpherePreparedStatement.createExecutionContext(ShardingSpherePreparedStatement.java:580)
    at org.apache.shardingsphere.driver.jdbc.core.statement.ShardingSpherePreparedStatement.execute(ShardingSpherePreparedStatement.java:426)
    at com.zaxxer.hikari.pool.ProxyPreparedStatement.execute(ProxyPreparedStatement.java:44)
    at com.zaxxer.hikari.pool.HikariProxyPreparedStatement.execute(HikariProxyPreparedStatement.java)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:568)
    at org.apache.ibatis.logging.jdbc.PreparedStatementLogger.invoke(PreparedStatementLogger.java:58)
    at jdk.proxy4/jdk.proxy4.$Proxy112.execute(Unknown Source)
    at org.apache.ibatis.executor.statement.PreparedStatementHandler.query(PreparedStatementHandler.java:65)
    at org.apache.ibatis.executor.statement.RoutingStatementHandler.query(RoutingStatementHandler.java:80)
    at org.apache.ibatis.executor.SimpleExecutor.doQuery(SimpleExecutor.java:65)
    at org.apache.ibatis.executor.BaseExecutor.queryFromDatabase(BaseExecutor.java:336)
    at org.apache.ibatis.executor.BaseExecutor.query(BaseExecutor.java:158)
    at com.github.pagehelper.PageInterceptor.intercept(PageInterceptor.java:169)
    at org.apache.ibatis.plugin.Plugin.invoke(Plugin.java:59)
    at jdk.proxy2/jdk.proxy2.$Proxy100.query(Unknown Source)
    at org.apache.ibatis.session.defaults.DefaultSqlSession.selectList(DefaultSqlSession.java:154)
    at org.apache.ibatis.session.defaults.DefaultSqlSession.selectList(DefaultSqlSession.java:147)
    at org.apache.ibatis.session.defaults.DefaultSqlSession.selectList(DefaultSqlSession.java:142)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:568)
    at org.mybatis.spring.SqlSessionTemplate$SqlSessionInterceptor.invoke(SqlSessionTemplate.java:425)
    at jdk.proxy2/jdk.proxy2.$Proxy84.selectList(Unknown Source)
    at org.mybatis.spring.SqlSessionTemplate.selectList(SqlSessionTemplate.java:224)
    at com.baomidou.mybatisplus.core.override.MybatisMapperMethod.executeForMany(MybatisMapperMethod.java:166)
    at com.baomidou.mybatisplus.core.override.MybatisMapperMethod.execute(MybatisMapperMethod.java:77)
    at com.baomidou.mybatisplus.core.override.MybatisMapperProxy$PlainMethodInvoker.invoke(MybatisMapperProxy.java:148)
    at com.baomidou.mybatisplus.core.override.MybatisMapperProxy.invoke(MybatisMapperProxy.java:89)
    at jdk.proxy3/jdk.proxy3.$Proxy85.selectList(Unknown Source)
    at com.baomidou.mybatisplus.extension.service.IService.list(IService.java:370)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:568)
    at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:343)
    at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:699)
    at com.micolor.saas.storage.serivce.impl.FileServiceImpl$$SpringCGLIB$$0.list(<generated>)
    at com.micolor.saas.storage.contorller.FileController.getFile(FileController.java:49)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:568)
    at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:207)
    at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:152)
    at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:117)
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:884)
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:797)
    at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87)
    at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1080)
    at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:973)
    at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1011)
    at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:914)
    at jakarta.servlet.http.HttpServlet.service(HttpServlet.java:731)
    at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:885)
    at jakarta.servlet.http.HttpServlet.service(HttpServlet.java:814)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:223)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:158)
    at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:185)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:158)
    at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:185)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:158)
    at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:185)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:158)
    at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:185)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:158)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:177)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:97)
    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:542)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:119)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:78)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:357)
    at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:400)
    at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
    at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:859)
    at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1734)
    at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:52)
    at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191)
    at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)
    at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
    at java.base/java.lang.Thread.run(Thread.java:833)
zhaojinchao95 commented 4 months ago

Please translate your issue title and body to english.

strongduanmu commented 4 months ago

For English only, other language will not accept.