apache / shardingsphere

Empowering Data Intelligence with Distributed SQL for Sharding, Scalability, and Security Across All Databases.
Apache License 2.0
19.97k stars 6.75k forks source link

The project is packaged into a jar and fails to run #24891

Closed Rain-ziyu closed 4 months ago

Rain-ziyu commented 1 year ago

Question

Run the project directly using the IDEA editor, and everything works fine. However, when running the jar file after using maven install, the spi implementation related to mysql parser cannot be found. Use shardingsphere-jdbc-core-spring-boot-starter 5.2.1 The error log is as follows Caused by: org.apache.ibatis.executor.ExecutorException: Error preparing statement. Cause: org.apache.shardingsphere.infra.util.spi.exception.ServiceProviderNotFoundServerException: SPI-00001: No implementation class load from SPIorg.apache.shardingsphere.sql.parser.spi.SQLVisitorFacadewith typeMySQL.STATEMENT. at org.apache.ibatis.executor.statement.BaseStatementHandler.prepare(BaseStatementHandler.java:97) at org.apache.ibatis.executor.statement.RoutingStatementHandler.prepare(RoutingStatementHandler.java:59) at jdk.internal.reflect.GeneratedMethodAccessor64.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.apache.ibatis.plugin.Invocation.proceed(Invocation.java:49) at com.baomidou.mybatisplus.extension.plugins.MybatisPlusInterceptor.intercept(MybatisPlusInterceptor.java:106) at org.apache.ibatis.plugin.Plugin.invoke(Plugin.java:62) at jdk.proxy2/jdk.proxy2.$Proxy301.prepare(Unknown Source) at org.apache.ibatis.executor.SimpleExecutor.prepareStatement(SimpleExecutor.java:87) at org.apache.ibatis.executor.SimpleExecutor.doQuery(SimpleExecutor.java:62) at org.apache.ibatis.executor.BaseExecutor.queryFromDatabase(BaseExecutor.java:325) at org.apache.ibatis.executor.BaseExecutor.query(BaseExecutor.java:156) at org.apache.ibatis.executor.CachingExecutor.query(CachingExecutor.java:109) at com.baomidou.mybatisplus.extension.plugins.MybatisPlusInterceptor.intercept(MybatisPlusInterceptor.java:81) at org.apache.ibatis.plugin.Plugin.invoke(Plugin.java:62) at jdk.proxy2/jdk.proxy2.$Proxy300.query(Unknown Source) at org.apache.ibatis.session.defaults.DefaultSqlSession.selectList(DefaultSqlSession.java:151) at org.apache.ibatis.session.defaults.DefaultSqlSession.selectList(DefaultSqlSession.java:145) at org.apache.ibatis.session.defaults.DefaultSqlSession.selectList(DefaultSqlSession.java:140) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.mybatis.spring.SqlSessionTemplate$SqlSessionInterceptor.invoke(SqlSessionTemplate.java:427) ... 143 more For English only, other languages will not accept.

Before asking a question, make sure you have:

Please pay attention on issues you submitted, because we maybe need more details. If no response anymore and we cannot reproduce it on current information, we will close it.

RaigorJiang commented 1 year ago

Hi @Rain-ziyu I want to investigate this issue, can you provide a reproducible demo?

Rain-ziyu commented 1 year ago

Hi @Rain-ziyu I want to investigate this issue, can you provide a reproducible demo?

The problem is that in this project of mine 项目地址 , everything is normal when I start server_blog locally, but when I compile it into a jar and run it, the above problem occurs. I also found that when I use mapper in a PostConstruct method then the above problem disappears. image I tried to test mybatisplus and shardingsphere-jdbc-core-spring-boot-starter separately, everything works fine, so I can't provide a simple reproduction, I will try my best

Rain-ziyu commented 1 year ago

Hi @Rain-ziyu I want to investigate this issue, can you provide a reproducible demo?

After my test, when calling the following form to query, it will cause No implementation class load from SPI org.apache.shardingsphere.driver.state.DriverState with type OK. image The same only appears in the jar running situation,IDEA runs without any problems

Rain-ziyu commented 1 year ago

I made a simple reproduction code, I hope it can help you troubleshoot the problem 代码地址

Rain-ziyu commented 1 year ago

idea run image jar run image

RaigorJiang commented 1 year ago

I made a simple reproduction code, I hope it can help you troubleshoot the problem 代码地址

@Rain-ziyu Thank you, I will investigate it later.

Rain-ziyu commented 1 year ago

I made a simple reproduction code, I hope it can help you troubleshoot the problem 代码地址

@Rain-ziyu Thank you, I will investigate it later.

Hello, do you have any idea about this problem?

RaigorJiang commented 1 year ago

Hi @Rain-ziyu Sorry, I checked the source code of version 5.2.1 and found no logic that could cause these problems.

In addition, the simple reproduction code you provided is no longer accessible. Has the problem been solved?

Rain-ziyu commented 1 year ago

你好@Rain-ziyu 抱歉,我查看了5.2.1版本的源代码,没有发现可能导致这些问题的逻辑。

此外,您提供的简单复制代码已无法访问。问题解决了吗?

I will modify it to public. The problem has been solved. Follow the guidance of GPT, I modified the source code to the custom thread pool mode. There is no problem.

like this:

CompletableFuture.supplyAsync(()
                -> roleMapper.listResourceRoles());

Modify to

ExecutorService executor = Executors.newFixedThreadPool(10); 

CompletableFuture<List<ResourceRole>> asyncArticleCount = CompletableFuture.supplyAsync(() -> roleMapper.listResourceRoles(), executor);
Rain-ziyu commented 1 year ago

The explanation given to me by GPT is that in your code, you use the CompletableFuture.supplyAsync() method to create an asynchronous task and submit it to the default ForkJoinPool.commonPool() thread pool for execution. However, ShardingSphere may use some custom thread pools or other configurations, which may be incompatible with the ForkJoinPool.commonPool() thread pool, resulting in some exceptions or errors.

Using ThreadPoolExecutor can avoid this problem, because it allows you to customize a new thread pool, and you can configure it according to your needs, so as to avoid the problem of incompatibility with ShardingSphere.

RaigorJiang commented 1 year ago

@Rain-ziyu Thank you, I'm curious how the thread pool here affects the SPI loading, it's really interesting, we're going to dig into that.

By the way, GPT is too powerful. 🤣

Rain-ziyu commented 1 year ago

Wish you an early resolution, this way of writing is a bit bloated for me

Ecruos-Hua commented 1 year ago

There is a same probleam in my project Snipaste_2023-11-09_15-15-12 And I find it could by solved when I do a normal sharding-query first, before using the CompletableFuture + sharding-query

terrymanu commented 4 months ago

shardingsphere-jdbc-core-spring-boot-starter is deprecated, please try new version and use ShardingSphere Driver.