”InlineShardingAlgorithm“ It's not thread-safe， Occasionally the following errors occur: Cause: java.sql.SQLException: Unknown exception: Cannot invoke method hashCode() on null object

[DengQin]

Question：

Most of the time it works, but occasionally errors, parameters and SQL are correct

1、My projects occasionally have the following errors ：

.278277293+08:00

Cause: java.sql.SQLException: Unknown exception: Cannot invoke method hashCode() on null object

2024-10-21T18:52:20.278289903+08:00 ; uncategorized SQLException; SQL state [HY000]; error code [30000]; Unknown exception: Cannot invoke method hashCode() on null object; nested exception is java.sql.SQLException: Unknown exception: Cannot invoke method hashCode() on null object 2024-10-21T18:52:20.278305818+08:00 at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:89) ~[spring-jdbc-5.2.6.RELEASE.jar:5.2.6.RELEASE] 2024-10-21T18:52:20.278318225+08:00 at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:81) ~[spring-jdbc-5.2.6.RELEASE.jar:5.2.6.RELEASE] 2024-10-21T18:52:20.278330711+08:00 at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:81) ~[spring-jdbc-5.2.6.RELEASE.jar:5.2.6.RELEASE] 2024-10-21T18:52:20.278339694+08:00 at org.mybatis.spring.MyBatisExceptionTranslator.translateExceptionIfPossible(MyBatisExceptionTranslator.java:88) ~[mybatis-spring-2.0.5.jar:2.0.5] 2024-10-21T18:52:20.278352522+08:00 at org.mybatis.spring.SqlSessionTemplate$SqlSessionInterceptor.invoke(SqlSessionTemplate.java:440) ~[mybatis-spring-2.0.5.jar:2.0.5] 2024-10-21T18:52:20.278360916+08:00 at com.sun.proxy.$Proxy141.selectList(Unknown Source) ~[?:?] 2024-10-21T18:52:20.278372966+08:00 at org.mybatis.spring.SqlSessionTemplate.selectList(SqlSessionTemplate.java:223) ~[mybatis-spring-2.0.5.jar:2.0.5]

2、sharding-jdbc version：5.4.1

3、sharding-jdbc configuration for my project：

4、SQL and param：

SQL： select id, scene, business_id, user_id, distribute_type, distribute_value, distribute_status, updated_at, distribute_time from t_benefit_distribute_record where user_id = ? and scene = ?

param： 466721317,"marketing_popup"

[DengQin]

It seems that there is a conflict with other sharding algorithms in the concurrency situation

[DengQin]

Concurrent modification problems with simultaneous access by multiple threads may occur

[DengQin]

This is my temporary solution, rewriting the InlineShardingAlgorithm via SPI

[linghengqian]

• This change was originally introduced by the SPI I designed, but it did not break existing unit tests. Can you open a PR with the new unit tests? Or provide a git repository with the unit tests? There has been a lack of attention paid to the handling of groovyshell class instances.

• By the way, the reason I introduced changes in this algorithm class is to get rid of groovy in the master branch of shardingsphere. It is not reasonable to use a specific SPI implementation class in #init(). Potential implementations of Python, JavaScript, Ruby, and WASM Row Value Expressions may still be put into this Java class. However, the relevant proposal was initially blocked by an issue on the GraalVM CE side. With the decoupling of GraalVM Truffle from the GraalVM SDK, I can re-examine the design of the relevant SPI.

[DengQin]

• This change was originally introduced by the SPI I designed, but it did not break existing unit tests. Can you open a PR with the new unit tests? Or provide a git repository with the unit tests? There has been a lack of attention paid to the handling of groovyshell class instances.
• By the way, the reason I introduced changes in this algorithm class is to get rid of groovy in the master branch of shardingsphere. It is not reasonable to use a specific SPI implementation class in #init(). Potential implementations of Python, JavaScript, Ruby, and WASM Row Value Expressions may still be put into this Java class. However, the relevant proposal was initially blocked by an issue on the GraalVM CE side. With the decoupling of GraalVM Truffle from the GraalVM SDK, I can re-examine the design of the relevant SPI.

InlineExpressionParser obtains the singleton object through SPI, but the inlineExpression field property is shared and is not thread-safe. For example: Step 1: A normal query, InlineExpressionParserFactory. NewInstance (algorithmExpression) performed, .evaluateClosure().rehydrate(new Expando(), null, null); Not yet executed.
Step 2: another request perform a InlineExpressionParserFactory. NewInstance (algorithmExpression), the first step will lead to the request in the value of the change. Step 3: An exception caused by multi-threaded modifications will appear

[DengQin]

Loading InlineExpressionParser through SPI is reasonable, but the properties in the parser must be thread-safe

[DengQin]

It is easy to occur when the number of requests is large and the number of sharding rules is large. If there is only one sharding strategy, it will not occur

[linghengqian]

• It sounds like it's really just a matter of using the volatile keyword for specific variables? The associated variable should not be read very often?

• I still haven't had time to verify it. Are you going to submit a PR with unit tests?