search

apache / shardingsphere

Empowering Data Intelligence with Distributed SQL for Sharding, Scalability, and Security Across All Databases.
Apache License 2.0
19.97k stars 6.75k forks source link

Is the config of Data Sharding + Read-Write Split correct ？ [ shardingproxy 5.0.0] #8314

Closed duzhanfei closed 3 years ago

duzhanfei commented 3 years ago

config in config-replica-query.yaml

schemaName: replica_query_db

dataSourceCommon: connectionTimeoutMilliseconds: 30000 idleTimeoutMilliseconds: 60000 maxLifetimeMilliseconds: 1800000 maxPoolSize: 50 minPoolSize: 1 maintenanceIntervalMilliseconds: 30000

dataSources: primary_ds: url: jdbc:mysql://127.5.2.31:3306/crm_customer_db_master0?serverTimezone=UTC&useSSL=false username: root password: root replica_ds_0: url: jdbc:mysql://127.5.2.32:3306/crm_customer_db_master0_slave0?serverTimezone=UTC&useSSL=false username: root password: root replica_ds_1: url: jdbc:mysql://127.5.2.33:3306/crm_customer_db_master0_slave1?serverTimezone=UTC&useSSL=false username: root password: root ds_default: url: jdbc:mysql://127.5.2.34:3306/cdsp_crm_db?serverTimezone=UTC&useSSL=false username: root password: root

rules:

terrymanu commented 3 years ago

What is your problem, did you meet exception?

duzhanfei commented 3 years ago

Yes, the probles is show in issues 8303. Thank you for your time.

https://github.com/apache/shardingsphere/issues/8303

At 2020-11-24 19:19:05, "Liang Zhang" notifications@github.com wrote:

What is your problem, did you meet exception?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.

duzhanfei commented 3 years ago

I wonder whether I made a mistake in configuration, so I mentioned this issue

duzhanfei commented 3 years ago

I only test the data sharding ,the result is the same

the config of config-sharding.yaml is show below:

schemaName: sharding_db

dataSourceCommon: connectionTimeoutMilliseconds: 30000 idleTimeoutMilliseconds: 60000 maxLifetimeMilliseconds: 1800000 maxPoolSize: 50 minPoolSize: 1 maintenanceIntervalMilliseconds: 30000

dataSources: ms_ds: url: jdbc:mysql://19.11.17.33:3306/crm_customer_db_master0?serverTimezone=UTC&useSSL=false username: root password: root

rules:

request infomation:

00000000

log:

[INFO ] 20:48:16.658 [ShardingSphere-Command-3] ShardingSphere-SQL - Logic SQL: SELECT @@session.transaction_read_only [INFO ] 20:48:16.658 [ShardingSphere-Command-3] ShardingSphere-SQL - SQLStatement: MySQLSelectStatement(limit=Optional.empty, lock=Optional.empty) [INFO ] 20:48:16.661 [ShardingSphere-Command-3] ShardingSphere-SQL - Actual SQL: ms_ds ::: SELECT @@session.transaction_read_only [INFO ] 20:48:16.681 [ShardingSphere-Command-4] ShardingSphere-SQL - Logic SQL: SELECT @@session.transaction_isolation [INFO ] 20:48:16.685 [ShardingSphere-Command-4] ShardingSphere-SQL - SQLStatement: MySQLSelectStatement(limit=Optional.empty, lock=Optional.empty) [INFO ] 20:48:16.685 [ShardingSphere-Command-4] ShardingSphere-SQL - Actual SQL: ms_ds ::: SELECT @@session.transaction_isolation [ERROR] 20:48:17.175 [ShardingSphere-Command-5] o.a.s.p.f.c.CommandExecutorTask - Exception occur: java.lang.NullPointerException: Cannot invoke method hashCode() on null object at org.codehaus.groovy.runtime.NullObject.hashCode(NullObject.java:174) at org.codehaus.groovy.runtime.NullObject$hashCode.call(Unknown Source) at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:47) at org.codehaus.groovy.runtime.callsite.NullCallSite.call(NullCallSite.java:34) at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:47) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:116) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:120) at Script24$_run_closure1.doCall(Script24.groovy:1) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:98) at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325) at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:264) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1034) at groovy.lang.Closure.call(Closure.java:420) at groovy.lang.Closure.call(Closure.java:414) at org.apache.shardingsphere.sharding.algorithm.sharding.inline.InlineShardingAlgorithm.doSharding(InlineShardingAlgorithm.java:69) at org.apache.shardingsphere.sharding.route.strategy.type.standard.StandardShardingStrategy.doSharding(StandardShardingStrategy.java:67) at org.apache.shardingsphere.sharding.route.strategy.type.standard.StandardShardingStrategy.doSharding(StandardShardingStrategy.java:56) at org.apache.shardingsphere.sharding.route.engine.type.standard.ShardingStandardRoutingEngine.routeTables(ShardingStandardRoutingEngine.java:214) at org.apache.shardingsphere.sharding.route.engine.type.standard.ShardingStandardRoutingEngine.route0(ShardingStandardRoutingEngine.java:194) at org.apache.shardingsphere.sharding.route.engine.type.standard.ShardingStandardRoutingEngine.routeByShardingConditionsWithCondition(ShardingStandardRoutingEngine.java:114) at org.apache.shardingsphere.sharding.route.engine.type.standard.ShardingStandardRoutingEngine.routeByShardingConditions(ShardingStandardRoutingEngine.java:107) at org.apache.shardingsphere.sharding.route.engine.type.standard.ShardingStandardRoutingEngine.getDataNodes(ShardingStandardRoutingEngine.java:84) at org.apache.shardingsphere.sharding.route.engine.type.standard.ShardingStandardRoutingEngine.route(ShardingStandardRoutingEngine.java:69) at org.apache.shardingsphere.sharding.route.engine.ShardingSQLRouter.createRouteContext(ShardingSQLRouter.java:70) at org.apache.shardingsphere.sharding.route.engine.ShardingSQLRouter.createRouteContext(ShardingSQLRouter.java:55) at org.apache.shardingsphere.infra.route.engine.impl.PartialSQLRouteExecutor.route(PartialSQLRouteExecutor.java:59) at org.apache.shardingsphere.infra.route.engine.SQLRouteEngine.route(SQLRouteEngine.java:57) at org.apache.shardingsphere.infra.context.kernel.KernelProcessor.generateExecutionContext(KernelProcessor.java:52) at org.apache.shardingsphere.proxy.backend.communication.jdbc.JDBCDatabaseCommunicationEngine.execute(JDBCDatabaseCommunicationEngine.java:77) at org.apache.shardingsphere.proxy.backend.text.query.QueryBackendHandler.execute(QueryBackendHandler.java:61) at org.apache.shardingsphere.proxy.frontend.mysql.command.query.text.query.MySQLComQueryPacketExecutor.execute(MySQLComQueryPacketExecutor.java:62) at org.apache.shardingsphere.proxy.frontend.command.CommandExecutorTask.executeCommand(CommandExecutorTask.java:100) at org.apache.shardingsphere.proxy.frontend.command.CommandExecutorTask.run(CommandExecutorTask.java:76) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source)

duzhanfei commented 3 years ago

I suspect that My configuration is wrong, because this is the basic function, but I do not know what is wrong.

terrymanu commented 3 years ago

The exception is show you clearly:

Exception occur:
java.lang.NullPointerException: Cannot invoke method hashCode() on null object
at org.codehaus.groovy.runtime.NullObject.hashCode(NullObject.java:174)

It says you are calling hashCode in groovy (inline expression) and throw NullPointException. Even you said the value is not null, but the exception show the value is null.

duzhanfei commented 3 years ago

I can insert data into the sub-table, this error occued when query by the sharding key has exist

terrymanu commented 3 years ago

what is your insert SQL?

duzhanfei commented 3 years ago

insert sql :

INSERT INTO pscst_prdt_cvr BIZ_DT, ECIF_CUST_NO, DEP_FLG, DBTCD_FLG, PLM_PY_FLG, PY_GTWY_FLG, SFSRV_EQP_SGNG_FLG, ATMBL_MRTG_FLG, HS_MRTG_FLG, HLPFM_ETMN_FLG, PSNL_MANG_LN_FLG, PSNL_CNSM_LN_FLG, ETC_FLG, CSCRD_FLG, MBLBNK_FLG, EBP_SGNG_FLG, params: 111 proxy log: 11
terrymanu commented 3 years ago

Ok, it looks like metadata not load. I just fix an issue about this, could you use the newest dev version and try it again?

terrymanu commented 3 years ago

No response anymore

ttttiny commented 3 years ago

insert sql :

INSERT INTO pscst_prdt_cvr BIZ_DT, ECIF_CUST_NO, DEP_FLG, DBTCD_FLG, PLM_PY_FLG, PY_GTWY_FLG, SFSRV_EQP_SGNG_FLG, ATMBL_MRTG_FLG, HS_MRTG_FLG, HLPFM_ETMN_FLG, PSNL_MANG_LN_FLG, PSNL_CNSM_LN_FLG, ETC_FLG, CSCRD_FLG, MBLBNK_FLG, EBP_SGNG_FLG, params:

111

proxy log:

11

这个问题咋样了0.0