apache / shardingsphere

Distributed SQL transaction & query engine for data sharding, scaling, encryption, and more - on any database.
Apache License 2.0
19.83k stars 6.71k forks source link

InlineExpressionParser cause metaspace out of memory error #22196

Open fbin87 opened 1 year ago

fbin87 commented 1 year ago

version:5.1.2

InlineExpressionParser.splitAndEvaluate method parse every input string, cause metaspace out of memory error

you can do like this, avoid compiling all input strings

`
public List splitAndEvaluate() { if(Strings.isNullOrEmpty(inlineExpression) ){ return Collections.emptyList(); } if (!isInlineExpression(inlineExpression)) { List res = new ArrayList<>(); res.add(inlineExpression); return res; } return flatten(evaluate(split())); }

`

Caused by: java.lang.OutOfMemoryError: Metaspace at java.lang.ClassLoader.defineClass1(Native Method) ~[?:1.8.0_342] at java.lang.ClassLoader.defineClass(ClassLoader.java:756) ~[?:1.8.0_342] at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) ~[?:1.8.0_342] at groovy.lang.GroovyClassLoader.access$400(GroovyClassLoader.java:89) ~[groovy-4.0.3.jar:4.0.3] at groovy.lang.GroovyClassLoader$ClassCollector.createClass(GroovyClassLoader.java:690) ~[groovy-4.0.3.jar:4.0.3] at groovy.lang.GroovyClassLoader$ClassCollector.onClassNode(GroovyClassLoader.java:707) ~[groovy-4.0.3.jar:4.0.3] at groovy.lang.GroovyClassLoader$ClassCollector.call(GroovyClassLoader.java:712) ~[groovy-4.0.3.jar:4.0.3] at org.codehaus.groovy.control.CompilationUnit$3.call(CompilationUnit.java:806) ~[groovy-4.0.3.jar:4.0.3] at org.codehaus.groovy.control.CompilationUnit$IPrimaryClassNodeOperation.doPhaseOperation(CompilationUnit.java:937) ~[groovy-4.0.3.jar:4.0.3] at org.codehaus.groovy.control.CompilationUnit.processPhaseOperations(CompilationUnit.java:692) ~[groovy-4.0.3.jar:4.0.3] at org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:666) ~[groovy-4.0.3.jar:4.0.3] at groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:373) ~[groovy-4.0.3.jar:4.0.3] at groovy.lang.GroovyClassLoader.lambda$parseClass$2(GroovyClassLoader.java:316) ~[groovy-4.0.3.jar:4.0.3] at groovy.lang.GroovyClassLoader$$Lambda$3030/978874569.provide(Unknown Source) ~[?:?] at org.codehaus.groovy.runtime.memoize.StampedCommonCache.compute(StampedCommonCache.java:163) ~[groovy-4.0.3.jar:4.0.3] at org.codehaus.groovy.runtime.memoize.StampedCommonCache.getAndPut(StampedCommonCache.java:154) ~[groovy-4.0.3.jar:4.0.3] at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:314) ~[groovy-4.0.3.jar:4.0.3] at groovy.lang.GroovyShell.parseClass(GroovyShell.java:572) ~[groovy-4.0.3.jar:4.0.3] at groovy.lang.GroovyShell.parse(GroovyShell.java:585) ~[groovy-4.0.3.jar:4.0.3] at groovy.lang.GroovyShell.parse(GroovyShell.java:639) ~[groovy-4.0.3.jar:4.0.3] at groovy.lang.GroovyShell.parse(GroovyShell.java:625) ~[groovy-4.0.3.jar:4.0.3] at groovy.lang.GroovyShell.parse(GroovyShell.java:634) ~[groovy-4.0.3.jar:4.0.3] at org.apache.shardingsphere.infra.expr.InlineExpressionParser.evaluate(InlineExpressionParser.java:110) ~[] at org.apache.shardingsphere.infra.expr.InlineExpressionParser.evaluate(InlineExpressionParser.java:100) ~[] at org.apache.shardingsphere.infra.expr.InlineExpressionParser.splitAndEvaluate(InlineExpressionParser.java:78) ~[] at org.apache.shardingsphere.sharding.rule.ShardingRule.getDataSourceNames(ShardingRule.java:209) ~[] at org.apache.shardingsphere.sharding.rule.ShardingRule.lambda$getDataSourceNames$3(ShardingRule.java:198) ~[] at org.apache.shardingsphere.sharding.rule.ShardingRule$$Lambda$3023/1285344774.accept(Unknown Source) ~[?:?] at java.util.ArrayList.forEach(ArrayList.java:1259) ~[?:1.8.0_342] at org.apache.shardingsphere.sharding.rule.ShardingRule.getDataSourceNames(ShardingRule.java:198) ~[] at org.apache.shardingsphere.sharding.rule.ShardingRule.(ShardingRule.java:118) ~[] at org.apache.shardingsphere.sharding.rule.builder.ShardingRuleBuilder.build(ShardingRuleBuilder.java:41) ~[]

tuichenchuxin commented 1 year ago

Hi, I found you had fixed this issue, could you offer a pull request to help others?

fbin87 commented 1 year ago

Hi, I found you had fixed this issue, could you offer a pull request to help others?

ok,I'm going to submit the code