search

fortify / fcli

fcli is a command-line utility for interacting with various Fortify products
https://fortify.github.io/fcli/
Other
29 stars 16 forks source link

-q "matches" operator does not handle null values #372

Closed psmf22 closed 11 months ago

psmf22 commented 12 months ago

The "matches" operator throws an IllegalStateException when being applied to a field with null value.

To reproduce execute fcli util sample-data list -q "stringValue matches 'value1|value2'

Stacktrace:

java.lang.IllegalStateException: Fcli unexpectedly terminated unsuccessfully java.lang.IllegalStateException: Error evaluating query expression: Message: EL1037E: First operand to matches operator must be a string. 'null' is not Expression: stringValue matches 'value1|value2' Record: { "id" : 15552, "stringValue" : null, "longValue" : 1000, "doubleValue" : 0.7, "booleanValue" : true, "dateValue" : "2000-01-01", "dateTimeValue" : "2000-01-01T00:00:00+00:00", "nestedObject" : { "stringValue" : "nestedObjectValue1", "booleanValue" : true }, "nestedObjectArray" : [ { "stringValue" : "nestedArrayValue1", "booleanValue" : true }, { "stringValue" : "nestedArrayValue2", "booleanValue" : false } ], "nestedStringAray" : [ "nestedArrayValue3", "nestedArrayValue4" ] } at com.fortify.cli.common.output.query.QueryExpression.matches(QueryExpression.java:32) at com.fortify.cli.common.output.writer.output.query.OutputWriterWithQuery.applyRecordOutputFilters(OutputWriterWithQuery.java:46) at com.fortify.cli.common.output.writer.output.standard.StandardOutputWriter.writeRecord(StandardOutputWriter.java:190) at com.fortify.cli.common.output.writer.output.standard.StandardOutputWriter.lambda$4(StandardOutputWriter.java:168) at java.base/java.util.ArrayList$Itr.forEachRemaining(ArrayList.java:1003) at com.fortify.cli.common.output.writer.output.standard.StandardOutputWriter.writeRecords(StandardOutputWriter.java:168) at com.fortify.cli.common.output.writer.output.standard.StandardOutputWriter.write(StandardOutputWriter.java:80) at com.fortify.cli.common.output.cli.mixin.AbstractOutputHelperMixin.write(AbstractOutputHelperMixin.java:80) at com.fortify.cli.common.output.cli.cmd.AbstractOutputCommand.run(AbstractOutputCommand.java:33) at picocli.CommandLine.executeUserObject(CommandLine.java:2103) at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2538) at picocli.CommandLine$RunLast.handle(CommandLine.java:2530) at picocli.CommandLine$RunLast.handle(CommandLine.java:1) at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2350) at picocli.CommandLine$RunLast.execute(CommandLine.java:2494) at picocli.CommandLine.execute(CommandLine.java:2247) at com.fortify.cli.app.runner.DefaultFortifyCLIRunner.run(DefaultFortifyCLIRunner.java:48) at com.fortify.cli.app.runner.DefaultFortifyCLIRunner.run(DefaultFortifyCLIRunner.java:59) at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104) at java.base/java.lang.reflect.Method.invoke(Method.java:577) at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:343) at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:328) at groovy.lang.MetaClassImpl.doInvokeMethod(MetaClassImpl.java:1332) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1087) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1006) at org.codehaus.groovy.runtime.InvokerHelper.invokePojoMethod(InvokerHelper.java:633) at org.codehaus.groovy.runtime.InvokerHelper.invokeMethod(InvokerHelper.java:624) at org.codehaus.groovy.runtime.DefaultGroovyMethods.invokeMethod(DefaultGroovyMethods.java:723) at com.fortify.cli.ftest._common.Fcli$ReflectiveRunner.run(Fcli.groovy:196) at com.fortify.cli.ftest._common.Fcli$run_closure3.doCall(Fcli.groovy:92) at com.fortify.cli.ftest._common.Fcli$run_closure3.call(Fcli.groovy) at org.codehaus.groovy.runtime.IOGroovyMethods.withCloseable(IOGroovyMethods.java:1616) at com.fortify.cli.ftest._common.Fcli._run(Fcli.groovy:91) at com.fortify.cli.ftest._common.Fcli.run(Fcli.groovy:44) at com.fortify.cli.ftest._common.Fcli.run(Fcli.groovy:67) at com.fortify.cli.ftest._common.Fcli.run(Fcli.groovy) at org.codehaus.groovy.vmplugin.v8.IndyInterface.fromCache(IndyInterface.java:321) at com.fortify.cli.ftest.core.QuerySpec.generate(QuerySpec.groovy:13) at org.codehaus.groovy.vmplugin.v8.IndyInterface.fromCache(IndyInterface.java:321) at com.fortify.cli.ftest.core.QuerySpec.$spock_feature_1_9(QuerySpec.groovy:126) at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104) at java.base/java.lang.reflect.Method.invoke(Method.java:577) at org.spockframework.util.ReflectionUtil.invokeMethod(ReflectionUtil.java:187) at org.spockframework.runtime.model.MethodInfo.lambda$new$0(MethodInfo.java:49) at org.spockframework.runtime.model.MethodInfo.invoke(MethodInfo.java:156) at org.spockframework.runtime.PlatformSpecRunner.invokeRaw(PlatformSpecRunner.java:407) at org.spockframework.runtime.PlatformSpecRunner.invoke(PlatformSpecRunner.java:390) at org.spockframework.runtime.PlatformSpecRunner.runFeatureMethod(PlatformSpecRunner.java:324) at org.spockframework.runtime.IterationNode.execute(IterationNode.java:50) at org.spockframework.runtime.SimpleFeatureNode.execute(SimpleFeatureNode.java:58) at org.spockframework.runtime.SimpleFeatureNode.execute(SimpleFeatureNode.java:15) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:151) at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141) at org.spockframework.runtime.SpockNode.sneakyInvoke(SpockNode.java:40) at org.spockframework.runtime.IterationNode.lambda$around$0(IterationNode.java:67) at org.spockframework.runtime.PlatformSpecRunner.lambda$createMethodInfoForDoRunIteration$5(PlatformSpecRunner.java:236) at org.spockframework.runtime.model.MethodInfo.invoke(MethodInfo.java:156) at org.spockframework.runtime.PlatformSpecRunner.invokeRaw(PlatformSpecRunner.java:407) at org.spockframework.runtime.PlatformSpecRunner.invoke(PlatformSpecRunner.java:390) at org.spockframework.runtime.PlatformSpecRunner.runIteration(PlatformSpecRunner.java:218) at org.spockframework.runtime.IterationNode.around(IterationNode.java:67) at org.spockframework.runtime.SimpleFeatureNode.lambda$around$0(SimpleFeatureNode.java:52) at org.spockframework.runtime.SpockNode.sneakyInvoke(SpockNode.java:40) at org.spockframework.runtime.FeatureNode.lambda$around$0(FeatureNode.java:41) at org.spockframework.runtime.PlatformSpecRunner.lambda$createMethodInfoForDoRunFeature$4(PlatformSpecRunner.java:199) at org.spockframework.runtime.model.MethodInfo.invoke(MethodInfo.java:156) at org.spockframework.runtime.PlatformSpecRunner.invokeRaw(PlatformSpecRunner.java:407) at org.spockframework.runtime.PlatformSpecRunner.invoke(PlatformSpecRunner.java:390) at org.spockframework.runtime.PlatformSpecRunner.runFeature(PlatformSpecRunner.java:192) at org.spockframework.runtime.FeatureNode.around(FeatureNode.java:41) at org.spockframework.runtime.SimpleFeatureNode.around(SimpleFeatureNode.java:52) at org.spockframework.runtime.SimpleFeatureNode.around(SimpleFeatureNode.java:15) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139) at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138) at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95) at java.base/java.util.ArrayList.forEach(ArrayList.java:1511) at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:155) at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141) at org.spockframework.runtime.SpockNode.sneakyInvoke(SpockNode.java:40) at org.spockframework.runtime.SpecNode.lambda$around$0(SpecNode.java:63) at org.spockframework.runtime.PlatformSpecRunner.lambda$createMethodInfoForDoRunSpec$0(PlatformSpecRunner.java:61) at org.spockframework.runtime.model.MethodInfo.invoke(MethodInfo.java:156) at org.spockframework.runtime.PlatformSpecRunner.invokeRaw(PlatformSpecRunner.java:407) at org.spockframework.runtime.PlatformSpecRunner.invoke(PlatformSpecRunner.java:390) at org.spockframework.runtime.PlatformSpecRunner.runSpec(PlatformSpecRunner.java:55) at org.spockframework.runtime.SpecNode.around(SpecNode.java:63) at org.spockframework.runtime.SpecNode.around(SpecNode.java:11) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139) at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138) at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95) at java.base/java.util.ArrayList.forEach(ArrayList.java:1511) at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:155) at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141) at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139) at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138) at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95) at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:35) at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57) at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:54) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:147) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:127) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:90) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:55) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:102) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:54) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:114) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:95) at org.junit.platform.launcher.core.DefaultLauncherSession$DelegatingLauncher.execute(DefaultLauncherSession.java:91) at org.junit.platform.launcher.core.SessionPerRequestLauncher.execute(SessionPerRequestLauncher.java:60) at org.eclipse.jdt.internal.junit5.runner.JUnit5TestReference.run(JUnit5TestReference.java:98) at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:40) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:529) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:756) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:452) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:210) Caused by: org.springframework.expression.spel.SpelEvaluationException: EL1037E: First operand to matches operator must be a string. 'null' is not at org.springframework.expression.spel.ast.OperatorMatches.getValueInternal(OperatorMatches.java:90) at org.springframework.expression.spel.ast.OperatorMatches.getValueInternal(OperatorMatches.java:42) at org.springframework.expression.spel.ast.SpelNodeImpl.getTypedValue(SpelNodeImpl.java:119) at org.springframework.expression.spel.standard.SpelExpression.getValue(SpelExpression.java:376) at com.fortify.cli.common.json.JsonHelper.evaluateSpelExpression(JsonHelper.java:75) at com.fortify.cli.common.output.query.QueryExpression.matches(QueryExpression.java:30) ... 123 more at com.fortify.cli.ftest._common.Fcli$FcliResult.expectSuccess(Fcli.groovy:148) at com.fortify.cli.ftest._common.Fcli.run_closure6(Fcli.groovy:65) at groovy.lang.Closure.call(Closure.java:433) at com.fortify.cli.ftest._common.Fcli.run(Fcli.groovy:45) at com.fortify.cli.ftest._common.Fcli.run(Fcli.groovy:67) at com.fortify.cli.ftest.core.QuerySpec.generate(QuerySpec.groovy:13) at com.fortify.cli.ftest.core.QuerySpec.core.query.matches(QuerySpec.groovy:126)

rsenden commented 12 months ago

Potentially we can implement and use a custom OperatorOverloader to handle this situation, overloading the matches operator to return false if any of the operands is null.

We'd need to configure this custom operator overloader on the SpEL evaluation context created in the JsonHelper::createSpelEvaluationContext method. However, we currently use a SimpleEvaluationContext, which doesn't support custom operator overloaders. So, in order to be able to use a custom overloader, we should either:

psmf22 commented 12 months ago

The OperatorOverloader does not apply to "matches" operations, there is no easy way to fix this. As a workaround guard your expressions with a null check:

If you want to include results with null values: stringValue == null || stringValue matches 'value1|value2'

To exclude results with null values: stringValue != null && stringValue matches 'value1|value2'

rsenden commented 11 months ago

As described in the previous comment, there's no easy/proper fix to handle this natively in fcli; users will need to explicitly handle null-values in their expressions if they wish to use the matches operator. Issue #382 has been opened to properly document this, and as such this issue will be closed.