Error when processing files

[geruiz]

Hi, During the scanner execution, errors of this type appear and cause the scanner to abort. The file have a valid PL/SQL procedure definition, using UTF-8 encoding without BOM.

What is the cause, and how can it be mitigated? Is it possible that it does not cause the scanner to be terminated?

I'm using sonar-plsql-cop-secondary-8.9.6.jar (I also tried with the standalone version with the same result).

11:49:16.116 ERROR: Error during SonarScanner execution java.lang.IllegalArgumentException: 0 is not a valid line for a file at org.sonar.api.utils.Preconditions.checkArgument(Preconditions.java:43) at org.sonar.api.batch.fs.internal.DefaultInputFile.checkValid(DefaultInputFile.java:352) at org.sonar.api.batch.fs.internal.DefaultInputFile.newPointer(DefaultInputFile.java:289) at org.sonar.api.batch.fs.internal.DefaultInputFile.selectLine(DefaultInputFile.java:312) at com.trivadis.sonar.plugin.AbstractPlSqlCopSensor.saveIssue(AbstractPlSqlCopSensor.java:134) at com.trivadis.sonar.plugin.AbstractPlSqlCopSensor.lambda$null$0(AbstractPlSqlCopSensor.java:106) at java.base/java.util.ArrayList.forEach(Unknown Source) at com.trivadis.sonar.plugin.AbstractPlSqlCopSensor.lambda$analyzeInputFile$1(AbstractPlSqlCopSensor.java:105) at java.base/java.util.Optional.ifPresent(Unknown Source) at com.trivadis.sonar.plugin.AbstractPlSqlCopSensor.analyzeInputFile(AbstractPlSqlCopSensor.java:102) at com.trivadis.sonar.plugin.AbstractPlSqlCopSensor.analyze(AbstractPlSqlCopSensor.java:88) at com.trivadis.sonar.plugin.AbstractPlSqlCopSensor.execute(AbstractPlSqlCopSensor.java:53) at org.sonar.scanner.sensor.AbstractSensorWrapper.analyse(AbstractSensorWrapper.java:64) at org.sonar.scanner.sensor.ModuleSensorsExecutor.execute(ModuleSensorsExecutor.java:88) at org.sonar.scanner.sensor.ModuleSensorsExecutor.lambda$execute$1(ModuleSensorsExecutor.java:61) at org.sonar.scanner.sensor.ModuleSensorsExecutor.withModuleStrategy(ModuleSensorsExecutor.java:79) at org.sonar.scanner.sensor.ModuleSensorsExecutor.execute(ModuleSensorsExecutor.java:61) at org.sonar.scanner.scan.SpringModuleScanContainer.doAfterStart(SpringModuleScanContainer.java:82) at org.sonar.core.platform.SpringComponentContainer.startComponents(SpringComponentContainer.java:188) at org.sonar.core.platform.SpringComponentContainer.execute(SpringComponentContainer.java:167) at org.sonar.scanner.scan.SpringProjectScanContainer.scan(SpringProjectScanContainer.java:403) at org.sonar.scanner.scan.SpringProjectScanContainer.scanRecursively(SpringProjectScanContainer.java:399) at org.sonar.scanner.scan.SpringProjectScanContainer.doAfterStart(SpringProjectScanContainer.java:368) at org.sonar.core.platform.SpringComponentContainer.startComponents(SpringComponentContainer.java:188) at org.sonar.core.platform.SpringComponentContainer.execute(SpringComponentContainer.java:167) at org.sonar.scanner.bootstrap.SpringGlobalContainer.doAfterStart(SpringGlobalContainer.java:137) at org.sonar.core.platform.SpringComponentContainer.startComponents(SpringComponentContainer.java:188) at org.sonar.core.platform.SpringComponentContainer.execute(SpringComponentContainer.java:167) at org.sonar.batch.bootstrapper.Batch.doExecute(Batch.java:72) at org.sonar.batch.bootstrapper.Batch.execute(Batch.java:66) at org.sonarsource.scanner.api.internal.batch.BatchIsolatedLauncher.execute(BatchIsolatedLauncher.java:46) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.sonarsource.scanner.api.internal.IsolatedLauncherProxy.invoke(IsolatedLauncherProxy.java:60) at com.sun.proxy.$Proxy0.execute(Unknown Source) at org.sonarsource.scanner.api.EmbeddedScanner.doExecute(EmbeddedScanner.java:189) at org.sonarsource.scanner.api.EmbeddedScanner.execute(EmbeddedScanner.java:138) at org.sonarsource.scanner.cli.Main.execute(Main.java:126) at org.sonarsource.scanner.cli.Main.execute(Main.java:81) at org.sonarsource.scanner.cli.Main.main(Main.java:62)

[PhilippSalvisberg]

Thank you for reporting this issue. Can you please provide the file so that we can reproduce the issue?

[geruiz]

I was looking for a file to attach and I realised that the error occurs when there are many files to process, because if I process it alone everything works fine. Is there a restriction on the number of files?

Thank you

[PhilippSalvisberg]

Have you configured your license key as shown in https://github.com/Trivadis/plsql-cop-sonar#standalone-plugin ? If not then the limitations of the trial/preview license apply.

However, you should get a proper error message when the license limit is reached.

[geruiz]

Hi, I don't have a licence key because I was evaluating this plugin for a possible purchase.

Does this error not appear when there is a valid licence key?

[PhilippSalvisberg]

The plugin should report a proper error and never crash. So it's a bug. I will look into it.

Here's an update preview/trial license you can use to register in SonarQube:

----- BEGIN LICENSE -----
maxFiles=0
maxLines=5000
maxCommands=200
maxBytes=400000
versionLt=5
validThru=2023-07-01

----- END LICENSE -----
----- BEGIN SIGNATURE -----
MCwCFGEbtLMpSRjxqK/1
+ztx4W25ljvxAhRKt5pa
Qz6eD0yqwocuejGHafLM
CQ==
----- END SIGNATURE -----

If you need a key without size limitations you can contact roland.stirnimann@accenture.com and request one.

[geruiz]

Thanks for the trial key, however, I keep getting the same error.

Also, I found this error with a file: 09:18:31.092 DEBUG: Syntax check OK! Resource: FILE.SQL 09:18:32.934 DEBUG: Parsing took: 131 ms 09:18:32.966 DEBUG: beforeModelLinked took: 0ms 09:18:32.979 DEBUG: doLinkModel took: 13ms 09:18:32.979 DEBUG: afterModelLinked took: 0ms 09:18:33.093 ERROR: Error executing EValidator org.eclipse.emf.common.util.BasicEList$BasicIndexOutOfBoundsException: index=0, size=0 at org.eclipse.emf.common.util.BasicEList.get(BasicEList.java:346) at com.trivadis.tvdcc.validators.TrivadisGuidelines3.isCursorForLoop(TrivadisGuidelines3.java:685) at com.trivadis.tvdcc.validators.TrivadisGuidelines3.checkGuideline3310(TrivadisGuidelines3.java:1062) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.eclipse.xtext.validation.AbstractDeclarativeValidator$MethodWrapper.invoke(AbstractDeclarativeValidator.java:129) at org.eclipse.xtext.validation.AbstractDeclarativeValidator.internalValidate(AbstractDeclarativeValidator.java:337) at org.eclipse.xtext.validation.AbstractInjectableValidator.validate(AbstractInjectableValidator.java:72) at org.eclipse.xtext.validation.CompositeEValidator.validate(CompositeEValidator.java:151) at org.eclipse.emf.ecore.util.Diagnostician.doValidate(Diagnostician.java:299) at org.eclipse.emf.ecore.util.Diagnostician.validate(Diagnostician.java:245) at org.eclipse.emf.ecore.util.Diagnostician.validate(Diagnostician.java:200) at org.eclipse.emf.ecore.util.Diagnostician.validate(Diagnostician.java:130) at com.trivadis.oracle.sqlplus.validation.SQLPLUSValidator.checkSqlPlusFile(SQLPLUSValidator.java:57) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.eclipse.xtext.validation.AbstractDeclarativeValidator$MethodWrapper.invoke(AbstractDeclarativeValidator.java:129) at org.eclipse.xtext.validation.AbstractDeclarativeValidator.internalValidate(AbstractDeclarativeValidator.java:337) at org.eclipse.xtext.validation.AbstractInjectableValidator.validate(AbstractInjectableValidator.java:72) at org.eclipse.xtext.validation.CompositeEValidator.validate(CompositeEValidator.java:151) at org.eclipse.emf.ecore.util.Diagnostician.doValidate(Diagnostician.java:299) at org.eclipse.emf.ecore.util.Diagnostician.validate(Diagnostician.java:245) at org.eclipse.xtext.validation.CancelableDiagnostician.validate(CancelableDiagnostician.java:41) at org.eclipse.emf.ecore.util.Diagnostician.validate(Diagnostician.java:200) at org.eclipse.emf.ecore.util.Diagnostician.validate(Diagnostician.java:142) at org.eclipse.xtext.validation.ResourceValidatorImpl.validate(ResourceValidatorImpl.java:147) at org.eclipse.xtext.validation.ResourceValidatorImpl.validate(ResourceValidatorImpl.java:125) at org.eclipse.xtext.validation.ResourceValidatorImpl.validate(ResourceValidatorImpl.java:91) at com.trivadis.tvdcc.process.ResourceValidatorImpl.performValidation(ResourceValidatorImpl.java:52) at com.trivadis.tvdcc.process.ResourceValidatorImpl.validate(ResourceValidatorImpl.java:40) at com.trivadis.tvdcc.process.ValidationTask.analyze(ValidationTask.java:146) at com.trivadis.tvdcc.process.ValidationTask.process(ValidationTask.java:108) at com.trivadis.sonar.plugin.AbstractPlSqlCopSensor.analyzeInputFile(AbstractPlSqlCopSensor.java:101) at com.trivadis.sonar.plugin.AbstractPlSqlCopSensor.analyze(AbstractPlSqlCopSensor.java:88) at com.trivadis.sonar.plugin.AbstractPlSqlCopSensor.execute(AbstractPlSqlCopSensor.java:53) at org.sonar.scanner.sensor.AbstractSensorWrapper.analyse(AbstractSensorWrapper.java:64) at org.sonar.scanner.sensor.ModuleSensorsExecutor.execute(ModuleSensorsExecutor.java:88) at org.sonar.scanner.sensor.ModuleSensorsExecutor.lambda$execute$1(ModuleSensorsExecutor.java:61) at org.sonar.scanner.sensor.ModuleSensorsExecutor.withModuleStrategy(ModuleSensorsExecutor.java:79) at org.sonar.scanner.sensor.ModuleSensorsExecutor.execute(ModuleSensorsExecutor.java:61) at org.sonar.scanner.scan.SpringModuleScanContainer.doAfterStart(SpringModuleScanContainer.java:82) at org.sonar.core.platform.SpringComponentContainer.startComponents(SpringComponentContainer.java:188) at org.sonar.core.platform.SpringComponentContainer.execute(SpringComponentContainer.java:167) at org.sonar.scanner.scan.SpringProjectScanContainer.scan(SpringProjectScanContainer.java:403) at org.sonar.scanner.scan.SpringProjectScanContainer.scanRecursively(SpringProjectScanContainer.java:399) at org.sonar.scanner.scan.SpringProjectScanContainer.doAfterStart(SpringProjectScanContainer.java:368) at org.sonar.core.platform.SpringComponentContainer.startComponents(SpringComponentContainer.java:188) at org.sonar.core.platform.SpringComponentContainer.execute(SpringComponentContainer.java:167) at org.sonar.scanner.bootstrap.SpringGlobalContainer.doAfterStart(SpringGlobalContainer.java:137) at org.sonar.core.platform.SpringComponentContainer.startComponents(SpringComponentContainer.java:188) at org.sonar.core.platform.SpringComponentContainer.execute(SpringComponentContainer.java:167) at org.sonar.batch.bootstrapper.Batch.doExecute(Batch.java:72) at org.sonar.batch.bootstrapper.Batch.execute(Batch.java:66) at org.sonarsource.scanner.api.internal.batch.BatchIsolatedLauncher.execute(BatchIsolatedLauncher.java:46) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.sonarsource.scanner.api.internal.IsolatedLauncherProxy.invoke(IsolatedLauncherProxy.java:60) at com.sun.proxy.$Proxy0.execute(Unknown Source) at org.sonarsource.scanner.api.EmbeddedScanner.doExecute(EmbeddedScanner.java:189) at org.sonarsource.scanner.api.EmbeddedScanner.execute(EmbeddedScanner.java:138) at org.sonarsource.scanner.cli.Main.execute(Main.java:126) at org.sonarsource.scanner.cli.Main.execute(Main.java:81)

although in this case the result was saved without error.

[PhilippSalvisberg]

Thanks for the trial key, however, I keep getting the same error.

Yes, I could reproduce it. There are cases where the sonar-scanner crashes when a license limit is reached instead of reporting the limit as an error and stopping analyzing subsequent files. What's expected is something like this:

...
INFO: Process resource test_suite_builder.pks
ERROR: license limit reached while analyzing test_suite_builder.pks, skipping other files
INFO: Sensor PlSQL COP Sensor [plsqlcop] (done) | time=4313ms
...

The crash is clearly a bug. We have to fix it.

Until it is fixed you have basically two options:

a) test with e reduced set of files that does not reach the limit b) request a trial license key that has no limitations (by e-mail, see comment above)

Also, I found this error with a file:

...
org.eclipse.emf.common.util.BasicEList$BasicIndexOutOfBoundsException: index=0, size=0
...

This is another bug in a validator check. Please provide us the file that causes this issue.

Thank you.

[geruiz]

I am providing you the file that causes the error, as well as the output of its execution.

Thank you.

execution_bug.zip

[PhilippSalvisberg]

@geruiz Thank you for the file. I can reproduce the behaviour. The root cause of the error is a syntax error in your file. There is a missing right parenthesis on line 110:

when you change

                AND OT.OID = NVL(v_OPERATION_TYPE,OT.OID);

to

                AND OT.OID = NVL(v_OPERATION_TYPE,OT.OID));

sonar-scanner completes without error.

Nevertheless, the validator should be able do deal with syntax errors.

[geruiz]

Thanks for your reply. I didn't look for syntax errors. :-(

[PhilippSalvisberg]

The syntax errors are reported. The errors thrown in the validator do not prohibit the processing of the file. You should get a similar result as shown the following screenshots. The error messages are not really good, however, they clearly tell you that db* CODECOP was not able to parse the file successfully.

[PhilippSalvisberg]

fixed with db* CODECOP for SonarQube v8.9.7