Add GitHub action to run SonarQube for METdataio pull requests and feature branches

JohnHalleyGotway commented 5 months ago

Describe the New Feature

This issue is to add a new SonarQube workflow to GitHub actions to automate the static code analysis for all pull requests. In addition, add a manual trigger workflow dispatch option where the reference branch can be manually specified.

Recommend adding this workflow to both the develop branch and the current main_v*so that the workflow dispatch option can be made available.

Recommend pushing results to a new SonarQube project named METdataio GHA at needham.rap.ucar.edu.

Recommend having the workflow report bad status if the number of SonarQube findings are increased relative to the SonarQube reference.

See issue dtcenter/MET#2379 and its two linked PR's as an example, but the implementation for a python-only repo should be more straight-forward.

Acceptance Testing

List input data types and sources. Describe tests required for new functionality.

Time Estimate

Estimate the amount of work required here. Issues should represent approximately 1 to 3 days of work.

Sub-Issues

Consider breaking the new feature down into sub-issues.

[ ] Add a checkbox for each sub-issue here.

Relevant Deadlines

List relevant project deadlines here or state NONE.

Funding Source

Needed for the Air Force - 2771024

Define the Metadata

Assignee

[ ] Select engineer(s) or no engineer required
[ ] Select scientist(s) or no scientist required

Labels

[x] Select component(s)
[x] Select priority
[x] Select requestor(s)

Projects and Milestone

[x] Select Repository and/or Organization level Project(s) or add alert: NEED CYCLE ASSIGNMENT label
[x] Select Milestone as the next official version or Future Versions

Define Related Issue(s)

Consider the impact to the other METplus components.

[x] METplus, MET, METdataio, METviewer, METexpress, METcalcpy, METplotpy

The following SonarQube issues are closely related:

[x] dtcenter/METplus#2537
[x] dtcenter/MET#2379
[x] dtcenter/METviewer#521
[x] dtcenter/METplotpy#429
[x] dtcenter/METcalcpy#366
[x] dtcenter/METdataio#289

New Feature Checklist

See the METplus Workflow for details.

[ ] Complete the issue definition above, including the Time Estimate and Funding source.
[ ] Fork this repository or create a branch of develop. Branch name: feature_<Issue Number>_<Description>
[ ] Complete the development and test your changes.
[ ] Add/update log messages for easier debugging.
[ ] Add/update unit tests.
[ ] Add/update documentation.
[ ] Add any new Python packages to the METplus Components Python Requirements table.
[ ] Push local changes to GitHub.
[ ] Submit a pull request to merge into develop. Pull request: feature <Issue Number> <Description>
[ ] Define the pull request metadata, as permissions allow. Select: Reviewer(s) and Development issues Select: Repository level development cycle Project for the next official release Select: Milestone as the next official version
[ ] Iterate until the reviewer(s) accept and merge your changes.
[ ] Delete your fork or branch.
[ ] Close this issue.

JohnHalleyGotway commented 5 months ago

Followed these instructions to better integrate SonarQube with GitHub: https://docs.sonarsource.com/sonarqube/10.2/devops-platform-integration/github-integration/

To create a new GitHub app for the DTCenter organization (https://github.com/apps/dtcenter-sonarqube-integration) to support SonarQube integration.

Installed that app in the DTCenter org settings (https://github.com/organizations/dtcenter/settings/apps) for 6 repositories: METplus, MET, METviewer, METplotpy, METcalcpy, METdataio

Followed these instructions to setup GitHub user authentication for our SonarQube server: https://docs.sonarsource.com/sonarqube/10.2/instance-administration/authentication/github/

To create a new GitHub app for the DTCenter organization (https://github.com/apps/dtcenter-sonarqube-user-auth) to support GitHub user authentication at our SonarQube server.

Members of the DTCenter GitHub organization are now able to authenticate at needham.rap.ucar.edu using their GitHub credentials.

I was able to re-use two existing GitHub actions provided by SonarSource:

sonarsource/sonarqube-scan-action@master
sonarsource/sonarqube-quality-gate-action@master

I did add steps before calling these actions to check out the code and configure the SonarQube properties file. Note that I did parse the version number from docs/version and include that in the properties file. So the version of the code will now be attached to the scan.

JohnHalleyGotway commented 5 months ago

I notice the following exception in the METdataio SonarQube scan. Need to ask @hsoh-u about this exception.

It also appears in the Nightly Build output in seneca:/d1/projects/MET/MET_regression/sonarqube_METdataio/NB20240403/run_sonarqube_20240403.log.

ERROR: Error when running: 'node -v'. Is Node.js available during analysis?
org.sonar.plugins.javascript.nodejs.NodeCommandException: Error when running: 'node -v'. Is Node.js available during analysis?
    at org.sonar.plugins.javascript.nodejs.NodeCommand.start(NodeCommand.java:85)
    at org.sonar.plugins.javascript.nodejs.NodeCommandBuilderImpl.getVersion(NodeCommandBuilderImpl.java:217)
    at org.sonar.plugins.javascript.nodejs.NodeCommandBuilderImpl.checkNodeCompatibility(NodeCommandBuilderImpl.java:173)
    at org.sonar.plugins.javascript.nodejs.NodeCommandBuilderImpl.build(NodeCommandBuilderImpl.java:146)
    at org.sonar.plugins.javascript.eslint.BridgeServerImpl.initNodeCommand(BridgeServerImpl.java:253)
    at org.sonar.plugins.javascript.eslint.BridgeServerImpl.startServer(BridgeServerImpl.java:181)
    at org.sonar.plugins.javascript.eslint.BridgeServerImpl.startServerLazily(BridgeServerImpl.java:293)
    at org.sonar.plugins.javascript.eslint.AbstractEslintSensor.execute(AbstractEslintSensor.java:73)
    at org.sonar.plugins.javascript.eslint.CssRuleSensor.execute(CssRuleSensor.java:96)
    at org.sonar.scanner.sensor.AbstractSensorWrapper.analyse(AbstractSensorWrapper.java:64)
    at org.sonar.scanner.sensor.ModuleSensorsExecutor.execute(ModuleSensorsExecutor.java:88)
    at org.sonar.scanner.sensor.ModuleSensorsExecutor.lambda$execute$1(ModuleSensorsExecutor.java:61)
    at org.sonar.scanner.sensor.ModuleSensorsExecutor.withModuleStrategy(ModuleSensorsExecutor.java:79)
    at org.sonar.scanner.sensor.ModuleSensorsExecutor.execute(ModuleSensorsExecutor.java:61)
    at org.sonar.scanner.scan.SpringModuleScanContainer.doAfterStart(SpringModuleScanContainer.java:82)
    at org.sonar.core.platform.SpringComponentContainer.startComponents(SpringComponentContainer.java:188)
    at org.sonar.core.platform.SpringComponentContainer.execute(SpringComponentContainer.java:167)
    at org.sonar.scanner.scan.SpringProjectScanContainer.scan(SpringProjectScanContainer.java:398)
    at org.sonar.scanner.scan.SpringProjectScanContainer.scanRecursively(SpringProjectScanContainer.java:394)
    at org.sonar.scanner.scan.SpringProjectScanContainer.doAfterStart(SpringProjectScanContainer.java:363)
    at org.sonar.core.platform.SpringComponentContainer.startComponents(SpringComponentContainer.java:188)
    at org.sonar.core.platform.SpringComponentContainer.execute(SpringComponentContainer.java:167)
    at org.sonar.scanner.bootstrap.SpringGlobalContainer.doAfterStart(SpringGlobalContainer.java:139)
    at org.sonar.core.platform.SpringComponentContainer.startComponents(SpringComponentContainer.java:188)
    at org.sonar.core.platform.SpringComponentContainer.execute(SpringComponentContainer.java:167)
    at org.sonar.batch.bootstrapper.Batch.doExecute(Batch.java:71)
    at org.sonar.batch.bootstrapper.Batch.execute(Batch.java:65)
    at org.sonarsource.scanner.api.internal.batch.BatchIsolatedLauncher.execute(BatchIsolatedLauncher.java:46)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    at java.base/java.lang.reflect.Method.invoke(Unknown Source)
    at org.sonarsource.scanner.api.internal.IsolatedLauncherProxy.invoke(IsolatedLauncherProxy.java:60)
    at com.sun.proxy.$Proxy0.execute(Unknown Source)
    at org.sonarsource.scanner.api.EmbeddedScanner.doExecute(EmbeddedScanner.java:189)
    at org.sonarsource.scanner.api.EmbeddedScanner.execute(EmbeddedScanner.java:138)
    at org.sonarsource.scanner.cli.Main.execute(Main.java:112)
    at org.sonarsource.scanner.cli.Main.execute(Main.java:75)
    at org.sonarsource.scanner.cli.Main.main(Main.java:61)
Caused by: java.io.IOException: Cannot run program "node": error=2, No such file or directory
    at java.base/java.lang.ProcessBuilder.start(Unknown Source)
    at java.base/java.lang.ProcessBuilder.start(Unknown Source)
    at org.sonar.plugins.javascript.nodejs.ProcessWrapperImpl.startProcess(ProcessWrapperImpl.java:43)
    at org.sonar.plugins.javascript.nodejs.NodeCommand.start(NodeCommand.java:82)
    ... 38 common frames omitted
Caused by: java.io.IOException: error=2, No such file or directory
    at java.base/java.lang.ProcessImpl.forkAndExec(Native Method)
    at java.base/java.lang.ProcessImpl.<init>(Unknown Source)
    at java.base/java.lang.ProcessImpl.start(Unknown Source)
    ... 42 common frames omitted

dtcenter / METdataio