search

department-of-veterans-affairs / codeql-tools

MIT License
4 stars 2 forks source link

Enforce the use of the latest CodeQL version and decommission the direct flagging of out of date CLI versions #83

Open lindluni opened 12 months ago

lindluni commented 12 months ago

Recently we pushed a change into the GitHub Actions-based workflow that ensures the latest version of CodeQL is always in use: https://github.com/department-of-veterans-affairs/codeql-tools/commit/b9294be412fcaa337ec9893af50c1bbc2b8a7b73

With that change, even if a user has preinstalled CodeQL on their self-hosted runners, GitHub will always download the latest version of CodeQL if it is not available on the runner.

I propose, we do the same for the Jenkins Shared Library as well. If the user sets the INSTALL_CODEQL flag to false, meaning they've preinstalled CodeQL, we check the version of CodeQL they have installed against the latest version of CodeQL and if their preinstalled version is not the latest, we ignore their request to skip installing CodeQL and download and leverage the latest version regardless.

With this, we can stop reporting out on whether scans are being performed with the latest version of the CLI as we have forced the use of the latest version and there is no way for teams to work around this requirement.

Now we only need to confirm that scans are being performed at least once a week to ensure the latest version of the CLI is being used, which is already being reported on in the missing database and missing analysis columns in the dashboard, as these columns report out two scenarios, if no scan has ever been performed, that language is included in these columns, and the if no scan has been performed on that language in the last 7 days, we treat it as if no scan has ever been performed and flag it as missing as well. So users will still receive email and issue notifications that they are missing scans (and by extension outdated CLI's) for specific languages.

This is a minimal effort to do, and would drastically reduce the number of checks we have to perform as part of enforcement and metrics, greatly reducing our runtime for those apps, and would prevent us from encountering scenarios where GitHub rapidly updates the released versions of the CodeQL tool and us erroneously sending out notifications to teams.

Boberski commented 12 months ago

We have discussed this internally and agree this would be an improvement on the Jenkins side, so it will work the same as on the GitHub CI side. We are good with it. Note tech note section Updating CodeQL CLI Bundle should remain, but it will need to be modified to say something like, that if it isn't updated, the CodeQL bundle will be downloaded with each scan.