Add all Hadolint rules

[Sancretor]

Is your feature request related to a problem? Please describe.**

The plugin only implements one hadolint rule, and we need all of them to be integrated.

Describe the solution you'd like

Add all the remaining rules inside the XML file that describes them.

Describe alternatives you've considered

No alternatives.

Additional context

Only one rule was implemented for testing purposes during first development phase. now that the plugin is working, we should add them all.

[JohannesBe]

I suppose this is related to the following messages?

INFO: An issue for rule 'Hadolint.SC2039' was detected by Hadolint but this rule is deactivated in current analysis.
INFO: An issue for rule 'Hadolint.DL3003' was detected by Hadolint but this rule is deactivated in current analysis.
INFO: An issue for rule 'Hadolint.DL4001' was detected by Hadolint but this rule is deactivated in current analysis.
INFO: An issue for rule 'Hadolint.DL3025' was detected by Hadolint but this rule is deactivated in current analysis.
INFO: An issue for rule 'Hadolint.DL3025' was detected by Hadolint but this rule is deactivated in current analysis.
INFO: An issue for rule 'Hadolint.DL3018' was detected by Hadolint but this rule is deactivated in current analysis.
INFO: An issue for rule 'Hadolint.DL3019' was detected by Hadolint but this rule is deactivated in current analysis.
INFO: An issue for rule 'Hadolint.DL3007' was detected by Hadolint but this rule is deactivated in current analysis.
INFO: An issue for rule 'Hadolint.DL3018' was detected by Hadolint but this rule is deactivated in current analysis.
INFO: An issue for rule 'Hadolint.DL3019' was detected by Hadolint but this rule is deactivated in current analysis.
INFO: An issue for rule 'Hadolint.DL3018' was detected by Hadolint but this rule is deactivated in current analysis.

Can I help you with implementing these rules? What should happen for each rule?

[JohannesBe]

Is it just adding them to the xml over here: https://github.com/cnescatlab/sonar-hadolint-plugin/blob/dev/src/main/resources/rules/hadolint-rules.xml ?

It seems that Hadolint has two kinds of rules, DL and SC rules. DL are real docker-related issues, so I guess we want to implement these for sure. Can we use the sonar shellchecker maybe to delegate / "borrow" SC rules (e.g. see https://github.com/sbaudoin/sonar-shellcheck/tree/master/src/main/resources/org/sonar/l10n/shellcheck/rules/shellcheck )? Need to check licenses though.

[Sancretor]

Hi @JohannesBe

I already started working on this on a dedicated branch, if you are interested : https://github.com/cnescatlab/sonar-hadolint-plugin/tree/feature/add-hadolint-rules

I was focusing on the native DL rules, and was thinking, just like you, about checking licences to use what was done in the shellcheck plugin.

Sorry for the late responses, I'm on holidays...

[JohannesBe]

Hi @Sancretor,

No problem, it's August after all :slightly_smiling_face:

I wrote a shellscript to retrieve all SC rules and parse them into xml. This way we can easily re-run our script to sync with the shellcheck sonar repository periodically.

On the way I noticed that the shellcheck repository has tags for their rules. Therefore I created #6.

#!/bin/bash
# shellcheck disable=SC2001

set -euxo pipefail

ORIG=$(pwd)
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"

function cleanup() {
    rm -rf "/tmp/shellcheck" || true
    cd "$ORIG"
}

trap cleanup EXIT

cd "/tmp"
rm -rf "shellcheck"
git clone "https://github.com/sbaudoin/sonar-shellcheck.git" "shellcheck"
cd "shellcheck"

set +x
FILES=$(find "src/" | grep -E '.*/SC[0-9]+\.json' | sort)

rm -rf blobs.json
touch blobs.json

for FILE in $FILES
do
    KEY=$(sed 's/.*\(SC[0-9]\+\)\.json/\1/g' <<< "$FILE")
    jq -c '{rule:{key:"'"$KEY"'",name:.title,description:.title,type:.type,remediationFunction:.remediation.func|ascii_upcase|sub("[^A-Z0-9]";"_"),remediationFunctionBaseEffort:.remediation.constantCost,status:.status,tags:{tag:.tags},cardinality:"SINGLE"}}' "$FILE" >> blobs.json
    echo -n "$KEY,"
done
echo ""
set -x

jq -s '{"hadolint-rules":.}' < blobs.json > full.json
npm i json2xml-cli .
./node_modules/json2xml-cli/src/json2xml.js -i full.json --header -o "/tmp/shellcheck/out.xml"

xmllint --format "/tmp/shellcheck/out.xml" > "$DIR/shellcheck.xml"
set +x
echo "------------------------------------------------------------------------------------"
echo "|                                                                                  |"
echo "| [SUCCESS]                                                                        |"
echo "| Conversion successful! File can be found at:                                     |"
echo -n "| " && printf "%-80s" "$DIR/shellcheck.xml" && echo " |"
echo "|                                                                                  |"
echo "------------------------------------------------------------------------------------"

Output: shellcheck.xml.txt

[JohannesBe]

Some comments:

• Finding a good json -> xml conversion tool proved to be a rabbit-hole. In the end I settled with a kind-of small node.js library called json2xml-cli, but I'm kind of not happy randomly installing node-js libraries. It is small-ish though, speaking in npm dependency-hell norms at least.
• The script requires the jq dependency, a well-established CLI tool for parsing JSON. I use this one to pre-parse the JSON into a useful format for the xml conversion tool.
• The shellcheck sonarqube extension does not have a distinction between name and description, for now I just copied their title field into both. Is this okay for us?
• The shellcheck plugin has an Apache license. We should therefore provide attribution, include the original license file and state the changes we made (e.g. json -> xml conversion)

[JohannesBe]

There is still a bug in the shell script I think, the <status></status> should be upper case. I think instead of

    jq -c '{rule:{key:"'"$KEY"'",name:.title,description:.title,type:.type,remediationFunction:.remediation.func|ascii_upcase|sub("[^A-Z0-9]";"_"),remediationFunctionBaseEffort:.remediation.constantCost,status:.status,tags:{tag:.tags},cardinality:"SINGLE"}}' "$FILE" >> blobs.json

the following should work:

    jq -c '{rule:{key:"'"$KEY"'",name:.title,description:.title,type:.type,remediationFunction:.remediation.func|ascii_upcase|sub("[^A-Z0-9]";"_"),remediationFunctionBaseEffort:.remediation.constantCost,status:.status|ascii_upcase,tags:{tag:.tags},cardinality:"SINGLE"}}' "$FILE" >> blobs.json

[JohannesBe]

There is also another bug regarding & for some rules, I will check if I can devise a fix for this soon™

[Sancretor]

Hi @JohannesBe

Again, having a script to automatically retrieve Shellcheck rules is a great idea ! Thank you ! Just like you, having a node.js script with dependencies is not what I would have liked.

Sorry for the editing... I've just had a look at Shellcheck repository, and instead of converting its rules to XML, we can simply use them as they are. We would have to change the way we import rules in the plugin, and splitting the big XML file into multiple json/html files. This would ensure coherence between the projects, don't you think ?