Closed wind57 closed 3 years ago
oh! so I can pass -o json
to helm datree
... this simplifies things. Though the output is weird:
"InvalidK8sFiles": [
{
"Path": "/tmp/manifest.yaml",
"ValidationErrors": [
{
"ErrorMessage": "For field apiVersion: apiVersion must be one of the following: \"rbac.authorization.k8s.io/v1\""
}
]
}
]
/tmp/manifest.yaml
- that is not my path, neither my file. I guess you copy the manifests and then validate them and show that incorrect path. Error: plugin "datree" exited with error
at the end; that is not part of the json itself. This is what I currently ended up doing :
IFS=$'\n'
declare -a datree_result
datree_result=$(helm datree test src/main/helm -o json | jq -c 'paths | select(.[-1] == "ErrorMessage")')
if [[ "${datree_result[@]}" =~ "ErrorMessage" ]]
then
echo "failure"
exit 1
fi
@wind57 I forgot to put myself as a watcher to this repo, so I missed your issue - sorry about that :disappointed:
I'm glad to see that you find a workaround, but I still want to answer your questions:
/tmp/manifest.yaml
Error: plugin "datree" exited with error
is an error code that is generated by helm (not by Datree), but we deployed a new version that is also handling itI'm wondering why do parse Datree's output? If it fails, it should have exit code 1 anyway...
FYI, our latest update includes a small change to the way we generate these tmp
files.
Since we added pipe support in datree
cli, we can now forward helm template
output directly to the datree cli. This results in a slightly different filename in tmp
folder.
Should be something like /var/folders/4k/7h6k03k11rv8gy62py4fxqxh0000gm/T/datree_temp_130557047.yaml
I'm wondering why do parse Datree's output? If it fails, it should have exit code 1 anyway...
Right. But what is the use of that to the people using this? I mean I would like to show a human readable error message of what went wrong and where. So I wanted to use -o json
, so that I could parse it with jq
and extract the error message(s).
It seems you validate errors vs "suggestions" differently, which makes my code really involved. Let me explain:
apiVersion: rbac.authorization.k8s.io/v1123
in one of the files, and run: helm datree test src/main/helm -o json | jq
, I would get a json that contains: "InvalidK8sFiles": [
{
"Path": "/tmp/manifest.yaml",
"ValidationErrors": [
{
"ErrorMessage": "could not find schema for ClusterRole"
}
]
}
]
Good. I can check if the output has ErrorMessage
and parse the output and get the proper error message and display that to the end user.
spec:
type: NodePort
ports:
- port: 8080
targetPort: 8080
nodePort: 30008
I get a json that contains and run : helm datree test src/main/helm -o json | jq
"EvaluationResults": {
"FileNameRuleMapper": {
"/tmp/manifest.yaml": {
"7": {
"ID": 7,
"Name": "Prevent Service from exposing node port",
"FailSuggestion": "Incorrect value for key `type` - `NodePort` will open a port on all nodes where it can be reached by the network external to the cluster",
"Count": 1
}
}
}
Yeah, no ErrorMessage
, so my code is now supposed to check in a various place for a different thing.
imho, a simpler type like:
"EvaluationResult" : {
"Type" : "Suggestion/Error"
"SuggestionIfPresent" : "....."
....
}
So my suggestion is a unified, single place, for errors. I hope this makes sense.
@dimabru as to your suggestion, you should really maintain a "mapping"... What I mean by that: if I have 20 manifest files in a single helm chart and I display the error being in /tmp/manifest.yaml
, the end user looking at those errors is going to be in the blur. Where exactly is the problem?
Thank you.
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
An interesting tool overall!
I do not see a way to use it in a CI/CD env, unless I am missing the obvious.
It would be great if it had something along the lines of
--quiet
and--error-count
, so that I could parse the result ofhelm datree test ...
and find our if I need to fail the build or not. Currently this is very cumbersome to do (I get the output and parse it, which to put it mildly, simply sucks).Thank you for looking into this.