search

grafeas / kritis

Deploy-time Policy Enforcer for Kubernetes applications
https://github.com/grafeas/kritis/blob/master/docs/binary-authorization.md
Apache License 2.0
694 stars 134 forks source link

v0.2.2 nil pointer dereference #594

Open zapman449 opened 3 years ago

zapman449 commented 3 years ago

Expected Behavior

not panic

Actual Behavior

panic

Steps to Reproduce the Problem

not sure

Environment, commands

n/a

Additional info

Add any other context about the problem here (e.g. screenshots, links)

2020/12/10 00:18:00 http: panic serving 10.168.3.200:56048: runtime error: invalid memory address or nil pointer dereference
goroutine 45 [running]:
net/http.(*conn).serve.func1(0xc0000ea0a0)
    /usr/local/go/src/net/http/server.go:1769 +0x139
panic(0x123a460, 0x212e600)
    /usr/local/go/src/runtime/panic.go:522 +0x1b5
github.com/grafeas/kritis/pkg/kritis/secrets.(*PgpKey).Fingerprint(...)
    /go/src/github.com/grafeas/kritis/pkg/kritis/secrets/pgpkey.go:69
github.com/grafeas/kritis/pkg/kritis/secrets.KeyAndFingerprint(0x0, 0x0, 0x472552, 0x2, 0x1583ae0, 0xc0005904e0, 0x47214a, 0xc0001e1960)
    /go/src/github.com/grafeas/kritis/pkg/kritis/secrets/pgpkey.go:127 +0x10b
github.com/grafeas/kritis/pkg/kritis/review.(*AttestorValidatingTransport).GetValidatedAttestations(0xc000364000, 0xc000044905, 0x9, 0xc0005ae9e0, 0x14, 0x13c87e6, 0x11, 0x13be9c0)
    /go/src/github.com/grafeas/kritis/pkg/kritis/review/validating_transport.go:44 +0xaf
github.com/grafeas/kritis/pkg/kritis/review.Reviewer.findUnsatisfiedAuths(0xc00051e660, 0xc000044905, 0x9, 0xc0000c9600, 0x1, 0x1, 0x159f640, 0xc00051e540, 0x0, 0xc0004f2a40, ...)
    /go/src/github.com/grafeas/kritis/pkg/kritis/review/review.go:149 +0x203
github.com/grafeas/kritis/pkg/kritis/review.Reviewer.ReviewGAP(0xc00051e660, 0xc000146bd0, 0x1, 0x1, 0xc000242240, 0x1, 0x1, 0xc000347880, 0x159f640, 0xc00051e540, ...)
    /go/src/github.com/grafeas/kritis/pkg/kritis/review/review.go:77 +0x24c
github.com/grafeas/kritis/pkg/kritis/admission.reviewGenericAttestationPolicy(0xc000146bd0, 0x1, 0x1, 0xc000044890, 0x6, 0xc000347880, 0xc000529050, 0xc000242240, 0x1, 0x1, ...)
    /go/src/github.com/grafeas/kritis/pkg/kritis/admission/admission.go:318 +0x2b5
github.com/grafeas/kritis/pkg/kritis/admission.reviewImages(0xc000146bd0, 0x1, 0x1, 0xc000044890, 0x6, 0xc000347880, 0xc000529050, 0xc000455860)
    /go/src/github.com/grafeas/kritis/pkg/kritis/admission/admission.go:270 +0x849
github.com/grafeas/kritis/pkg/kritis/admission.reviewPod(0xc000347880, 0xc000529050, 0xc000455860)
    /go/src/github.com/grafeas/kritis/pkg/kritis/admission/admission.go:336 +0x1f1
github.com/grafeas/kritis/pkg/kritis/admission.handlePod(0xc000528ea0, 0xc000529050, 0xc000455860, 0xc00052c601, 0x18)
    /go/src/github.com/grafeas/kritis/pkg/kritis/admission/admission.go:138 +0x148
github.com/grafeas/kritis/pkg/kritis/admission.ReviewHandler(0x15acb00, 0xc000310000, 0xc000176700, 0xc000455860)
    /go/src/github.com/grafeas/kritis/pkg/kritis/admission/admission.go:213 +0x391
main.main.func1(0x15acb00, 0xc000310000, 0xc000176700)
    /go/src/github.com/grafeas/kritis/cmd/kritis/admission/main.go:138 +0x48
net/http.HandlerFunc.ServeHTTP(0xc0003687e0, 0x15acb00, 0xc000310000, 0xc000176700)
    /usr/local/go/src/net/http/server.go:1995 +0x44
net/http.(*ServeMux).ServeHTTP(0x21479a0, 0x15acb00, 0xc000310000, 0xc000176700)
    /usr/local/go/src/net/http/server.go:2375 +0x1d6
net/http.serverHandler.ServeHTTP(0xc000379e10, 0x15acb00, 0xc000310000, 0xc000176700)
    /usr/local/go/src/net/http/server.go:2774 +0xa8
net/http.(*conn).serve(0xc0000ea0a0, 0x15b4f40, 0xc000354040)
    /usr/local/go/src/net/http/server.go:1878 +0x851
created by net/http.(*Server).Serve
    /usr/local/go/src/net/http/server.go:2884 +0x2f4
coatsnmore commented 3 years ago

I am also seeing this issue.

We are beginning our journey to implement binary authorization and are hoping to run Grafeas/Kritis standalone in our own cluster. To begin our learning, we are following the standalone docs.

The error @zapman449 indicated happens when running ./no_attestation.sh. I am not a Go Developer or certificate expert, so I don't really understand the issue, but it seems like the Kritis Validation Hook doesn't like something about the key fingerprint.

Expected Behavior

When I run kubectl apply -f pod.yaml at the end of no_attestation.sh I expect a message back like Error from server: error when creating "pod.yaml": admission webhook "kritis-validation-hook.grafeas.io" denied the request: image gcr.io/kritis-tutorial/java-with-vulnz@sha256:<hash> is not attested, as indicated in the documentation.

Actual Behavior

kubectl apply -f pod.yaml results in Error from server (InternalError): error when creating "pod.yaml": Internal error occurred: failed calling webhook "kritis-validation-hook.grafeas.io": Post https://kritis-validation-hook.default.svc:443/?timeout=30s: EOF

When looking at the logs, I can see the same logs as OP of this issue.

image

and indicates there is some issue at https://github.com/grafeas/kritis/blob/master/pkg/kritis/secrets/pgpkey.go#L69

Steps to Reproduce

Follow https://github.com/grafeas/kritis/blob/master/docs/standalone/README.md

frayer commented 3 years ago

I'm having the same issue, and my working theory at the moment is that the interface on AttestationAuthority has changed in code and in the instructions since the v0.2.2 release. The Helm chart used when you deploy the Kritis server though still references the v0.2.2 release, and it's not able to find the public key data due to the schema change on AttestationAuthority, leading to some nil error pointers at runtime.

I'll be going through the same routine described in docs/standalone to validate, but from the v0.2.2 tag.

Compare the interface of AttestationAuthority at the v0.2.2 tag:

apiVersion: kritis.grafeas.io/v1beta1
kind: AttestationAuthority
metadata:
  name: kritis-authority
  namespace: default
spec:
  noteReference: v1beta1/projects/standalone
  privateKeySecretName: attestor
  publicKeyData: $PUBLIC_KEY

to what it currently is on master today:

apiVersion: kritis.grafeas.io/v1beta1
kind: AttestationAuthority
metadata:
  name: kritis-authority
  namespace: default
spec:
  noteReference: v1beta1/projects/standalone
  publicKeys:
  - keyType: PGP 
    keyId: $KEY_FINGERPRINT
    asciiArmoredPgpPublicKey: $PUBLIC_KEY
frayer commented 3 years ago

Working through the docs/standalone README in the v0.2.2 branch I have gotten past step 2, but am now stuck in step 3 when running ./attestation_created.sh. I'm getting the same issue described in #384

frayer commented 3 years ago

I'm able to get past the issue described in #384 now. The interface is nil error I was getting was because I had encrypted the private key generated by gpg with a password. As described in the PrivateKey type, it's PrivateKey field is not available until calling Decrypt(passphrase []byte) when encrypted like this (makes sense). Therefore the field was nil.

Two approaches for getting past that:

  1. The public/private keypair are generated when executing docs/standalone/no_attestation.sh. You can skip encrypting the keys by just pressing <Enter> when prompted for a password. You'll get a warning when doing so.
  2. If you do password protect the keys, you'll need that password to decrypt the private key before using it to create a signature on your attestation. The current implementation of create_attestation.go eventually expects to find that in the attestor K8s Secret created when executing docs/standalone/no_attestation.sh. That Secret is created by the following:
kubectl create secret generic attestor --from-file=public=gpg.pub --from-file=private=gpg.priv

and can be updated to the following to capture the passphrase used if you encrypted the keys

kubectl create secret generic attestor --from-file=public=gpg.pub --from-file=private=gpg.priv --from-literal=passphrase=<password provided to gpg during creation>

After that I was also getting an error related to an invalid Note reference on the AttestationAuthority, and had to change the Note reference to projects/kritis from v1beta1/projects/standalone which is inside no_attestation.sh. It now looks like the following:

apiVersion: kritis.grafeas.io/v1beta1
kind: AttestationAuthority
metadata:
  name: kritis-authority
  namespace: default
spec:
  noteReference: projects/kritis
  privateKeySecretName: attestor
  publicKeyData: $PUBLIC_KEY

I'm now able to POST the occurrence and attestation signature to Grafeas using create_attestation.go, but am now getting a Base64 decoding issue logged from the kritis-validation-hook Pod when trying to kubectl apply -f pod.yaml.

I0107 22:14:29.314411       1 admission.go:172] Starting admission review handler
version: v0.2.2
commit: bea073f2a2f299af94363dc399b7780fde8f2afc
I0107 22:14:29.315323       1 admission.go:137] handling pod java in...
I0107 22:14:29.315348       1 admission.go:258] Reviewing images for &Pod{ ... removed ... }
I0107 22:14:29.319934       1 admission.go:269] Found 1 Generic Attestation Policies
I0107 22:14:29.320693       1 review.go:68] Check if gcr.io/kritis-tutorial/java-with-vulnz@sha256:358687cfd3ec8e1dfeb2bf51b5110e4e16f6df71f64fba01986f720b2fcba68a has valid Attestations.
I0107 22:14:29.320725       1 review.go:71] Validating against GenericAttestationPolicy my-gap
I0107 22:14:29.371644       1 validating_transport.go:65] Cannot base64 decode signature: illegal base64 data at input byte 0
I0107 22:14:29.371673       1 strategy.go:49] Handling attestation via LoggingStrategy
I0107 22:14:29.371680       1 strategy.go:53] No valid attestations found for image gcr.io/kritis-tutorial/java-with-vulnz@sha256:358687cfd3ec8e1dfeb2bf51b5110e4e16f6df71f64fba01986f720b2fcba68a. Proceeding with next checks
I0107 22:14:29.371709       1 admission.go:319] Denying &Pod{ ... removed ... }
I0107 22:14:29.375552       1 admission.go:281] No ISPs found in namespace default
ooq commented 3 years ago

@frayer Thanks for the investigation, it looks like the bug might be due to schema change in AttestationAuthority. I'll look into a fix.

frayer commented 3 years ago

I got through that last Base64 decoding hurdle for at least v0.2.2 of Kritis. Kritis was expecting the PGP Signature to be Base64 encoded, so I made that change in create_attestation.go before it writes the signature to K8s. After that I'm able to get the pass/fail behavior expected when attempting to apply the Pod.

All the changes I did can be viewed in this diff: https://github.com/grafeas/kritis/compare/v0.2.2...frayer:v0.2.2-standalone-install

I also updated to Helm 3 and Grafeas v0.1.6 along the way.

I'm not sure it's merge-worthy for a PR at this point since, as you mentioned, some of the Schemas may have evolved since the v0.2.2 release. It could help those trying to get v0.2.2 to work though.

I'd be happy to help with updating the docs in master, but also wondering if there's a new versioned release (beta or otherwise) in the works where there would be an updated kritis-server image to reference. That shows up in the docs when installing via Helm, which at the moment still points at gcr.io/kritis-project/kritis-server:v0.2.2, leading to some of the mismatches I originally started with.

Are the new Schema types captured in gcr.io/kritis-project/kritis-server:latest ?