kubewarden / kubewarden-controller

Manage admission policies in your Kubernetes cluster with ease
https://kubewarden.io
Apache License 2.0
191 stars 33 forks source link

k8s-objects library automatically resets to indirect after 'go mod tidy' #558

Closed anishnagaraj closed 1 year ago

anishnagaraj commented 1 year ago

Is there an existing issue for this?

Current Behavior

In one of my policy I have to validate the argoproj.io/v1alpha1/AppProject object. To do this I need the following library in my go.mod

github.com/argoproj/argo-cd/v2 v2.4.15

But upon running go mod tidy the library github.com/kubewarden/k8s-objects v1.27.0-kw2 moves from the direct dependencies section to the indirection section.

Because of this, whenever I try to run this command tinygo build -o policy.wasm -target=wasi -no-debug ., I get the following error.

../../../../../../../../usr/local/go/src/os/user/cgo_lookup_cgo.go:14:6: not implemented: build constraints in #cgo line ../../../../../../../../usr/local/go/src/os/user/cgo_lookup_cgo.go:18:10: fatal: 'pwd.h' file not found ../../../../../../../../usr/local/go/src/os/user/getgrouplist_unix.go:12:10: fatal: 'grp.h' file not found

Expected Behavior

I expect policy.wasm to be built

Steps To Reproduce

  1. Use the following go.mod content:

go 1.20

require (
    github.com/argoproj/argo-cd/v2 v2.4.15
    github.com/francoispqt/onelog v0.0.0-20190306043706-8c2bb31b10a4
    github.com/kubewarden/k8s-objects v1.27.0-kw2
    github.com/kubewarden/policy-sdk-go v0.5.0
    github.com/stretchr/testify v1.8.4
    github.com/wapc/wapc-guest-tinygo v0.3.3
    gopkg.in/yaml.v3 v3.0.1
)

require (
    github.com/davecgh/go-spew v1.1.1 // indirect
    github.com/francoispqt/gojay v0.0.0-20181220093123-f2cc13a668ca // indirect
    github.com/go-openapi/strfmt v0.21.3 // indirect
    github.com/kr/text v0.2.0 // indirect
    github.com/niemeyer/pretty v0.0.0-20200227124842-a10e7caefd8e // indirect
    github.com/pmezard/go-difflib v1.0.0 // indirect
    gopkg.in/check.v1 v1.0.0-20200227125254-8fa46927fb4f // indirect
)

replace (
github.com/go-openapi/strfmt => github.com/kubewarden/strfmt v0.1.3

    // https://github.com/golang/go/issues/33546#issuecomment-519656923
    github.com/go-check/check => github.com/go-check/check v0.0.0-20180628173108-788fd7840127

    github.com/golang/protobuf => github.com/golang/protobuf v1.4.2
    github.com/gorilla/websocket => github.com/gorilla/websocket v1.4.2
    github.com/grpc-ecosystem/grpc-gateway => github.com/grpc-ecosystem/grpc-gateway v1.16.0
    github.com/improbable-eng/grpc-web => github.com/improbable-eng/grpc-web v0.0.0-20181111100011-16092bd1d58a

    // Avoid CVE-2022-28948
    gopkg.in/yaml.v3 => gopkg.in/yaml.v3 v3.0.1

    // https://github.com/kubernetes/kubernetes/issues/79384#issuecomment-505627280
    k8s.io/api => k8s.io/api v0.23.1
    k8s.io/apiextensions-apiserver => k8s.io/apiextensions-apiserver v0.23.1
    k8s.io/apimachinery => k8s.io/apimachinery v0.23.1
    k8s.io/apiserver => k8s.io/apiserver v0.23.1
    k8s.io/cli-runtime => k8s.io/cli-runtime v0.23.1
    k8s.io/client-go => k8s.io/client-go v0.23.1
    k8s.io/cloud-provider => k8s.io/cloud-provider v0.23.1
    k8s.io/cluster-bootstrap => k8s.io/cluster-bootstrap v0.23.1
    k8s.io/code-generator => k8s.io/code-generator v0.23.1
    k8s.io/component-base => k8s.io/component-base v0.23.1
    k8s.io/component-helpers => k8s.io/component-helpers v0.23.1
    k8s.io/controller-manager => k8s.io/controller-manager v0.23.1
    k8s.io/cri-api => k8s.io/cri-api v0.23.1
    k8s.io/csi-translation-lib => k8s.io/csi-translation-lib v0.23.1
    k8s.io/kube-aggregator => k8s.io/kube-aggregator v0.23.1
    k8s.io/kube-controller-manager => k8s.io/kube-controller-manager v0.23.1
    k8s.io/kube-proxy => k8s.io/kube-proxy v0.23.1
    k8s.io/kube-scheduler => k8s.io/kube-scheduler v0.23.1
    k8s.io/kubectl => k8s.io/kubectl v0.23.1
    k8s.io/kubelet => k8s.io/kubelet v0.23.1
    k8s.io/legacy-cloud-providers => k8s.io/legacy-cloud-providers v0.23.1
    k8s.io/metrics => k8s.io/metrics v0.23.1
    k8s.io/mount-utils => k8s.io/mount-utils v0.23.1
    k8s.io/pod-security-admission => k8s.io/pod-security-admission v0.23.1
    k8s.io/sample-apiserver => k8s.io/sample-apiserver v0.23.1

) 
  1. Run go mod tidy and go mod vendor

  2. Run tinygo build -o policy.wasm -target=wasi -no-debug .

Environment

- OS:Linux, Mac
- Architecture:arm64

Anything else?

No response

flavio commented 1 year ago

That's a really good question.

AppProject makes use of the Kubernetes types. We cannot consume them with TinyGo, that's why we had to generate our lightweight version of them.

Moreover, the AppProject also has a series of additional methods that provide additional features. I see for example that portions of GRPC are included. This could make the compilation fail or, even if it passes, they will make the resulting binary bigger.

The key question is: what is your validation going to do? Does it need all these helper methods, functions and structs defined inside of the source code that is being included?

Solution 1: query the JSON

You can use the gjson library to extract the relevant fields from the incoming request. This approach is described inside of our docs.

Let's say you want to validate AppProject.Spec.Destinations. You can use gjson to extract this portion from the request and perform your validation.

Solution 2: write your own struct

By default, Go ignores unknown keys when unmarshalling a JSON object into a struct. You can leverage that to create your own stripped down version of AppProject. This struct will include only the fields you care about.

For example:

import (
    apimachinery_pkg_apis_meta_v1 "github.com/kubewarden/k8s-objects/apimachinery/pkg/apis/meta/v1"
)

type AppProject struct {
    APIVersion string `json:"apiVersion,omitempty"`
    Kind string `json:"kind,omitempty"`
    Metadata *apimachinery_pkg_apis_meta_v1.ObjectMeta `json:"metadata,omitempty"`
    Spec *AppProjectSpec `json:"spec,omitempty"`
}

type AppProjectSpec struct {
    // Destinations contains list of destinations available for deployment
    Destinations []ApplicationDestination `json:"destinations,omitempty" protobuf:"bytes,2,name=destination"`
}

// ApplicationDestination holds information about the application's destination
type ApplicationDestination struct {
    // Server specifies the URL of the target cluster's Kubernetes control plane API. This must be set if Name is not set.
    Server string `json:"server,omitempty" protobuf:"bytes,1,opt,name=server"`
    // Namespace specifies the target namespace for the application's resources.
    // The namespace will only be set for namespace-scoped resources that have not set a value for .metadata.namespace
    Namespace string `json:"namespace,omitempty" protobuf:"bytes,2,opt,name=namespace"`
    // Name is an alternate way of specifying the target cluster by its symbolic name. This must be set if Server is not set.
    Name string `json:"name,omitempty" protobuf:"bytes,3,opt,name=name"`

    // nolint:govet
    isServerInferred bool `json:"-"`
}

As you can see, I just copied and pasted the relevant snippet from the struct definitions from the Argo project. I just replaced some metadata attributes to use the k8s objects module provided by Kubewarden. Note: you can even avoid defining them if you don't care about their contents.

Wrapping up

I hope that helps. Feel free to provide us more details about the type of validation you're trying to build. We can help you figuring out the right approach.

We are also considering a way to generate some TinyGo compatible struct definitions starting from an existing Go file (or an openapi spec).

anishnagaraj commented 1 year ago

This helps! I just want to validate the labels for now. So I will go with solution 1 to ge the kind, apigroup and labels.

anishnagaraj commented 1 year ago

Solution 1 is suitable for my scenario

flavio commented 1 year ago

If you need to validate labels, you could use the safe-labels policy and deploy it to target the ArgoProject resources.

However, here here you can see how safe-labels uses qjson to validate the labels.

anishnagaraj commented 1 year ago

But then I also need the kind and apiVersion to make sure the policy is applied on the right object.

flavio commented 1 year ago

But then I also need the kind and apiVersion to make sure the policy is applied on the right object.

Yes, exactly