Quick start is broken (sa: workload:default cannot get featureflags)

agardnerIT commented 4 months ago

$ kubectl -n workload logs busybox-curl-658868f655-bnk7t -c flagd

                 ______   __       ________   _______    ______      
                /_____/\ /_/\     /_______/\ /______/\  /_____/\     
                \::::_\/_\:\ \    \::: _  \ \\::::__\/__\:::_ \ \    
                 \:\/___/\\:\ \    \::(_)  \ \\:\ /____/\\:\ \ \ \   
                  \:::._\/ \:\ \____\:: __  \ \\:\\_  _\/ \:\ \ \ \  
                   \:\ \    \:\/___/\\:.\ \  \ \\:\_\ \ \  \:\/.:| | 
                    \_\/     \_____\/ \__\/\__\/ \_____\/   \____/_/ 

2024-02-26T01:06:30.627Z        info    cmd/start.go:100        flagd version: v0.9.0 (534b5bf654384689964c0bab5f543457d29dab8f), built at: 2024-02-20  {"component": "start"}
2024-02-26T01:06:30.628Z        info    kubernetes/kubernetes_sync.go:90        starting kubernetes sync notifier for resource: flags/sample-flags      {"component": "sync", "sync": "kubernetes"}
2024-02-26T01:06:30.628Z        info    flag-evaluation/connect_service.go:223  Flag Evaluation listening at [::]:8080  {"component": "service"}
2024-02-26T01:06:30.628Z        info    flag-evaluation/connect_service.go:243  metrics and probes listening at 8014    {"component": "service"}
2024-02-26T01:06:30.633Z        error   kubernetes/kubernetes_sync.go:96        error with the initial fetch: unable to fetch FeatureFlag flags/sample-flags: featureflags.core.openfeature.dev "sample-flags" is forbidden: User "system:serviceaccount:workload:default" cannot get resource "featureflags" in API group "core.openfeature.dev" in the namespace "flags"     {"component": "sync", "sync": "kubernetes"}
github.com/open-feature/flagd/core/pkg/sync/kubernetes.(*Sync).Sync
        /src/core/pkg/sync/kubernetes/kubernetes_sync.go:96
github.com/open-feature/flagd/core/pkg/runtime.(*Runtime).Start.func2
        /src/core/pkg/runtime/runtime.go:81
golang.org/x/sync/errgroup.(*Group).Go.func1
        /go/pkg/mod/golang.org/x/sync@v0.5.0/errgroup/errgroup.go:75
2024-02-26T01:06:30.633Z        info    runtime/runtime.go:89   Shutting down server... {"component": "runtime"}
2024-02-26T01:06:30.633Z        info    runtime/runtime.go:91   Server successfully shutdown.   {"component": "runtime"}
2024-02-26T01:06:30.633Z        fatal   cmd/start.go:138        errgroup closed with error: sync provider returned error: error with the initial fetch: unable to fetch FeatureFlag flags/sample-flags: featureflags.core.openfeature.dev "sample-flags" is forbidden: User "system:serviceaccount:workload:default" cannot get resource "featureflags" in API group "core.openfeature.dev" in the namespace "flags"   {"component": "start"}
github.com/open-feature/flagd/flagd/cmd.init.func1
        /src/flagd/cmd/start.go:138
github.com/spf13/cobra.(*Command).execute
        /go/pkg/mod/github.com/spf13/cobra@v1.8.0/command.go:987
github.com/spf13/cobra.(*Command).ExecuteC
        /go/pkg/mod/github.com/spf13/cobra@v1.8.0/command.go:1115
github.com/spf13/cobra.(*Command).Execute
        /go/pkg/mod/github.com/spf13/cobra@v1.8.0/command.go:1039
github.com/open-feature/flagd/flagd/cmd.Execute
        /src/flagd/cmd/root.go:37
main.main
        /src/flagd/main.go:30
runtime.main
        /usr/local/go/src/runtime/proc.go:271

Cert Manager Version

Installed from Helm chart: v1.14.3

Operator Version

Installed from Helm chart: v0.5.4

kubectl version

$ kubectl version
Client Version: v1.29.2
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
Server Version: v1.29.2

toddbaert commented 4 months ago

I just ran through this on a new cluster, and it seems to work fine. The only thing I think I did differently than you is that I used an older version of cert-manager (1.13.2 as in the quick-start).

Can you give any other details? I somehow doubt this is related to cert mgr.

I'm also using a 1.27.3 cluster. I'll update and try again with something newer.

thisthat commented 4 months ago

I just also tried with cert-manager installed via Helm at v1.14.3, maybe you have a SA policy configured? 🤔 the error message unable to fetch FeatureFlag flags/sample-flags: featureflags.core.openfeature.dev "sample-flags" is forbidden: seems that you have a policy that doesn't allow your ServiceAccount mounted in flagD to hit the KubeAPIs to fetch the CRD

open-feature / open-feature-operator