search

grafana / crossplane-provider-grafana

Crossplane provider of https://github.com/grafana/terraform-provider-grafana. Generated by https://github.com/upbound/upjet
Apache License 2.0
26 stars 14 forks source link

version 0.2.0 does not install any CRDS #15

Closed omerfsen closed 1 year ago

omerfsen commented 1 year ago

When I install grafana 0.2.0 provider

 kubectl crossplane install provider grafana/crossplane-provider-grafana:v0.2.0  provider-grafana

and do 

k get crds -A |grep crossplane |grep grafana

returns no result but when we do 

 kubectl crossplane install provider grafana/crossplane-provider-grafana:v0.1.0  provider-grafana

k get crds -A |grep crossplane |grep grafana
alertnotifications.grafana.jet.crossplane.io                              2023-02-24T10:24:27Z
apikeys.grafana.jet.crossplane.io                                         2023-02-24T10:24:27Z
builtinroleassignments.grafana.jet.crossplane.io                          2023-02-24T10:24:27Z
cloudapikeys.grafana.jet.crossplane.io                                    2023-02-24T10:24:27Z
cloudplugininstallations.grafana.jet.crossplane.io                        2023-02-24T10:24:27Z
cloudstacks.grafana.jet.crossplane.io                                     2023-02-24T10:24:27Z
dashboardpermissions.grafana.jet.crossplane.io                            2023-02-24T10:24:27Z
dashboards.grafana.jet.crossplane.io                                      2023-02-24T10:24:27Z
datasourcepermissions.grafana.jet.crossplane.io                           2023-02-24T10:24:27Z
datasources.grafana.jet.crossplane.io                                     2023-02-24T10:24:27Z
folderpermissions.grafana.jet.crossplane.io                               2023-02-24T10:24:27Z
folders.grafana.jet.crossplane.io                                         2023-02-24T10:24:27Z
librarypanels.grafana.jet.crossplane.io                                   2023-02-24T10:24:27Z
machinelearningjobs.grafana.jet.crossplane.io                             2023-02-24T10:24:27Z
organizations.grafana.jet.crossplane.io                                   2023-02-24T10:24:27Z
playlists.grafana.jet.crossplane.io                                       2023-02-24T10:24:27Z
providerconfigs.grafana.jet.crossplane.io                                 2023-02-24T10:24:27Z
providerconfigusages.grafana.jet.crossplane.io                            2023-02-24T10:24:27Z
reports.grafana.jet.crossplane.io                                         2023-02-24T10:24:27Z
roles.grafana.jet.crossplane.io                                           2023-02-24T10:24:27Z
syntheticmonitoringchecks.grafana.jet.crossplane.io                       2023-02-24T10:24:27Z
syntheticmonitoringinstallations.grafana.jet.crossplane.io                2023-02-24T10:24:27Z
syntheticmonitoringprobes.grafana.jet.crossplane.io                       2023-02-24T10:24:27Z
teamexternalgroups.grafana.jet.crossplane.io                              2023-02-24T10:24:27Z
teampreferences.grafana.jet.crossplane.io                                 2023-02-24T10:24:27Z
teams.grafana.jet.crossplane.io                                           2023-02-24T10:24:27Z
users.grafana.jet.crossplane.io                                           2023-02-24T10:24:27Z
omerfsen commented 1 year ago

AWS EKS v1.24.8-eks-ffeb93d

Client Version: version.Info{Major:"1", Minor:"26", GitVersion:"v1.26.1", GitCommit:"8f94681cd294aa8cfd3407b8191f6c70214973a4", GitTreeState:"clean", BuildDate:"2023-01-18T15:51:24Z", GoVersion:"go1.19.5", Compiler:"gc", Platform:"linux/amd64"}
Kustomize Version: v4.5.7
Server Version: version.Info{Major:"1", Minor:"24+", GitVersion:"v1.24.8-eks-ffeb93d", GitCommit:"abb98ec0631dfe573ec5eae40dc48fd8f2017424", GitTreeState:"clean", BuildDate:"2022-11-29T18:45:03Z", GoVersion:"go1.18.8", Compiler:"gc", Platform:"linux/amd64"}
omerfsen commented 1 year ago

Also one thing I have noticed, If i do use kubectl apply it does not work, but if I issue kubectl crossplane install provider ... i see provider is INSTALLED and HEALTHY (even with 0.1.0 provider version)


$  cat 1_provider.yml 
apiVersion: pkg.crossplane.io/v1
kind: Provider
metadata:
  name: provider-grafana
spec:
  package: grafana/provider-grafana:v0.1.0

$  k  apply -f 1_provider.yml 
provider.pkg.crossplane.io/provider-grafana created

$  k get providers -A
NAME                   INSTALLED   HEALTHY   PACKAGE                                                         AGE
kubernetes-provider    True        True      xpkg.upbound.io/crossplane-contrib/provider-kubernetes:v0.6.0   12h
provider-grafana                             grafana/provider-grafana:v0.1.0                                 7s
provider-helm          True        True      xpkg.upbound.io/crossplane-contrib/provider-helm:v0.13.0        12h
upbound-aws-provider   True        True      xpkg.upbound.io/upbound/provider-aws:v0.27.0                    12h

$ k delete providers provider-grafana
provider.pkg.crossplane.io "provider-grafana" deleted

$  kubectl crossplane install provider grafana/crossplane-provider-grafana:v0.1.0  provider-grafana
provider.pkg.crossplane.io/provider-grafana created

$ k get providers -A
NAME                   INSTALLED   HEALTHY   PACKAGE                                                         AGE
kubernetes-provider    True        True      xpkg.upbound.io/crossplane-contrib/provider-kubernetes:v0.6.0   12h
provider-grafana       True        True      grafana/crossplane-provider-grafana:v0.1.0                      7s
provider-helm          True        True      xpkg.upbound.io/crossplane-contrib/provider-helm:v0.13.0        12h
upbound-aws-provider   True        True      xpkg.upbound.io/upbound/provider-aws:v0.27.0                    12h
omerfsen commented 1 year ago

Another issue is if we use:

https://raw.githubusercontent.com/grafana/crossplane-provider-grafana/main/examples/cloud-stack-chain/1.provider/providerconfig.yaml

it states:


error: resource mapping not found for name: "grafana-cloud-provider" namespace: "" from "3_providerconfig.yml": no matches for kind "ProviderConfig" in version "grafana.crossplane.io/v1beta1"
ensure CRDs are installed first


cat 1_provider.yml
---- 
apiVersion: pkg.crossplane.io/v1
kind: Provider
metadata:
  name: provider-grafana
spec:
  package: grafana/provider-grafana:v0.1.0

cat 2_secret.yml 
---
apiVersion: v1
kind: Secret
metadata:
  name: grafana-cloud-creds
  namespace: upbound-system
type: Opaque
stringData:
  credentials: |
    {
      "cloud_api_key": "${grafana_token}"
    }

cat  3_providerconfig.yml 
---
apiVersion: grafana.crossplane.io/v1beta1
kind: ProviderConfig
metadata:
  name: grafana-cloud-provider
spec:
  credentials:
    source: Secret
    secretRef:
      name: grafana-cloud-creds
      namespace: upbound-system
      key: credentials

The reason for that seems to be tht ProviderConfig for grafana uses "terrajet" instead of Upbound.

kubectl get crds -A |grep -i ProviderConfig
identityproviderconfigs.eks.aws.upbound.io                                2023-02-23T23:28:45Z
providerconfigs.aws.upbound.io                                            2023-02-23T23:28:37Z.
providerconfigs.grafana.jet.crossplane.io      <!!!!!!!                   2023-02-24T11:32:11Z
providerconfigs.helm.crossplane.io                                        2023-02-23T23:28:21Z
providerconfigs.kubernetes.crossplane.io                                  2023-02-23T23:28:14Z
providerconfigusages.aws.upbound.io                                       2023-02-23T23:28:37Z
providerconfigusages.grafana.jet.crossplane.io                            2023-02-24T11:32:11Z
providerconfigusages.helm.crossplane.io                                   2023-02-23T23:28:21Z
providerconfigusages.kubernetes.crossplane.io                             2023-02-23T23:28:14Z
julienduchesne commented 1 year ago

Strange... We still use v0.1.0 internally. I'll check this out and get back to you

julienduchesne commented 1 year ago

Hey @omerfsen. I just released v0.3.0 and tried it on one of our clusters and all CRDs were installed. Let me know if it's fixed!

omerfsen commented 1 year ago

Will try and return.

omerfsen commented 1 year ago

Still same, But we know why now.. See kubectl describe providers command output, I suspect you have not made these 0.2.0 and 0.3.0 public so others can download without authentication..

Btw why do you use EOL'ed https://github.com/crossplane-contrib/provider-jet-aws .. You must use https://github.com/upbound/provider-aws

In my cluster as you can see:

NAME                   INSTALLED   HEALTHY   PACKAGE                                                         AGE     LABELS
aws-provider           True        True      xpkg.upbound.io/crossplane-contrib/provider-aws:v0.36.0         7d9h    <none>
kubernetes-provider    True        True      xpkg.upbound.io/crossplane-contrib/provider-kubernetes:v0.6.0   7d23h   <none>
provider-grafana                             grafana/crossplane-provider-grafana:v0.3.0                      117s    <none>
provider-helm          True        True      xpkg.upbound.io/crossplane-contrib/provider-helm:v0.13.0        7d23h   <none>
upbound-aws-provider   True        True      xpkg.upbound.io/upbound/provider-aws:v0.27.0                    7d23h   <none>

Here is my commands:

$ kubectl crossplane install provider grafana/crossplane-provider-grafana:v0.1.0  provider-grafana                               
provider.pkg.crossplane.io/provider-grafana created

$ k get crds |grep -i crossplane |grep grafana                      
alertnotifications.grafana.jet.crossplane.io                              2023-03-03T23:28:16Z
apikeys.grafana.jet.crossplane.io                                         2023-03-03T23:28:16Z
builtinroleassignments.grafana.jet.crossplane.io                          2023-03-03T23:28:16Z
cloudapikeys.grafana.jet.crossplane.io                                    2023-03-03T23:28:16Z
cloudplugininstallations.grafana.jet.crossplane.io                        2023-03-03T23:28:16Z
cloudstacks.grafana.jet.crossplane.io                                     2023-03-03T23:28:16Z
dashboardpermissions.grafana.jet.crossplane.io                            2023-03-03T23:28:16Z
dashboards.grafana.jet.crossplane.io                                      2023-03-03T23:28:16Z
datasourcepermissions.grafana.jet.crossplane.io                           2023-03-03T23:28:16Z
datasources.grafana.jet.crossplane.io                                     2023-03-03T23:28:16Z
folderpermissions.grafana.jet.crossplane.io                               2023-03-03T23:28:16Z
folders.grafana.jet.crossplane.io                                         2023-03-03T23:28:16Z
librarypanels.grafana.jet.crossplane.io                                   2023-03-03T23:28:16Z
machinelearningjobs.grafana.jet.crossplane.io                             2023-03-03T23:28:16Z
organizations.grafana.jet.crossplane.io                                   2023-03-03T23:28:16Z
playlists.grafana.jet.crossplane.io                                       2023-03-03T23:28:16Z
providerconfigs.grafana.jet.crossplane.io                                 2023-03-03T23:28:16Z
providerconfigusages.grafana.jet.crossplane.io                            2023-03-03T23:28:16Z
reports.grafana.jet.crossplane.io                                         2023-03-03T23:28:16Z
roles.grafana.jet.crossplane.io                                           2023-03-03T23:28:16Z
syntheticmonitoringchecks.grafana.jet.crossplane.io                       2023-03-03T23:28:16Z
syntheticmonitoringinstallations.grafana.jet.crossplane.io                2023-03-03T23:28:16Z
syntheticmonitoringprobes.grafana.jet.crossplane.io                       2023-03-03T23:28:16Z
teamexternalgroups.grafana.jet.crossplane.io                              2023-03-03T23:28:16Z
teampreferences.grafana.jet.crossplane.io                                 2023-03-03T23:28:16Z
teams.grafana.jet.crossplane.io                                           2023-03-03T23:28:16Z
users.grafana.jet.crossplane.io                                           2023-03-03T23:28:16Z

$ k delete providers  provider-grafana 
provider.pkg.crossplane.io "provider-grafana" deleted

$  kubectl crossplane install provider grafana/crossplane-provider-grafana:v0.3.0  provider-grafana      
provider.pkg.crossplane.io/provider-grafana created

$ k get crds |grep -i crossplane |grep grafana                         
<NO RESULT>

$ k get providers -A
NAME                   INSTALLED   HEALTHY   PACKAGE                                                         AGE
aws-provider           True        True      xpkg.upbound.io/crossplane-contrib/provider-aws:v0.36.0         7d9h
kubernetes-provider    True        True      xpkg.upbound.io/crossplane-contrib/provider-kubernetes:v0.6.0   8d
provider-grafana                             grafana/crossplane-provider-grafana:v0.3.0                      2m48s. <!!!!!!!!!!!!!!!!!!!!!!
provider-helm          True        True      xpkg.upbound.io/crossplane-contrib/provider-helm:v0.13.0        8d
upbound-aws-provider   True        True      xpkg.upbound.io/upbound/provider-aws:v0.27.0                    8d

$ k describe providers provider-grafana                                                                                          in bash at 23:32:36
Name:         provider-grafana
Namespace:    
Labels:       <none>
Annotations:  <none>
API Version:  pkg.crossplane.io/v1
Kind:         Provider
Metadata:
  Creation Timestamp:  2023-03-03T23:28:58Z
  Generation:          1
  Managed Fields:
    API Version:  pkg.crossplane.io/v1
    Fields Type:  FieldsV1
    fieldsV1:
      f:spec:
        .:
        f:ignoreCrossplaneConstraints:
        f:package:
        f:packagePullPolicy:
        f:revisionActivationPolicy:
        f:revisionHistoryLimit:
        f:skipDependencyResolution:
    Manager:         kubectl-crossplane
    Operation:       Update
    Time:            2023-03-03T23:28:58Z
  Resource Version:  8317327
  UID:               294a29c7-7010-4c3d-a137-f3fbab7542b3
Spec:
  Ignore Crossplane Constraints:  false
  Package:                        grafana/crossplane-provider-grafana:v0.3.0
  Package Pull Policy:            IfNotPresent
  Revision Activation Policy:     Automatic
  Revision History Limit:         0
  Skip Dependency Resolution:     false
Events:
  Type     Reason         Age                  From                                 Message
  ----     ------         ----                 ----                                 -------
  Warning  UnpackPackage  39s (x9 over 3m43s)  packages/provider.pkg.crossplane.io  cannot unpack package: failed to fetch package digest from remote: failed to fetch package descriptor with a GET request after a previous HEAD request failure: HEAD https://index.docker.io/v2/grafana/crossplane-provider-grafana/manifests/v0.3.0: unexpected status code 404 Not Found (HEAD responses have no body, use GET for details): GET https://index.docker.io/v2/grafana/crossplane-provider-grafana/manifests/v0.3.0: MANIFEST_UNKNOWN: manifest unknown; unknown tag=v0.3.0
julienduchesne commented 1 year ago

grafana/crossplane-provider-grafana:v0.3.0

This is wrong. It should be xpkg.upbound.io/grafana/provider-grafana:v0.3.0 as shown in the install instructions here: https://marketplace.upbound.io/providers/grafana/provider-grafana/v0.3.0

omerfsen commented 1 year ago

If you look at

https://hub.docker.com/r/grafana/crossplane-provider-grafana-controller/tags

there is none for 0.2.0 or 0.3.0 but only for 0.1.0

julienduchesne commented 1 year ago

Sorry for the confusion, these are old repositories from when we were just trying out Crossplane. Everything is now on the upbound marketplace. I've taken these Dockerhub repositories down

omerfsen commented 1 year ago

Ok i think I have found out the reason:

kubectl crossplane install provider grafana/crossplane-provider-grafana:v0.3.0  provider-grafana

vs 

up ctp provider install grafana/provider-grafana:v0.3.0

kubectl command tries to download docker.io image but up commands does it successfully.

Since I am using terraform I will try yml installation method now and return to you:

---
apiVersion: pkg.crossplane.io/v1
kind: Provider
metadata:
  name: provider-grafana
spec:
  package: grafana/provider-grafana:v0.3.0
---
apiVersion: v1
kind: Secret
metadata:
  name: grafana-cloud-creds
  namespace: upbound-system
type: Opaque
stringData:
  credentials: |
    {
      "cloud_api_key": "${grafana_token}"
    }
---
apiVersion: grafana.crossplane.io/v1beta1
kind: ProviderConfig
metadata:
  name: grafana-cloud-provider
spec:
  credentials:
    source: Secret
    secretRef:
      name: grafana-cloud-creds
      namespace: upbound-system
      key: credentials

After up command we have 0.3.0 Installed & Healthy now.. (which is good) and CRDs are there..

 k get providers -A
NAME                       INSTALLED   HEALTHY   PACKAGE                                                         AGE
aws-provider               True        True      xpkg.upbound.io/crossplane-contrib/provider-aws:v0.36.0         7d9h
grafana-provider-grafana   True        True      xpkg.upbound.io/grafana/provider-grafana:v0.3.0                 7m19s
kubernetes-provider        True        True      xpkg.upbound.io/crossplane-contrib/provider-kubernetes:v0.6.0   8d
provider-helm              True        True      xpkg.upbound.io/crossplane-contrib/provider-helm:v0.13.0        8d
upbound-aws-provider       True        True      xpkg.upbound.io/upbound/provider-aws:v0.27.0                    8d

$ k get crds |grep -i crossplane |grep grafana                                             in bash at 23:57:29                    in bash at 23:57:29
apikeys.cloud.grafana.crossplane.io                                       2023-03-03T23:50:11Z
apikeys.oss.grafana.crossplane.io                                         2023-03-03T23:50:11Z
checks.sm.grafana.crossplane.io                                           2023-03-03T23:50:11Z
contactpoints.alerting.grafana.crossplane.io                              2023-03-03T23:50:11Z
dashboardpermissions.oss.grafana.crossplane.io                            2023-03-03T23:50:11Z
dashboards.oss.grafana.crossplane.io                                      2023-03-03T23:50:11Z
datasources.oss.grafana.crossplane.io                                     2023-03-03T23:50:11Z
escalationchains.oncall.grafana.crossplane.io                             2023-03-03T23:50:11Z
escalations.oncall.grafana.crossplane.io                                  2023-03-03T23:50:11Z
folderpermissions.oss.grafana.crossplane.io                               2023-03-03T23:50:11Z
folders.oss.grafana.crossplane.io                                         2023-03-03T23:50:11Z
installations.sm.grafana.crossplane.io                                    2023-03-03T23:50:11Z
integrations.oncall.grafana.crossplane.io                                 2023-03-03T23:50:11Z
messagetemplates.alerting.grafana.crossplane.io                           2023-03-03T23:50:11Z
mutetimings.alerting.grafana.crossplane.io                                2023-03-03T23:50:11Z
notificationpolicies.alerting.grafana.crossplane.io                       2023-03-03T23:50:11Z
oncallshifts.oncall.grafana.crossplane.io                                 2023-03-03T23:50:11Z
organizations.oss.grafana.crossplane.io                                   2023-03-03T23:50:11Z
outgoingwebhooks.oncall.grafana.crossplane.io                             2023-03-03T23:50:11Z
probes.sm.grafana.crossplane.io                                           2023-03-03T23:50:11Z
providerconfigs.grafana.crossplane.io                                     2023-03-03T23:50:11Z
providerconfigusages.grafana.crossplane.io                                2023-03-03T23:50:11Z
reports.enterprise.grafana.crossplane.io                                  2023-03-03T23:50:11Z
routes.oncall.grafana.crossplane.io                                       2023-03-03T23:50:11Z
rulegroups.alerting.grafana.crossplane.io                                 2023-03-03T23:50:11Z
schedules.oncall.grafana.crossplane.io                                    2023-03-03T23:50:11Z
stacks.cloud.grafana.crossplane.io                                        2023-03-03T23:50:11Z
storeconfigs.grafana.crossplane.io                                        2023-03-03T23:50:11Z
teams.oss.grafana.crossplane.io                                           2023-03-03T23:50:11Z
users.oss.grafana.crossplane.io                                           2023-03-03T23:50:11Z
julienduchesne commented 1 year ago

kubectl crossplane install provider xpkg.upbound.io/grafana/crossplane-provider-grafana:v0.3.0 provider-grafana would probably work as well

And the YAML install should look like:

apiVersion: pkg.crossplane.io/v1
kind: Provider
metadata:
  name: provider-grafana
spec:
  package: xpkg.upbound.io/grafana/provider-grafana:v0.3.0

All the examples in the examples/ dir should be functional

omerfsen commented 1 year ago

Kubectl cli must be :

kubectl crossplane install provider xpkg.upbound.io/grafana/provider-grafana:v0.3.0 provider-grafana


$  up ctp provider install grafana/provider-grafana:v0.3.0 --name provider-grafana
provider-grafana installed

$ k get providers -A
NAME                   INSTALLED   HEALTHY   PACKAGE                                                         AGE
aws-provider           True        True      xpkg.upbound.io/crossplane-contrib/provider-aws:v0.36.0         7d10h
kubernetes-provider    True        True      xpkg.upbound.io/crossplane-contrib/provider-kubernetes:v0.6.0   8d
provider-grafana       True        True      xpkg.upbound.io/grafana/provider-grafana:v0.3.0                 5s
provider-helm          True        True      xpkg.upbound.io/crossplane-contrib/provider-helm:v0.13.0        8d
upbound-aws-provider   True        True      xpkg.upbound.io/upbound/provider-aws:v0.27.0                    8d

$ k delete providers provider-grafana
provider.pkg.crossplane.io "provider-grafana" deleted

$ kubectl crossplane install provider xpkg.upbound.io/grafana/provider-grafana:v0.3.0  provider-grafana
provider.pkg.crossplane.io/provider-grafana created

$ k get providers -A                      
NAME                   INSTALLED   HEALTHY   PACKAGE                                                         AGE
aws-provider           True        True      xpkg.upbound.io/crossplane-contrib/provider-aws:v0.36.0         7d10h
kubernetes-provider    True        True      xpkg.upbound.io/crossplane-contrib/provider-kubernetes:v0.6.0   8d
provider-grafana       True        True      xpkg.upbound.io/grafana/provider-grafana:v0.3.0                 3s
provider-helm          True        True      xpkg.upbound.io/crossplane-contrib/provider-helm:v0.13.0        8d
upbound-aws-provider   True        True      xpkg.upbound.io/upbound/provider-aws:v0.27.0                    8d

and thank you

apiVersion: pkg.crossplane.io/v1
kind: Provider
metadata:
  name: provider-grafana
spec:
  package: xpkg.upbound.io/grafana/provider-grafana:v0.3.0

worked ..

omerfsen commented 1 year ago

As it is all cleared out closing this issue. Thank you @julienduchesne