timescale / tobs

tobs - The Observability Stack for Kubernetes. Easy install of a full observability stack into a k8s cluster with Helm charts.
Apache License 2.0
563 stars 60 forks source link

custom-secret-scripts secret not being created when timescaledb-single is disabled. #536

Closed siva-kubikk closed 2 years ago

siva-kubikk commented 2 years ago

What happened?

Trying to install tobs with helm and the below values.yaml. I'm using timescale cloud service as the database.

timescaledb-single:
  # disable the chart if an existing TimescaleDB instance is used
  enabled: false

promscale:
  enabled: true
  connection:
    uri: "postgres://tsdbadmin:xxxxxxxxxx@xxxxxx.xxxxxxx.tsdb.cloud.timescale.com:99999/tsdb?sslmode=require"

The tobs-grafana pod keeps restarting with this error:

Events:
  Type     Reason     Age                From               Message
  ----     ------     ----               ----               -------
  Normal   Scheduled  40s                default-scheduler  Successfully assigned tobs/tobs-grafana-7486b49884-5sxlb to ip-10-10-4-56.ec2.internal
  Normal   Pulled     39s                kubelet            Container image "busybox:1.31.1" already present on machine
  Normal   Created    39s                kubelet            Created container init-chown-data
  Normal   Started    38s                kubelet            Started container init-chown-data
  Normal   Pulled     38s                kubelet            Container image "quay.io/kiwigrid/k8s-sidecar:1.19.2" already present on machine
  Normal   Created    38s                kubelet            Created container grafana-sc-dashboard
  Normal   Pulled     38s                kubelet            Container image "quay.io/kiwigrid/k8s-sidecar:1.19.2" already present on machine
  Normal   Started    38s                kubelet            Started container grafana-sc-dashboard
  Normal   Created    37s                kubelet            Created container grafana-sc-datasources
  Normal   Started    37s                kubelet            Started container grafana-sc-datasources
  Warning  Failed     13s (x5 over 37s)  kubelet            Error: secret "custom-secret-scripts" not found
  Normal   Pulled     1s (x6 over 37s)   kubelet            Container image "grafana/grafana:9.0.6" already present on machine

Did you expect to see something different?

How to reproduce it (as minimally and precisely as possible):

helm install --wait --version '12.0.0' --values  values.yaml tobs  timescale/tobs

Environment

Client Version: version.Info{Major:"1", Minor:"22", GitVersion:"v1.22.4", GitCommit:"b695d79d4f967c403a96986f1750a35eb75e75f1", GitTreeState:"clean", BuildDate:"2021-11-17T15:41:42Z", GoVersion:"go1.16.10", Compiler:"gc", Platform:"darwin/arm64"}
Server Version: version.Info{Major:"1", Minor:"22+", GitVersion:"v1.22.11-eks-18ef993", GitCommit:"b9628d6d3867ffd84c704af0befd31c7451cdc37", GitTreeState:"clean", BuildDate:"2022-07-06T18:06:23Z", GoVersion:"go1.16.15", Compiler:"gc", Platform:"linux/amd64"}```

* Kubernetes cluster kind:

Deployed EKS cluster with Kubikk.

* tobs Logs:

insert tobs logs relevant to the issue here



**Anything else we need to know?**:
nhudson commented 2 years ago

This is a known issue in tobs. It's being addressed with this PR #531

nhudson commented 2 years ago

You did point out a potential issue I'll need to fix as well. I believe we don't currently look at the uri value.

nhudson commented 2 years ago

So after some discussion with the team, this is by design. The custom-secret-scrips is meant only to be picked up by TimescaleDB that is deployed using tobs. If you are connecting to an external TimescaleDB and providing a URI to connect, we still support that, but you will have to setup the grafana role manually.

We do plan to provide better documentation in the tobs repo, but for now this doc should guide you on what you need to do to get that setup. https://docs.timescale.com/promscale/latest/roles-and-permissions/#example-permissions.

I am going to close this issue, if you feel like it should be reopened please do so.