Error when deploying helm chart on EKS

[mariuskimmina]

Bug description

Hey, we are trying to test out signoz on EKS, following your guide. We are using Terraform for all of our infra we tried to use the helm chart as follows:

resource "kubernetes_namespace" "signoz" {
  metadata {
    name = var.namespace
    labels = {
      app = "signoz"
    }
  }
}

resource "helm_release" "signoz" {
  name       = "signoz"
  namespace  = var.namespace
  repository = "https://charts.signoz.io"
  chart      = "signoz"
  depends_on = [kubernetes_namespace.signoz]

  set {
    name  = "global.storageClass"
    value = "gp2-resizable"
  }

  set {
    name  = "clickhouse.cloud"
    value = "aws"
  }

  set {
    name  = "clickhouse.installCustomStorageClass"
    value = "true"
  }

  set {
    name  = "frontend.image.tag"
    value = "0.15.0"
  }

  set {
    name  = "queryService.image.tag"
    value = "0.15.0"
  }
}

Calling the module

module "signoz" {
  source                          = "../../modules/signoz"
  environment                     = var.environment
}

But during the plan stage of terraform we encounter the following error:

Error: unable to build kubernetes objects from release manifest: resource mapping not found for name: "signoz-clickhouse" namespace: "signoz" from "": no matches for kind "ClickHouseInstallation" in version "clickhouse.altinity.com/v1"
ensure CRDs are installed first
  with module.signoz.helm_release.signoz,
  on ../../modules/signoz/main.tf line 10, in resource "helm_release" "signoz":
  10: resource "helm_release" "signoz" {

Version information

• Signoz version: 0.15.0

[pranay01]

@mariuskimmina Thanks for opening the issue. Are you doing a fresh install of SigNoz or you are trying to upgrade from an existing version?

[mariuskimmina]

It's a fresh install

[mariuskimmina]

Also forgot to mention: we are running on Kubernetes version 1.24

[mariuskimmina]

Found an issue on SO that suggests that the issue is probably the k8s version: https://stackoverflow.com/questions/74781273/helm-install-fails-on-k3s-ensure-crds-are-installed-first

Has the chart been successfully tested on 1.24 before?

[mariuskimmina]

Now this is weird, when I used the helm chart directly, removing terraform form the equation, it worked. When I then removed the helm chart and applied Terraform again, it also works. When I apply Terraform again on another test environment we have where I didn't manually install with helm first - it still fails.

[prashant-shahi]

Has the chart been successfully tested on 1.24 before?

Yes, tested in 1.24 EKS last week. We are tracking it here: https://github.com/SigNoz/signoz/issues/2197

When I then removed the helm chart and applied Terraform again, it also works

@mariuskimmina I see. It is most likely due to CRDs in Helm not being installed by Terraform. But when you use helm directly once, CRDs are installed. Now, you can apply terraform without any issue.

[mariuskimmina]

Seems to be, but normally that should not be the case. Terraform generally handles installing the CRDSs, if you look at https://registry.terraform.io/providers/hashicorp/helm/latest/docs/resources/release there is an option skip_crds that defaults to false. I did not set this to true either. Without this option being set Terraform normally installs all CRDs just fine.