unable to override ingress rules for prometheus-operator

[smartpcr]

Describe the bug: I am trying to create prometheus-operator with customized ingress rule so that grafana/prometheus UI is exposed to internet. Have to add common.yaml under config in order for it to work. However, I cannot override values defined in common.yaml for different environment when I run fab command.

To Reproduce:

In repo fabrikate-definitions, add the following folder:

• fabrikate-prometheus-operator
  • config
  • common.yaml
  • manifests
  • prometheus-operator-namespace.yaml
  • component.yaml

yaml files

• component.yaml

  name: "fabrikate-prometheus-operator"
  type: static
  path: "./manifests"
  hooks:
  before-install:
  - echo 'installing prometheus-operator'
  after-install:
  - echo 'installed prometheus-operator'
  subcomponents:
  - name: "prometheus-operator"
  type: helm
  source: "https://github.com/helm/charts"
  method: "git"
  path: "stable/prometheus-operator"
  version: "f4743a18769c5405413e1d6cd897bd9c877ce1b1"

• common.yaml

  config:
  subcomponents:
  prometheus-operator:
  namespace: monitoring
  injectNamespace: true
  config:
    coreDns:
      enabled: true
    kubeDns:
      enabled: false
    alertmanager:
      alertmanagerSpec:
        logLevel: "debug"
        storage:
          volumeClaimTemplate:
            spec:
              accessModes: ["ReadWriteOnce"]
              resources:
                requests:
                  storage: 10Gi
      ingress:
        enabled: true
        annotations:
          kubernetes.io/ingress.class: nginx
          kubernetes.io/tls-acme: "true"
        hosts:
          - "alertmanager-space-westus2.mydomain"
        tls:
          - secretName: "mysslcert"
            hosts:
              - "alertmanager-space-westus2.mydomain"
    prometheus:
      prometheusSpec:
        storageSpec:
          volumeClaimTemplate:
            spec:
              accessModes: ["ReadWriteOnce"]
              resources:
                requests:
                  storage: 10Gi
      ingress:
        enabled: true
        annotations:
          kubernetes.io/ingress.class: nginx
          kubernetes.io/tls-acme: "true"
        hosts:
          - "prometheus-space-westus2.mydomain"
        tls:
          - secretName: "mysslcert"
            hosts:
              - "prometheus-space-westus2.mydomain"
    grafana:
      adminUser: "admin"
      adminPassword: "grafana"
      persistence:
        enabled: true
        accessModes: ["ReadWriteOnce"]
        size: 10Gi
      ingress:
        enabled: true
        annotations:
          kubernetes.io/ingress.class: nginx
          kubernetes.io/tls-acme: "true"
        hosts:
          - "grafana-space-westus2.mydomain"
        tls:
          - secretName: "mysslcert"
            hosts:
              - "grafana-space-westus2.mydomain"
  
    kubelet:
      serviceMonitor:
        https: false

  Within HLD repo, I referenced fabrikate-prometheus-operator git repo and path using method git. under config folder, I have the following two files:

• common.yaml

  config:
  subcomponents:
  prometheus-operator:
  namespace: monitoring
  injectNamespace: true
  config:
    coreDns:
      enabled: true
    kubeDns:
      enabled: false
    alertmanager:
      ingress:
        enabled: "true"
    grafana:
      adminUser: "admin"
      adminPassword: "grafana"
      ingress:
        enabled: "true"
    prometheus:
      ingress:
        enabled: "true"

• dev.yaml

  config:
  subcomponents:
  prometheus-operator:
  config:
    alertmanager:
      ingress:
        enabled: true
        annotations:
          kubernetes.io/ingress.class: nginx
          kubernetes.io/tls-acme: "true"
        hosts:
          - "alertmanager-dev.sace.works"
        tls:
          - secretName: "sslcert-sace-works"
            hosts:
              - "alertmanager-dev.sace.works"
    prometheus:
      ingress:
        enabled: true
        annotations:
          kubernetes.io/ingress.class: nginx
          kubernetes.io/tls-acme: "true"
        hosts:
          - "prometheus-dev.sace.works"
        tls:
          - secretName: "sslcert-sace-works"
            hosts:
              - "prometheus-dev.sace.works"
    grafana:
      ingress:
        enabled: true
        annotations:
          kubernetes.io/ingress.class: nginx
          kubernetes.io/tls-acme: "true"
        hosts:
          - "grafana-dev.sace.works"
        tls:
          - secretName: "sslcert-sace-works"
            hosts:
              - "grafana-dev.sace.works"

When I run the following commands:

fab install
fab generate dev

I got the following ingress settings in generated file: prometheus-operator.yaml:

---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  annotations:
    kubernetes.io/ingress.class: nginx
    kubernetes.io/tls-acme: "true"
  labels:
    app: prometheus-operator-prometheus
    chart: prometheus-operator-6.4.1
    heritage: Tiller
    release: prometheus-operator
  name: prometheus-operator-prometheus
  namespace: monitoring
spec:
  rules:
  - host: prometheus-space-westus2.mydomain
    http:
      paths:
      - backend:
          serviceName: prometheus-operator-prometheus
          servicePort: 9090
        path: /
  tls:
  - hosts:
    - prometheus-space-westus2.mydomain
    secretName: mysslcert
---

Expected behavior:

I expect values within file dev.yaml under config folder would be applied and generate the following ingress settings (note the difference: sace.works instead of mydomain)

---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  annotations:
    kubernetes.io/ingress.class: nginx
    kubernetes.io/tls-acme: "true"
  labels:
    app: prometheus-operator-prometheus
    chart: prometheus-operator-6.4.1
    heritage: Tiller
    release: prometheus-operator
  name: prometheus-operator-prometheus
  namespace: monitoring
spec:
  rules:
  - host: prometheus-dev.sace.works
    http:
      paths:
      - backend:
          serviceName: prometheus-operator-prometheus
          servicePort: 9090
        path: /
  tls:
  - hosts:
    - prometheus-dev.sace.works
    secretName: sslcert-sace-works

Screenshots:

Additional context:

Desktop (please complete the following information):

• OS:

Additional context

[edaena]

@smartpcr where are you creating/adding the second common.yaml and dev.yaml files?

[smartpcr]

I have two separate repos:

one on github that mimics fabrikate-definitions and include prometheus-operator.

the other repo is in AzureDevOps that have component.yaml referencing github repo, and this repo contains config files: common.yaml and dev.yaml

I run fab command within second repo.

[edaena]

@smartpcr what does your component.yaml look like? I suspect you might be missing another config layer. I created this repo as a sample to mimic your setup edaena/tmp-definitions prometheus-operator is what would be in the fabrikate-definitions repo, the test-prom-op-config is what would be in your other repo. The dev.yaml config looks like:

config:
subcomponents:
  fabrikate-prometheus-operator:
    subcomponents:
      prometheus-operator:
        config:
          alertmanager:
            ingress:
              enabled: true
              annotations:
                kubernetes.io/ingress.class: nginx
                kubernetes.io/tls-acme: "true"
              hosts:
                - "alertmanager-dev.sace.works"
              tls:
                - secretName: "sslcert-sace-works"
                  hosts:
                    - "alertmanager-dev.sace.works"
          prometheus:
            ingress:
              enabled: true
              annotations:
                kubernetes.io/ingress.class: nginx
                kubernetes.io/tls-acme: "true"
              hosts:
                - "prometheus-dev.sace.works"
              tls:
                - secretName: "sslcert-sace-works"
                  hosts:
                    - "prometheus-dev.sace.works"
          grafana:
            ingress:
              enabled: true
              annotations:
                kubernetes.io/ingress.class: nginx
                kubernetes.io/tls-acme: "true"
              hosts:
                - "grafana-dev.sace.works"
              tls:
                - secretName: "sslcert-sace-works"
                  hosts:
                    - "grafana-dev.sace.works"

As you can see, the first subcomponent is fabrikate-prometheus-operator which is what's specified in the component.yaml:

name: "test-prom-op-config"
subcomponents:
  - name: "fabrikate-prometheus-operator"
    type: component
    source: "git@github.com:edaena/tmp-fabrikate-definitions.git"
    path: "prometheus-operator"
    method: git

Not sure if your component.yaml looks similar to this.

[smartpcr]

just got a chance to look at it. Thanks for taking the effort trying to reproduce the problem. You found the cause of my problem. :)