COPRS / rs-issues

This repository contains all the issues of the COPRS project (Scrum tickets, ivv bugs, epics ...)
2 stars 2 forks source link

[BUG] [OPS] Failed to import Grafana dashboard with ansible #608

Closed suberti-ads closed 1 year ago

suberti-ads commented 2 years ago

Environment:

Current Behavior: Following procedure: https://github.com/COPRS/infrastructure/blob/main/docs/user_manuals/how-to/Grafana%20operator%20workflow.md json file has been exported: 1-cluster-overview.json.tar.gz

Following sample provided here: https://github.com/COPRS/infrastructure/blob/main/apps/autoscaling/grafana-dashboards.yaml

We should write grafanadashbord definition like this:

apiVersion: integreatly.org/v1alpha1
kind: GrafanaDashboard
metadata: 
  name: rs-infra-scaler
  labels:
    app.kubernetes.io/component: rs-infra-scaler
spec:
  customFolderName: autoscaling
  datasources:
    - inputName: "DS_PROMETHEUS"
      datasourceName: "Thanos"
    - inputName: "DS_LOKI"
      datasourceName: "Loki"
  url: https://github.com/COPRS/infrastructure/raw/release/0.10.0/scaler/dashboards/rs-infra-scaler-dashboard.json

So we apply following configuration with exported fole

suberti@refsys-client:~/Documents/Git/rs-config/apps/dashboardoperation/json$ cat ../1-cluster-overview.yaml 
apiVersion: integreatly.org/v1alpha1
kind: GrafanaDashboard
metadata: 
  name: cluster-overview
  labels:
    app.kubernetes.io/component: cluster-overview
spec:
  customFolderName: "6. Analysis"
  datasources:
    - inputName: "DS_PROMETHEUS"
      datasourceName: "Thanos"
  url: https://<REPO>/apps/dashboardoperation/json/1-cluster-overview.json

Hereafter kustomize configuration used

suberti@refsys-client:~/Documents/Git/rs-config/apps/dashboardoperation$ cat kustomization.yaml 
commonLabels:
  app.kubernetes.io/instance: "{{ app_name }}"

namespace: monitoring

resources:
  - 1-cluster-overview.yaml

update failed:

suberti@refsys-client:~/Documents/Git/rs-config/infrastructure$ ansible-playbook apps.yaml -i inventory/sample/hosts.yaml -e app=dashboardoperation

PLAY [Deploy reference system] ****************************************************************************************************************************************************************************

TASK [app-installer : Browse each package] ****************************************************************************************************************************************************************
included: /home/suberti/Documents/Git/rs-config/infrastructure/roles/app-installer/tasks/install_package.yaml for gw2-cluster-ops => (item=../apps)

TASK [app-installer : apps | Collect apps in package] *****************************************************************************************************************************************************
ok: [gw2-cluster-ops]

TASK [app-installer : apps | Browse each app] *************************************************************************************************************************************************************
skipping: [gw2-cluster-ops] => (item=../apps/00-apisix-crds) 
skipping: [gw2-cluster-ops] => (item=../apps/00-eck-operator-crds) 
skipping: [gw2-cluster-ops] => (item=../apps/00-grafana-operator-crds) 
skipping: [gw2-cluster-ops] => (item=../apps/00-kafka-operator-crds) 
skipping: [gw2-cluster-ops] => (item=../apps/00-kube-prometheus-stack-crds) 
skipping: [gw2-cluster-ops] => (item=../apps/00-namespaces) 
skipping: [gw2-cluster-ops] => (item=../apps/00-stash-crds) 
skipping: [gw2-cluster-ops] => (item=../apps/10-cert-manager) 
skipping: [gw2-cluster-ops] => (item=../apps/20-linkerd) 
skipping: [gw2-cluster-ops] => (item=../apps/30-linkerd-viz) 
skipping: [gw2-cluster-ops] => (item=../apps/40-rook-ceph) 
skipping: [gw2-cluster-ops] => (item=../apps/50-rook-ceph-cluster) 
skipping: [gw2-cluster-ops] => (item=../apps/60-eck-operator) 
skipping: [gw2-cluster-ops] => (item=../apps/60-grafana-operator) 
skipping: [gw2-cluster-ops] => (item=../apps/60-kafka-operator) 
skipping: [gw2-cluster-ops] => (item=../apps/60-keda) 
skipping: [gw2-cluster-ops] => (item=../apps/60-prometheus-operator) 
skipping: [gw2-cluster-ops] => (item=../apps/60-stash-operator) 
skipping: [gw2-cluster-ops] => (item=../apps/70-elasticsearch-processing) 
skipping: [gw2-cluster-ops] => (item=../apps/70-elasticsearch-security) 
skipping: [gw2-cluster-ops] => (item=../apps/70-fluent-bit) 
skipping: [gw2-cluster-ops] => (item=../apps/70-kafka) 
skipping: [gw2-cluster-ops] => (item=../apps/70-mongodb) 
skipping: [gw2-cluster-ops] => (item=../apps/70-openldap) 
skipping: [gw2-cluster-ops] => (item=../apps/70-postgresql) 
skipping: [gw2-cluster-ops] => (item=../apps/70-thanos) 
skipping: [gw2-cluster-ops] => (item=../apps/71-grafanadatasources) 
skipping: [gw2-cluster-ops] => (item=../apps/80-fluentd) 
skipping: [gw2-cluster-ops] => (item=../apps/80-prometheus-stack) 
skipping: [gw2-cluster-ops] => (item=../apps/90-loki) 
skipping: [gw2-cluster-ops] => (item=../apps/apisix) 
skipping: [gw2-cluster-ops] => (item=../apps/f1-applications) 
skipping: [gw2-cluster-ops] => (item=../apps/falco) 
skipping: [gw2-cluster-ops] => (item=../apps/falco-exporter) 
skipping: [gw2-cluster-ops] => (item=../apps/finops-object-storage-exporter) 
skipping: [gw2-cluster-ops] => (item=../apps/finops-resources-exporter) 
skipping: [gw2-cluster-ops] => (item=../apps/grafana) 
skipping: [gw2-cluster-ops] => (item=../apps/graylog) 
skipping: [gw2-cluster-ops] => (item=../apps/keycloak) 
skipping: [gw2-cluster-ops] => (item=../apps/kubectl-oidc) 
skipping: [gw2-cluster-ops] => (item=../apps/nmap) 
skipping: [gw2-cluster-ops] => (item=../apps/processing-apisix) 
skipping: [gw2-cluster-ops] => (item=../apps/rs-core) 
skipping: [gw2-cluster-ops] => (item=../apps/sdp-apisix) 
skipping: [gw2-cluster-ops] => (item=../apps/sdp-security-policies) 
skipping: [gw2-cluster-ops] => (item=../apps/spring-cloud-dataflow) 
included: /home/suberti/Documents/Git/rs-config/infrastructure/roles/app-installer/tasks/install_app.yaml for gw2-cluster-ops => (item=../apps/dashboardoperation)

TASK [app-installer : apps - dashboardoperation | Check whether kustomization.yaml file exists] ***********************************************************************************************************
ok: [gw2-cluster-ops]

TASK [app-installer : apps - dashboardoperation | Fail if no kustomization file found] ********************************************************************************************************************
skipping: [gw2-cluster-ops]

TASK [app-installer : apps - dashboardoperation | Include vars from kustomization.yaml] *******************************************************************************************************************
ok: [gw2-cluster-ops]

TASK [app-installer : apps - dashboardoperation | Check whether .helm_repository_config.yaml exists] ******************************************************************************************************
ok: [gw2-cluster-ops]

TASK [app-installer : apps - dashboardoperation | Include vars from .helm_repository_config.yaml] *********************************************************************************************************
skipping: [gw2-cluster-ops]

TASK [app-installer : apps - dashboardoperation | Create tmp directory for k8s resources] *****************************************************************************************************************
changed: [gw2-cluster-ops]

TASK [app-installer : apps - dashboardoperation | Template and send resources to remote] ******************************************************************************************************************
skipping: [gw2-cluster-ops] => (item={'root': '/home/suberti/Documents/Git/rs-config/apps/dashboardoperation/', 'path': 'json', 'state': 'directory', 'uid': 1002, 'gid': 1002, 'owner': 'suberti', 'group': 'suberti', 'mode': '0775', 'size': 4096, 'mtime': 1665062761.5356574, 'ctime': 1665062761.5356574}) 
changed: [gw2-cluster-ops] => (item={'root': '/home/suberti/Documents/Git/rs-config/apps/dashboardoperation/', 'path': '1-cluster-overview.yaml', 'state': 'file', 'src': '/home/suberti/Documents/Git/rs-config/apps/dashboardoperation/1-cluster-overview.yaml', 'uid': 1002, 'gid': 1002, 'owner': 'suberti', 'group': 'suberti', 'mode': '0664', 'size': 379, 'mtime': 1665062751.023695, 'ctime': 1665062751.039695})
changed: [gw2-cluster-ops] => (item={'root': '/home/suberti/Documents/Git/rs-config/apps/dashboardoperation/', 'path': 'kustomization.yaml', 'state': 'file', 'src': '/home/suberti/Documents/Git/rs-config/apps/dashboardoperation/kustomization.yaml', 'uid': 1002, 'gid': 1002, 'owner': 'suberti', 'group': 'suberti', 'mode': '0664', 'size': 124, 'mtime': 1665063095.2187467, 'ctime': 1665063095.2387466})
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: }
failed: [gw2-cluster-ops] (item={'root': '/home/suberti/Documents/Git/rs-config/apps/dashboardoperation/', 'path': 'json/1-cluster-overview.json', 'state': 'file', 'src': '/home/suberti/Documents/Git/rs-config/apps/dashboardoperation/json/1-cluster-overview.json', 'uid': 1002, 'gid': 1002, 'owner': 'suberti', 'group': 'suberti', 'mode': '0664', 'size': 109560, 'mtime': 1665050318.179636, 'ctime': 1665050365.395601}) => {"ansible_loop_var": "item", "changed": false, "item": {"ctime": 1665050365.395601, "gid": 1002, "group": "suberti", "mode": "0664", "mtime": 1665050318.179636, "owner": "suberti", "path": "json/1-cluster-overview.json", "root": "/home/suberti/Documents/Git/rs-config/apps/dashboardoperation/", "size": 109560, "src": "/home/suberti/Documents/Git/rs-config/apps/dashboardoperation/json/1-cluster-overview.json", "state": "file", "uid": 1002}, "msg": "AnsibleError: template error while templating string: expected token 'end of print statement', got '{'. String: {\n  \"__inputs\": [\n    {\n      \"name\": \"DS_THANOS\",\n      \"label\": \"Thanos\",\n      \"description\": \"\",\n      \"type\": \"datasource\[...]
TASK [app-installer : apps - dashboardoperation | Remove resources tmp dir] *******************************************************************************************************************************
changed: [gw2-cluster-ops]

TASK [app-installer : apps - dashboardoperation | Warn about not deleted tmp folder] **********************************************************************************************************************
skipping: [gw2-cluster-ops]

PLAY RECAP ************************************************************************************************************************************************************************************************
gw2-cluster-ops            : ok=8    changed=2    unreachable=0    failed=1    skipped=3    rescued=0    ignored=0   

Hereafter verbose error:

[..]
The full traceback is:
Traceback (most recent call last):
  File "/home/suberti/.local/lib/python3.8/site-packages/ansible/template/__init__.py", line 1048, in do_template
    t = myenv.from_string(data)
  File "/home/suberti/.local/lib/python3.8/site-packages/jinja2/environment.py", line 941, in from_string
    return cls.from_code(self, self.compile(source), globals, None)
  File "/home/suberti/.local/lib/python3.8/site-packages/jinja2/environment.py", line 638, in compile
    self.handle_exception(source=source_hint)
  File "/home/suberti/.local/lib/python3.8/site-packages/jinja2/environment.py", line 832, in handle_exception
    reraise(*rewrite_traceback_stack(source=source))
  File "/home/suberti/.local/lib/python3.8/site-packages/jinja2/_compat.py", line 28, in reraise
    raise value.with_traceback(tb)
  File "<unknown>", line 1299, in template
jinja2.exceptions.TemplateSyntaxError: expected token 'end of print statement', got '{'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/suberti/.local/lib/python3.8/site-packages/ansible/plugins/action/template.py", line 139, in run
    resultant = self._templar.do_template(template_data, preserve_trailing_newlines=True, escape_backslashes=False)
  File "/home/suberti/.local/lib/python3.8/site-packages/ansible/template/__init__.py", line 1050, in do_template
    raise AnsibleError("template error while templating string: %s. String: %s" % (to_native(e), to_native(data)))
ansible.errors.AnsibleError: template error while templating string: expected token 'end of print statement', got '{'. String: {
  "__inputs": [
    {
      "name": "DS_THANOS",
      "label": "Thanos",
      "description": "",
      "type": "datasource",
      "pluginId": "prometheus",
      "pluginName": "Prometheus"
[...]

Expected Behavior: Ansible playbook successfully finish. GrafanaDashboard has been created in kubenetes.

Bug Generic Definition of Ready (DoR)

Bug Generic Definition of Done (DoD)

edepremare-cs commented 2 years ago

Dear @suberti-ads, The error comes from this file: /home/suberti/Documents/Git/rs-config/apps/dashboardoperation/json/1-cluster-overview.json This file seems to have some double brackets in it and ansible will try to use this file as a template, like all the files in the folder.

Two solutions:

LAQU156 commented 2 years ago

IVV_CCB_2022_w41 : Under Analysis

LAQU156 commented 2 years ago

IVV_CCB_2022_w42 : Still under analysis

pcuq-ads commented 2 years ago

Following SPRINT 14 review, we were said that a solution has been found. We are wainting for @edepremare-cs feed-back.

LAQU156 commented 2 years ago

IVV_CCB_2022_w43 : Still under analysis

edepremare-cs commented 2 years ago

In the latest release (1.3.0), the app deployment will only run its templating on the files located at the root of the app's directory. Therefore it will be possible to place any file in a subdirectory (json in this issue) and it will not throw any error.

Woljtek commented 2 years ago

IVV_CCB_2022_w44: Already corrected in SPRINT 14. Accepted CS (for traceability), priority minor

LAQU156 commented 1 year ago

CS_CCB_2023_w05 : Icebox

pcuq-ads commented 1 year ago

RSRRv2_SystemCCB : obsolete. A solution for dashboard persistance on disk is under development.