search

hitachienergy / epiphany

Cloud and on-premises automation for Kubernetes centered industrial grade solutions.
Apache License 2.0
138 stars 107 forks source link

[BUG] Adaptive mode: not all files are downloaded when `k8s_as_cloud_service` flag is used #3222

Closed przemyslavic closed 2 years ago

przemyslavic commented 2 years ago

Describe the bug There is a bug in adaptive mode causing that not all necessary files are downloaded when k8s_as_cloud_service flag is set to true.

How to reproduce Steps to reproduce the behavior:

  1. Prepare manifest file: enable only repository component and set k8s_as_cloud_service flag to true
  2. Run download-requirements.py script

Output:

2022-07-15 13:15:22,603 [INFO]: Manifest summary:
--------------------------------------------------
Components requested:
- repository

Features requested:
- filebeat
- firewall
- image-registry
- node-exporter
- repository

Files to download:
- https://charts.bitnami.com/bitnami/node-exporter-2.3.17.tgz
- https://charts.rook.io/release/rook-ceph-cluster-v1.8.8.tgz
- https://charts.rook.io/release/rook-ceph-v1.8.8.tgz
- https://helm.elastic.co/helm/filebeat/filebeat-7.12.1.tgz

Images to download:
- registry:2.8.0

There is no node-exporter binary included.

When the flag is set to false then node exporter appears:

2022-07-15 13:16:22,109 [INFO]: Manifest summary:
--------------------------------------------------
Components requested:
- repository

Features requested:
- filebeat
- firewall
- image-registry
- node-exporter
- repository

Files to download:
- https://github.com/prometheus/node_exporter/releases/download/v1.3.1/node_exporter-1.3.1.linux-amd64.tar.gz

Images to download:
- registry:2.8.0

The same with logging component. When the flag is set to false there are listed a few files related to opensearch:

Files to download:
- https://artifacts.opensearch.org/releases/bundle/opensearch-dashboards/1.2.0/opensearch-dashboards-1.2.0-linux-x64.tar.gz
- https://artifacts.opensearch.org/releases/bundle/opensearch/1.2.4/opensearch-1.2.4-linux-x64.tar.gz
- https://github.com/opensearch-project/perftop/releases/download/1.2.0.0/opensearch-perf-top-1.2.0.0-linux-x64.zip
- https://github.com/prometheus/node_exporter/releases/download/v1.3.1/node_exporter-1.3.1.linux-amd64.tar.gz

and when the flag is set to true there are only helm chart files:

Files to download:
- https://charts.bitnami.com/bitnami/node-exporter-2.3.17.tgz
- https://charts.rook.io/release/rook-ceph-cluster-v1.8.8.tgz
- https://charts.rook.io/release/rook-ceph-v1.8.8.tgz
- https://helm.elastic.co/helm/filebeat/filebeat-7.12.1.tgz

Most likely the problem is in this condition: https://github.com/epiphany-platform/epiphany/blob/develop/ansible/playbooks/roles/repository/files/download-requirements/src/config/config.py#L273 by which particular files are added to the files_to_exclude list if they are not a dependency for kubernetes_master.

Expected behavior A clear and concise description of what you expected to happen.

Config files If applicable, add config files to help explain your problem.

Environment

epicli version: [2.0.1dev]

Additional context Add any other context about the problem here.

DoD checklist