search

cloud-bulldozer / benchmark-operator

The Chuck Norris of cloud benchmarks
Apache License 2.0
282 stars 129 forks source link

Image-pull does not show the metrics - manager container logs shows syntax error in ansible playbook #645

Closed KarthikDev closed 3 years ago

KarthikDev commented 3 years ago

Describe the bug Image Pull creates the pods successfully. Pods are in running state. However this is the error i could see in manager container logs:

TASK [include_role] **** fatal: [localhost]: FAILED! => { "msg": "The conditional check 'system_metrics.collection | bool' failed. The error was: error while evaluating conditional (system_metrics.collection | bool): 'system_metrics' is undefined\n\nThe error appears to be in '/opt/ansible/playbooks/benchmark.yml': line 103, column 5, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n - include_role:\n ^ here\n" }

rsevilla87 commented 3 years ago

Can you paste the output of the benchmark?

kubectl get benchmark <your_benchmar> -o yaml
KarthikDev commented 3 years ago

TASK [include_role] **** fatal: [localhost]: FAILED! => {  "msg": "The conditional check 'cr_state is defined and cr_state.resources[0].status is defined and not cr_state.resources[0].status.complete|bool and (cr_state.resources[0].status.state is not defined or cr_state.resources[0].status.state != \"Error\")' failed. The error was: error while evaluating conditional (cr_state is defined and cr_state.resources[0].status is defined and not cr_state.resources[0].status.complete|bool and (cr_state.resources[0].status.state is not defined or cr_state.resources[0].status.state != \"Error\")): 'dict object' has no attribute 'complete'\n\nThe error appears to be in '/opt/ansible/playbooks/benchmark.yml': line 74, column 9, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n - include_role:\n ^ here\n" }

KarthikDev commented 3 years ago

apiVersion: ripsaw.cloudbulldozer.io/v1alpha1 kind: Benchmark metadata: annotations: kubectl.kubernetes.io/last-applied-configuration: | {"apiVersion":"ripsaw.cloudbulldozer.io/v1alpha1","kind":"Benchmark","metadata":{"annotations":{},"name":"image-pull","namespace":"benchmark-operator"},"spec":{"workload":{"args":{"image_list":["docker://registry.artifactory.intra.abc.in/redis/redis:3.2-alpine"],"pod_count":2,"retries":1,"timeout":100},"name":"image_pull"}}} creationTimestamp: "2021-08-17T13:44:11Z" generation: 1 name: image-pull namespace: benchmark-operator resourceVersion: "113048818" selfLink: /apis/ripsaw.cloudbulldozer.io/v1alpha1/namespaces/benchmark-operator/benchmarks/image-pull uid: 7eba1a82-8f11-4c27-970b-3757b950fb8f spec: metadata: collection: false force: false image: quay.io/cloud-bulldozer/backpack:latest privileged: false serviceaccount: default ssl: false stockpileSkipTags: [] stockpileTags:

rsevilla87 commented 3 years ago

Thanks for reporting @KarthikDev , the error you mentioned is not related to the workload itself. It should be handled properly though, will file a PR to address it.

rsevilla87 commented 3 years ago

it should be fixed by one of the CRD changes proposed at:

KarthikDev commented 3 years ago

Hi @rsevilla87 , The image-pull output, will it be only offloaded and viewed in Elastic search or can it be seen as part of controller-manager pods logs?

KarthikDev commented 3 years ago

Hi @rsevilla87 , Looks like image-pull has broken again. Benchmark/image-pull is in Building state.

TASK [Image pull pods] **** fatal: [localhost]: FAILED! => { "msg": "The task includes an option with an undefined variable. The error was: 'dict object' has no attribute 'url'\n\nThe error appears to be in '/opt/ansible/roles/image_pull/tasks/main.yml': line 4, column 5, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n - name: Image pull pods\n ^ here\n" }

{"level":"error","ts":1629893782.951839,"logger":"logging_event_handler","msg":"","name":"image-pull","namespace":"benchmark-operator","gvk":"ripsaw.cloudbulldozer.io/v1alpha1, Kind=Benchmark","event_type":"runner_on_failed","job":"3328451335138149956","EventData.Task":"Image pull pods","EventData.TaskArgs":"","EventData.FailedTaskPath":"/opt/ansible/roles/image_pull/tasks/main.yml:4","error":"[playbook task failed]","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\t/go/pkg/mod/github.com/go-logr/zapr@v0.2.0/zapr.go:132\ngithub.com/operator-framework/operator-sdk/internal/ansible/events.loggingEventHandler.Handle\n\t/workspace/internal/ansible/events/log_events.go:110"}