search

kube-burner / kube-burner-ocp

OpenShift integrations and workloads for kube-burner
https://kube-burner.github.io/kube-burner-ocp/
Apache License 2.0
4 stars 18 forks source link

[RFE] Allow option to index regardless of workload success #44

Closed dry923 closed 1 month ago

dry923 commented 6 months ago

Is your feature request related to a problem? Please describe. When running a workload there are many times when I want to view the data that would be indexed regardless of the tests success/failure/error. For example, if a test times out I may still want to view the usage of the system but I cannot since it was not indexed as indexing is skipped on error.

Describe the solution you'd like A flag like "--index-on-failure" which can be defaulted to false but would allow the user to have kube-burner attempt to index the data regardless of if the workload "fails|errors|succeeds."

Describe alternatives you've considered I can run the index command after a failed run but this is manual and I may not be sitting there waiting for a job to finish. Additionally, if in an automated job you may never have the opportunity to manually index after.

Additional context Here is what happened when I ran into this:

When running a large 500 node cluster I wanted to generate load on the system to investigate usage patterns of specific nodes. When running 501 workers with 4509 iterations it takes hours for everything to complete. The final verification portion timed out and since it "errored" none of the data was indexed other than the metadata.

snip of the end of the test time="2024-04-03 15:08:01" level=info msg="4500/4509 iterations completed" file="create.go:132" time="2024-04-03 15:08:18" level=info msg="Waiting up to 4h0m0s for actions to be completed" file="create.go:181" I0403 16:34:44.111008 414183 trace.go:236] Trace[981480722]: "Reflector ListAndWatch" name:pkg/mod/k8s.io/client-go@v0.27.2/tools/cache/reflector.go:231 (03-Apr-2024 16:34:18.246) (total time: 25864ms): Trace[981480722]: ---"Objects listed" error:<nil> 25810ms (16:34:44.057) Trace[981480722]: [25.864339508s] [25.864339508s] END time="2024-04-03 16:48:17" level=error msg="4h0m0s timeout reached" file="job.go:272" time="2024-04-03 16:48:17" level=error msg="4h0m0s timeout reached" file="helpers.go:183" time="2024-04-03 16:48:17" level=info msg="Indexing cluster metadata document" file="helpers.go:221" time="2024-04-03 16:48:18" level=info msg="Indexing finished in 799ms: created=1" file="helpers.go:229" time="2024-04-03 16:48:18" level=info msg="👋 Exiting kube-burner 72670a7f-eae9-4a56-9a88-8cc62fa25011" file="helpers.go:189"

If it would have also indexed the data regardless of the test erroring from timeout I would have still had useful data to go over.

github-actions[bot] commented 2 months ago

This issue has become stale and will be closed automatically within 7 days.