search

opensearch-project / opensearch-benchmark

OpenSearch Benchmark - a community driven, open source project to run performance tests for OpenSearch
https://opensearch.org/docs/latest/benchmark/
Apache License 2.0
112 stars 79 forks source link

[Bug]: Unable to list test_executions when there are aggregates with user tags #685

Closed IanHoang closed 1 month ago

IanHoang commented 1 month ago

Describe the bug

When I run list test_executions, an error occurs:

$ opensearch-benchmark list test_executions

   ____                  _____                      __       ____                  __                         __
  / __ \____  ___  ____ / ___/___  ____ ___________/ /_     / __ )___  ____  _____/ /_  ____ ___  ____ ______/ /__
 / / / / __ \/ _ \/ __ \\__ \/ _ \/ __ `/ ___/ ___/ __ \   / __  / _ \/ __ \/ ___/ __ \/ __ `__ \/ __ `/ ___/ //_/
/ /_/ / /_/ /  __/ / / /__/ /  __/ /_/ / /  / /__/ / / /  / /_/ /  __/ / / / /__/ / / / / / / / / /_/ / /  / ,<
\____/ .___/\___/_/ /_/____/\___/\__,_/_/   \___/_/ /_/  /_____/\___/_/ /_/\___/_/ /_/_/ /_/ /_/\__,_/_/  /_/|_|
    /_/

[ERROR] Cannot list. 'list' object has no attribute 'items'.

Getting further help:
*********************
* Check the log files in /Users/hoangia/.benchmark/logs for errors.
* Read the documentation at https://opensearch.org/docs.
* Ask a question on the forum at https://forum.opensearch.org/.
* Raise an issue at https://github.com/opensearch-project/OpenSearch-Benchmark/issues and include the log files in /Users/hoangia/.benchmark/logs.

-------------------------------
[INFO] FAILURE (took 0 seconds)
-------------------------------

OSB assumes user-tags in each test execution as a key-value pair source

--user-tag: Defines user-specific key-value pairs to be used in metric record as meta information, for example, intention:baseline-ticket-12345.

This error is occurring because when OSB tries to fetch the user tags for aggregated test executions, it is unable to get the user tags because it's a list. 

 "user-tags": [
  "57632daf-5031-45c0-abe4-3d4ac8378c1b",
  "32b6c048-7f74-411f-b6c9-2f6abd271a71",
  "9d4cc35c-501f-49a1-80ea-f195bcee26d9"
 ],

To reproduce

  1. Download latest OSB version (1.10.0). Issue also occurs in 1.9.1 and before.
  2. Run opensearch-benchmark list test_executions

Expected behavior

No response

Screenshots

If applicable, add screenshots to help explain your problem.

Host / Environment

No response

Additional context

No response

Relevant log output

No response

IanHoang commented 1 month ago

We can easily fix this by adding keys to each ID. Example:

 "user-tags": [
  "tid_1": "57632daf-5031-45c0-abe4-3d4ac8378c1b",
  "tid_2": "32b6c048-7f74-411f-b6c9-2f6abd271a71",
  "tid_3": "9d4cc35c-501f-49a1-80ea-f195bcee26d9"
 ],
IanHoang commented 1 month ago

Synced offline with @OVI3D0. Issue was on my end as I had outdated aggregate feature.