search

Blazemeter / taurus

Automation-friendly framework for Continuous Testing by
http://gettaurus.org
Apache License 2.0
2.01k stars 462 forks source link

BZT incorrectly reports error when running a gRPC based scenario with a K6Executor #1785

Open elambert opened 10 months ago

elambert commented 10 months ago

I have a scenario that uses the K6 executor to apply a gRPC workload against a server. When I use bzt 1.16.26 to run the scenario, the test appears to execute successfully and I do see results in kpi.csv file. But during post processing, I see the following error in the console

...
22:31:17 ERROR: Child Process Error: Empty results, most likely context-create (K6Executor) failed. Actual reason for this can be found in logs under /Users/lambere/git/ftl/build/blazemeter/context-create/artifacts/2023-11-09_22-31-11.718091

...

snippet from bzt.log

[2023-11-09 22:31:17,629 INFO Engine] Post-processing...
[2023-11-09 22:31:17,629 DEBUG Engine.local] Post-process k6/context-create
[2023-11-09 22:31:17,629 DEBUG Engine] Add existing artifact (move=False): /Users/lambere/git/ftl/build/blazemeter/context-create/artifacts/2023-11-09_22-31-11.718091/kpi.csv
[2023-11-09 22:31:17,629 DEBUG Engine] No need to copy /Users/lambere/git/ftl/build/blazemeter/context-create/artifacts/2023-11-09_22-31-11.718091/kpi.csv
[2023-11-09 22:31:17,631 DEBUG Engine.local] Exception in post_process of K6Executor: Empty results, most likely context-create (K6Executor) failed. Actual reason for this can be found in logs under /Users/lambere/git/ftl/build/blazemeter/cont
ext-create/artifacts/2023-11-09_22-31-11.718091 Traceback (most recent call last):
  File "/opt/homebrew/Cellar/bzt/1.16.26/libexec/lib/python3.12/site-packages/bzt/modules/provisioning.py", line 165, in post_process
    raise ToolError(message, diagnostics)
bzt.ToolError: Empty results, most likely context-create (K6Executor) failed. Actual reason for this can be found in logs under /Users/lambere/git/ftl/build/blazemeter/context-create/artifacts/2023-11-09_22-31-11.718091

[2023-11-09 22:31:17,634 DEBUG Engine] post_process: Empty results, most likely context-create (K6Executor) failed. Actual reason for this can be found in logs under /Users/lambere/git/ftl/build/blazemeter/context-create/artifacts/2023-11-09_2
2-31-11.718091
Traceback (most recent call last):
  File "/opt/homebrew/Cellar/bzt/1.16.26/libexec/lib/python3.12/site-packages/bzt/engine/engine.py", line 356, in post_process
    module.post_process()
  File "/opt/homebrew/Cellar/bzt/1.16.26/libexec/lib/python3.12/site-packages/bzt/modules/provisioning.py", line 174, in post_process
    reraise(exc_info, exc_value)

As stated above, there are results in kpi.csv file. The file contains values for the following metrics:

metric_name,timestamp,metric_value,check,error,error_code,expected_response,group,method,name,proto,scenario,service,status,subproto,tls_version,url,extra_tags,metadata
data_sent,1699597873,0.000000,,,,,::setup,,,,,,,,,,,
data_received,1699597873,0.000000,,,,,::setup,,,,,,,,,,,
iteration_duration,1699597873,0.041542,,,,,::setup,,,,,,,,,,,
grpc_req_duration,1699597873,122.119750,,,,,,Create,<REDACTED>,,default,<REDACTED>,0,,,<REDACTED>,,
checks,1699597873,1.000000,status is OK,,,,,,,,default,,,,,,,
...

Note, I've redacted some values from the contents of the file.

I DO NOT see this problem when I use the K6 Executor to run a "REST" based workload.

Environment Details

[2023-11-09 22:31:11,672 INFO root] Taurus CLI Tool v1.16.26
[2023-11-09 22:31:11,672 DEBUG root] Build: 14557 Fri Sep 22 06:30:08 UTC 2023
[2023-11-09 22:31:11,672 DEBUG root] Extended git info:  93b5d487 Changelog for 1.16.26
[2023-11-09 22:31:11,672 DEBUG root] Command-line options: {'log': '/var/folders/5g/vcbxp4bn64v09lrd7cnrfy8h0000gp/T/bzt_plyp7lh6.log', 'option': None, 'quiet': None, 'verbose': None, 'no_system_configs': None, 'aliases': []}
[2023-11-09 22:31:11,673 DEBUG root] Python: CPython 3.12.0
[2023-11-09 22:31:11,673 DEBUG root] OS: uname_result(system='Darwin', node='beezer', release='22.6.0', version='Darwin Kernel Version 22.6.0: Wed Jul  5 22:21:53 PDT 2023; root:xnu-8796.141.3~6/RELEASE_ARM64_T6020', machine='arm64')
[2023-11-09 22:31:11,673 DEBUG root] Path to interpreter: /opt/homebrew/Cellar/bzt/1.16.26/libexec/bin/python
[2023-11-09 22:31:11,673 DEBUG root] Path to packages: ['/opt/homebrew/Cellar/bzt/1.16.26/libexec/bin', '/opt/homebrew/Cellar/python@3.12/3.12.0/Frameworks/Python.framework/Versions/3.12/lib/python312.zip', '/opt/homebrew/Cellar/python@3.12/3.12.0/Frameworks/Python.framework/Versions/3.12/lib/python3.12', '/opt/homebrew/Cellar/python@3.12/3.12.0/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload', '/opt/homebrew/Cellar/bzt/1.16.26/libexec/lib/python3.12/site-packages', '/opt/homebrew/opt/libcython/libexec/lib/python3.12/site-packages', '/opt/homebrew/opt/six/lib/python3.12/site-packages', '/opt/homebrew/opt/numpy/lib/python3.12/site-packages', '/opt/homebrew/opt/python-certifi/lib/python3.12/site-packages', '/opt/homebrew/opt/python-lxml/lib/python3.12/site-packages', '/opt/homebrew/opt/python-pytz/lib/python3.12/site-packages', '/opt/homebrew/opt/pyyaml/lib/python3.12/site-packages', '/opt/homebrew/lib/python3.12/site-packages']
[2023-11-09 22:31:11,673 DEBUG root] Default python: None
[2023-11-09 22:31:11,673 DEBUG root] Default python3: /usr/bin/python3
[2023-11-09 22:31:11,673 DEBUG root] Using personal config: /Users/lambere/.bzt-rc
[2023-11-09 22:31:11,673 INFO root] Starting with configs: ['/Users/lambere/.bzt-rc', './taurus.yaml']
kamyarz-aws commented 4 months ago

I have a similar issue with Websocket @elambert did you find a fix for your problem?

vigneshwar-selvaraj commented 3 months ago

I have a similar issue @elambert did you find a fix for your problem?