search

envoyproxy / envoy-perf

Envoy performance testing
https://envoyproxy.io
Apache License 2.0
130 stars 36 forks source link

Run salvo benmark with Binary Benchmark encountered bazel #147

Closed xu1zhou closed 1 year ago

xu1zhou commented 2 years ago

I was following salvo document to run a binary benchmark within container environment. Building salvo works fine but got error msg below while trying to run bazel-bin/salvo --job demo-binary.yaml:

2022-06-06 07:25:28,165: 4107 [ DEBUG] [base_benchmark] Running benchmark: Local Binary Benchmark [<src.lib.benchmark.binary_benchmark.Benchmark object at 0x7f9f70cf0e80>]                                                                                                                                        
2022-06-06 07:25:28,165: 4107 [ DEBUG] [base_benchmark] Running benchmark: Local Binary Benchmark [<src.lib.benchmark.binary_benchmark.Benchmark object at 0x7f9f70d08cc0>]                                                                                                                                        
2022-06-06 07:25:28,167: 4107 [ DEBUG] [base_benchmark] Running benchmark: Local Binary Benchmark [<src.lib.benchmark.binary_benchmark.Benchmark object at 0x7f9f70d08cf8>]                                                                                                                                        
2022-06-06 07:25:28,167: 4107 [ INFO ] [run_benchmark] ==================== Running Binary Benchmark for e95717bcd0672910826de4a9bb2f402f6e057b2c ====================                                                                                                                                             
2022-06-06 07:25:28,167: 4107 [ DEBUG] [base_builder] Using HOME=/tmp/salvo/tmp7v0jk9vn
2022-06-06 07:25:28,167: 4107 [ DEBUG] [source_tree] Pulling [SRCID_NIGHTHAWK] from origin: [https://github.com/envoyproxy/nighthawk.git]
2022-06-06 07:25:28,168: 4107 [ DEBUG] [cmd_exec] Executing command: [git status] with args [OrderedDict([('cwd', '/tmp/salvo/tmpsta08hel')])]
2022-06-06 07:25:28,171: 4107 [ ERROR] [cmd_exec] Unable to execute [git status]: Command '['git', 'status']' returned non-zero exit status 128.
2022-06-06 07:25:28,172: 4107 [ DEBUG] [cmd_exec] Returning output: [fatal: not a git repository (or any of the parent directories): .git
]
2022-06-06 07:25:28,172: 4107 [ INFO ] [source_tree] Source likely does not exist on disk
2022-06-06 07:25:28,172: 4107 [ DEBUG] [cmd_exec] Executing command: [git clone https://github.com/envoyproxy/nighthawk.git .] with args [OrderedDict([('cwd', '/tmp/salvo/tmpsta08hel')])]                                                                                                                        
2022-06-06 07:26:36,757: 4107 [ DEBUG] [cmd_exec] Returning output: [Cloning into '.'...
]
2022-06-06 07:26:36,758: 4107 [ DEBUG] [nighthawk_builder] NightHawk source path: [/tmp/salvo/tmpsta08hel]
2022-06-06 07:26:36,758: 4107 [ DEBUG] [cmd_exec] Executing command: [bazel clean] with args [OrderedDict([('cwd', '/tmp/salvo/tmpsta08hel')])]
2022-06-06 07:26:37,279: 4107 [ ERROR] [cmd_exec] Unable to execute [bazel clean]: Command '['bazel', 'clean']' returned non-zero exit status 1.
2022-06-06 07:26:37,280: 4107 [ DEBUG] [cmd_exec] Returning output: [2022/06/06 07:26:36 Downloading https://releases.bazel.build/6.0.0-pre.20220421.3/release/bazel-6.0.0-pre.20220421.3-linux-x86_64...                                                                                                          
2022/06/06 07:26:37 could not download Bazel: HTTP GET https://releases.bazel.build/6.0.0-pre.20220421.3/release/bazel-6.0.0-pre.20220421.3-linux-x86_64 failed with error 404                                                                                                                                     
]
Traceback (most recent call last):
  File "/home/yizhou/envoy-perf/salvo/bazel-bin/salvo.runfiles/salvo/salvo.py", line 76, in <module>
    sys.exit(main())
  File "/home/yizhou/envoy-perf/salvo/bazel-bin/salvo.runfiles/salvo/salvo.py", line 70, in main
    benchmark.execute()
  File "/home/yizhou/envoy-perf/salvo/bazel-bin/salvo.runfiles/salvo/src/lib/run_benchmark.py", line 387, in execute
    benchmark.execute_benchmark()
  File "/home/yizhou/envoy-perf/salvo/bazel-bin/salvo.runfiles/salvo/src/lib/benchmark/binary_benchmark.py", line 134, in execute_benchmark
    self._prepare_nighthawk()
  File "/home/yizhou/envoy-perf/salvo/bazel-bin/salvo.runfiles/salvo/src/lib/benchmark/binary_benchmark.py", line 103, in _prepare_nighthawk
    self._nighthawk_builder.build_nighthawk_binaries()
  File "/home/yizhou/envoy-perf/salvo/bazel-bin/salvo.runfiles/salvo/src/lib/builder/nighthawk_builder.py", line 101, in build_nighthawk_binaries
    self.prepare_nighthawk_source()
  File "/home/yizhou/envoy-perf/salvo/bazel-bin/salvo.runfiles/salvo/src/lib/builder/nighthawk_builder.py", line 78, in prepare_nighthawk_source
    self._run_bazel_clean()
  File "/home/yizhou/envoy-perf/salvo/bazel-bin/salvo.runfiles/salvo/src/lib/builder/base_builder.py", line 74, in _run_bazel_clean
    output = cmd_exec.run_command(cmd, cmd_params)
  File "/home/yizhou/envoy-perf/salvo/bazel-bin/salvo.runfiles/salvo/src/lib/cmd_exec.py", line 50, in run_command
    subprocess.check_call(cmd_array, stdout=tmpfile, stderr=tmpfile, **parameters._asdict())
  File "/usr/lib/python3.6/subprocess.py", line 311, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['bazel', 'clean']' returned non-zero exit status 1.
envoybuild@1f239f6a5336:/home/yizhou/envoy-perf/salvo$ wget https://releases.bazel.build/6.0.0-pre.20220421.3/release/bazel-6.0.0-pre.20220421.3-linux-x86_64                                                                                                                                                      
--2022-06-06 07:32:52--  https://releases.bazel.build/6.0.0-pre.20220421.3/release/bazel-6.0.0-pre.20220421.3-linux-x86_64
2022-06-06 07:32:52 ERROR 404: Not Found.

It seems that salvo was trying to wget a non-exist bazel release pkg cause the problem. I will keep looking into the salvo code to find more detail.

xu1zhou commented 2 years ago

@gyohuangxin

gyohuangxin commented 2 years ago

@xu1zhou Thanks for report this issue. I will take a look.

gyohuangxin commented 2 years ago

@xu1zhou I meet this issue too. It seems it will download bazel when executing bazel clean. However, I'm not sure why it fails to download. I will dig into it.

2022-06-06 14:55:34,219: 5206 [ INFO ] [run_benchmark] ==================== Running Binary Benchmark for v1.21.0 ====================
2022-06-06 14:55:34,219: 5206 [ DEBUG] [base_builder] Using HOME=/tmp/salvo/tmp0ppwhmte
2022-06-06 14:55:34,220: 5206 [ DEBUG] [source_tree] Pulling [SRCID_NIGHTHAWK] from origin: [https://github.com/envoyproxy/nighthawk.git]
2022-06-06 14:55:34,220: 5206 [ DEBUG] [cmd_exec] Executing command: [git status] with args [OrderedDict([('cwd', '/tmp/salvo/tmp13kk2zlu')])]
2022-06-06 14:55:34,222: 5206 [ ERROR] [cmd_exec] Unable to execute [git status]: Command '['git', 'status']' returned non-zero exit status 128.
2022-06-06 14:55:34,222: 5206 [ DEBUG] [cmd_exec] Returning output: [fatal: not a git repository (or any of the parent directories): .git
]
2022-06-06 14:55:34,222: 5206 [ INFO ] [source_tree] Source likely does not exist on disk
2022-06-06 14:55:34,222: 5206 [ DEBUG] [cmd_exec] Executing command: [git clone https://github.com/envoyproxy/nighthawk.git .] with args [OrderedDict([('cwd', '/tmp/salvo/tmp13kk2zlu')])]
2022-06-06 14:55:35,554: 5206 [ DEBUG] [cmd_exec] Returning output: [Cloning into '.'...
]
2022-06-06 14:55:35,554: 5206 [ DEBUG] [nighthawk_builder] NightHawk source path: [/tmp/salvo/tmp13kk2zlu]
2022-06-06 14:55:35,555: 5206 [ DEBUG] [cmd_exec] Executing command: [bazel clean] with args [OrderedDict([('cwd', '/tmp/salvo/tmp13kk2zlu')])]
2022-06-06 14:55:35,646: 5206 [ ERROR] [cmd_exec] Unable to execute [bazel clean]: Command '['bazel', 'clean']' returned non-zero exit status 1.
2022-06-06 14:55:35,647: 5206 [ DEBUG] [cmd_exec] Returning output: [2022/06/06 14:55:35 Downloading https://releases.bazel.build/6.0.0-pre.20220421.3/release/bazel-6.0.0-pre.20220421.3-linux-x86_64...
2022/06/06 14:55:35 could not download Bazel: HTTP GET https://releases.bazel.build/6.0.0-pre.20220421.3/release/bazel-6.0.0-pre.20220421.3-linux-x86_64 failed with error 404
]
Traceback (most recent call last):
  File "/envoy-perf/salvo/./bazel-bin/salvo.runfiles/salvo/salvo.py", line 76, in <module>
    sys.exit(main())
  File "/envoy-perf/salvo/./bazel-bin/salvo.runfiles/salvo/salvo.py", line 70, in main
    benchmark.execute()
  File "/envoy-perf/salvo/bazel-bin/salvo.runfiles/salvo/src/lib/run_benchmark.py", line 387, in execute
    benchmark.execute_benchmark()
  File "/envoy-perf/salvo/bazel-bin/salvo.runfiles/salvo/src/lib/benchmark/binary_benchmark.py", line 134, in execute_benchmark
    self._prepare_nighthawk()
  File "/envoy-perf/salvo/bazel-bin/salvo.runfiles/salvo/src/lib/benchmark/binary_benchmark.py", line 103, in _prepare_nighthawk
    self._nighthawk_builder.build_nighthawk_binaries()
  File "/envoy-perf/salvo/bazel-bin/salvo.runfiles/salvo/src/lib/builder/nighthawk_builder.py", line 101, in build_nighthawk_binaries
    self.prepare_nighthawk_source()
  File "/envoy-perf/salvo/bazel-bin/salvo.runfiles/salvo/src/lib/builder/nighthawk_builder.py", line 78, in prepare_nighthawk_source
    self._run_bazel_clean()
  File "/envoy-perf/salvo/bazel-bin/salvo.runfiles/salvo/src/lib/builder/base_builder.py", line 74, in _run_bazel_clean
    output = cmd_exec.run_command(cmd, cmd_params)
  File "/envoy-perf/salvo/bazel-bin/salvo.runfiles/salvo/src/lib/cmd_exec.py", line 50, in run_command
    subprocess.check_call(cmd_array, stdout=tmpfile, stderr=tmpfile, **parameters._asdict())
  File "/usr/lib/python3.6/subprocess.py", line 311, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['bazel', 'clean']' returned non-zero exit status 1.
xu1zhou commented 2 years ago

@gyohuangxin when run on baremeteal, salvo try to download from " https://github.com/bazelbuild/bazel/releases/download/6.0.0-pre.20220421.3/bazel-6.0.0-pre.20220421.3-linux-x86_64" , which is avaliable to get resource.

gyohuangxin commented 2 years ago

@xu1zhou Thanks for the information, it's strange.

github-actions[bot] commented 1 year ago

This issue has been automatically marked as stale because it has not had activity in the last 90 days. It will be closed in the next 14 days unless it is tagged "help wanted" or "no stalebot" or other activity occurs. Thank you for your contributions.

github-actions[bot] commented 1 year ago

This issue has been automatically closed because it has not had activity in the last 104 days. If this issue is still valid, please ping a maintainer and ask them to label it as "help wanted" or "no stalebot". Thank you for your contributions.