spcl / serverless-benchmarks

SeBS: serverless benchmarking suite for automatic performance analysis of FaaS platforms.
https://mcopik.github.io/projects/sebs/
BSD 3-Clause "New" or "Revised" License
150 stars 68 forks source link

Error Running Benchmarks ( 020.network-benchmark , 030.clock-synchronization ) #191

Closed prajinkhadka closed 8 months ago

prajinkhadka commented 8 months ago

Describe the bug Error running the benchmarks "020.network-benchmark , 030.clock-synchronization". I haven't checked others as of now. The function does successfully create in the lambda but it seems the input parameters that are to be passed while invoking are incorrect.

To Reproduce Steps to reproduce the behavior:

  1. Go to master branch and clone.
  2. Change Python3.7 to 3.8 ( in example.json )
  3. Run: ./sebs.py benchmark invoke 030.clock-synchronization test --config config/example.json --deployment aws --verbose
  4. See the error

Expected behavior The function should have been invoked 5 times. I also have added the logs below and printed the input_config that is being passed while invoking the function which is :

{'output-bucket': '020.network-benchmark-0-output-333133d5-8755-4e'}

but the lambda function is expecting other parameters :
https://github.com/spcl/serverless-benchmarks/blob/245dbbf7f9b0c4a3cc7b4eabb46b4f0fe4458561/benchmarks/000.microbenchmarks/020.network-benchmark/python/function.py#L11

Logs

(python-venv) Prajin@node0:~/sebs_git/serverless-benchmarks$ ./sebs.py benchmark invoke 020.network-benchmark test --config config/example.json --deployment aws --verbose
[21:48:31.280564] SeBS-981d Created experiment output at /users/Prajin/sebs_git/serverless-benchmarks
[21:48:31.582240] AWS.Resources-3131 Using cached resources for AWS
[21:48:31.582612] AWS.Config-dc8d Using cached config for AWS
[21:48:31.648217] Benchmark-4688 Building benchmark 020.network-benchmark. Reason: no cached code package.
[21:48:31.657482] AWS-54bc Created /users/Prajin/sebs_git/serverless-benchmarks/020.network-benchmark_code/python/3.8/020.network-benchmark.zip archive
[21:48:31.657771] AWS-54bc Zip archive size 0.002676 MB
[21:48:31.658666] Benchmark-4688 Created code package (source hash: f36d86256cf7cd9d41f86257d70d8076), for run on aws with python:3.8
[21:48:31.659466] AWS-54bc Creating new function! Reason: function 020_network_benchmark_python_3_8 not found in cache.
[21:48:31.851969] AWS-54bc Creating function 020_network_benchmark_python_3_8 from /users/Prajin/sebs_git/serverless-benchmarks/cache/020.network-benchmark/aws/python/3.8/020.network-benchmark.zip
[21:48:32.098049] AWS-54bc Waiting for Lambda function to be created...
[21:48:33.258620] AWS-54bc Lambda function has been created.
[21:48:33.581485] AWS.S3-ede3 Bucket 020.network-benchmark-0-output-333133d5-8755-4e for 020.network-benchmark-0-output already exists, skipping.
[21:48:33.833347] AWS.Resources-3131 Using existing HTTP API 020_network_benchmark_python_3_8-http-api
[21:48:33.996567] AWS-54bc Created HTTP trigger for 020_network_benchmark_python_3_8 function. Sleep 5 seconds to avoid cloud errors.
[21:48:39.002153] SeBS-981d Beginning repetition 1/5
the input config here is {'output-bucket': '020.network-benchmark-0-output-333133d5-8755-4e'}
[21:48:39.002298] AWS.HTTPTrigger-bbde Invoke function https://co47cwes0e.execute-api.us-east-1.amazonaws.com
[21:48:41.929462] AWS.HTTPTrigger-bbde Invocation on URL https://co47cwes0e.execute-api.us-east-1.amazonaws.com failed!
[21:48:41.929724] AWS.HTTPTrigger-bbde Output: {'message': 'Internal Server Error'}
ERROR:root:Failed invocation of function! Output: {'message': 'Internal Server Error'}
Traceback (most recent call last):
  File "./sebs.py", line 30, in __call__
    return self.main(*args, **kwargs)
  File "/users/Prajin/sebs/serverless-benchmarks/python-venv/lib/python3.8/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
  File "/users/Prajin/sebs/serverless-benchmarks/python-venv/lib/python3.8/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/users/Prajin/sebs/serverless-benchmarks/python-venv/lib/python3.8/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/users/Prajin/sebs/serverless-benchmarks/python-venv/lib/python3.8/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/users/Prajin/sebs/serverless-benchmarks/python-venv/lib/python3.8/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "./sebs.py", line 72, in wrapper
    return func(*args, **kwargs)
  File "./sebs.py", line 97, in wrapper
    return func(*args, **kwargs)
  File "./sebs.py", line 252, in invoke
    ret = trigger.sync_invoke(input_config)
  File "/users/Prajin/sebs_git/serverless-benchmarks/sebs/aws/triggers.py", line 112, in sync_invoke
    return self._http_invoke(payload, self.url)
  File "/users/Prajin/sebs_git/serverless-benchmarks/sebs/faas/function.py", line 218, in _http_invoke
    raise RuntimeError(f"Failed invocation of function! Output: {output}")
RuntimeError: Failed invocation of function! Output: {'message': 'Internal Server Error'}
[21:48:41.932649] Benchmark-2961 Update cached config /users/Prajin/sebs_git/serverless-benchmarks/cache/aws.json

System (please complete the following information):

Additional context It seems the problem is not being passed the correct values while invoking the function.

mcopik commented 8 months ago

@prajinkhadka These are internal benchmarks used by experiments, and we documented this. These are not meant to directly invoked by the user: https://github.com/spcl/serverless-benchmarks/blob/master/docs/benchmarks.md

In PR #193, I improved the presentation to highlight notes and warnings :)

Please let me know if you think that documentation should be improved here.