search

spcl / serverless-benchmarks

SeBS: serverless benchmarking suite for automatic performance analysis of FaaS platforms.
https://mcopik.github.io/projects/sebs/
BSD 3-Clause "New" or "Revised" License
149 stars 67 forks source link

Openwhisk regression test : Succesfully executed 0 out of 0 functions #147

Closed HelloWorldGitHubUser closed 1 year ago

HelloWorldGitHubUser commented 1 year ago

when I run the regression test on Openwhisk, the result is quite tricky:

(python-venv) lyan@user-1288H-V6:~/sebs2/serverless-benchmarks$ ./sebs.py benchmark regression test --config config/openwhisk.json --deployment openwhisk [10:09:51.818885] SeBS-5866 Created experiment output at /home/lyan/sebs2/serverless-benchmarks/regression-output Succesfully executed 0 out of 0 functions

why is it '0 out of 0'? But when I proceed with benchmarks and experiment part, everything seems right. btw, since I haven't deploy with aws, the regression can't proceed with aws but the result will be '0 out of 10' but not '0 out of 0'. So maybe there's sth wrong with the Openwhisk thing? : )

mcopik commented 1 year ago

@HelloWorldGitHubUser Interesting - did you try running with --verbose?

mcopik commented 1 year ago

@HelloWorldGitHubUser @jchigu Thank you for raising the issue!

Indeed, regression was not fully enabled on OpenWhisk. It has been fixed - now it executes correctly on the dev branch. The solution will be merged with master soon.