search

spcl / serverless-benchmarks

SeBS: serverless benchmarking suite for automatic performance analysis of FaaS platforms.
https://mcopik.github.io/projects/sebs/
BSD 3-Clause "New" or "Revised" License
149 stars 67 forks source link

config._project_name = dct["project_name"] KeyError: 'project_name' #148

Closed jchigu closed 1 year ago

jchigu commented 1 year ago

(python-venv) root@localhost:~/serverless-benchmarks# ./sebs.py benchmark regression test --config config/example.json --deployment gcp --verbose [17:53:23.412149] SeBS-0d09 Created experiment output at /root/serverless-benchmarks/regression-output Select test test_110.dynamic-html Select test test_120.uploader Select test test_210.thumbnailer Select test test_220.video-processing Select test test_311.compression Select test test_411.image-recognition Select test test_501.graph-pagerank Select test test_502.graph-mst Select test test_503.graph-bfs Select test test_504.dna-visualisation [17:53:24.269833] GCPCredentials-6f32 Using cached credentials for GCP [17:53:24.289503] GCPResources-8715 No cached resources for GCP found, using user configuration. [17:53:24.289780] GCPConfig-129c Loading cached config for GCP [17:53:24.283485] GCPCredentials-0fa7 Using cached credentials for GCP [17:53:24.290140] GCPResources-d37c No cached resources for GCP found, using user configuration. [17:53:24.290418] GCPConfig-f195 Loading cached config for GCP [17:53:24.282259] GCPCredentials-9a09 Using cached credentials for GCP [17:53:24.283056] GCPCredentials-2830 Using cached credentials for GCP [17:53:24.297367] GCPResources-464c No cached resources for GCP found, using user configuration. [17:53:24.297794] GCPConfig-51fd Loading cached config for GCP [17:53:24.270354] GCPCredentials-a53b Using cached credentials for GCP [17:53:24.299276] GCPResources-cad8 No cached resources for GCP found, using user configuration. [17:53:24.301333] GCPConfig-6540 Loading cached config for GCP [17:53:24.282689] GCPCredentials-838d Using cached credentials for GCP [17:53:24.302476] GCPResources-287e No cached resources for GCP found, using user configuration. [17:53:24.302752] GCPConfig-2426 Loading cached config for GCP [17:53:24.281769] GCPCredentials-6fa9 Using cached credentials for GCP [17:53:24.304682] GCPResources-95db No cached resources for GCP found, using user configuration. [17:53:24.304956] GCPConfig-75c5 Loading cached config for GCP [17:53:24.285508] GCPCredentials-aad9 Using cached credentials for GCP [17:53:24.307333] GCPResources-f342 No cached resources for GCP found, using user configuration. [17:53:24.307603] GCPConfig-d2a1 Loading cached config for GCP [17:53:24.285793] GCPCredentials-c485 Using cached credentials for GCP [17:53:24.308673] GCPResources-8823 No cached resources for GCP found, using user configuration. [17:53:24.308939] GCPConfig-2870 Loading cached config for GCP

sebs.regression.GCPTestSequence.test_503.graph-bfs: fail sebs.regression.GCPTestSequence.test_503.graph-bfs: Traceback (most recent call last): File "/root/serverless-benchmarks/sebs/regression.py", line 38, in test deployment_client = self.get_deployment(benchmark_name) File "/root/serverless-benchmarks/sebs/regression.py", line 145, in get_deployment deployment_client = self.client.get_deployment( File "/root/serverless-benchmarks/sebs/sebs.py", line 111, in get_deployment deployment_config = Config.deserialize(config, self.cache_client, handlers) File "/root/serverless-benchmarks/sebs/faas/config.py", line 127, in deserialize return func(config[name] if name in config else config, cache, handlers) File "/root/serverless-benchmarks/sebs/gcp/config.py", line 177, in deserialize GCPConfig.initialize(config_obj, cached_config) File "/root/serverless-benchmarks/sebs/gcp/config.py", line 220, in initialize config._project_name = dct["project_name"] KeyError: 'project_name'

sebs.regression.GCPTestSequence.test_502.graph-mst: fail sebs.regression.GCPTestSequence.test_502.graph-mst: Traceback (most recent call last): File "/root/serverless-benchmarks/sebs/regression.py", line 38, in test deployment_client = self.get_deployment(benchmark_name) File "/root/serverless-benchmarks/sebs/regression.py", line 145, in get_deployment deployment_client = self.client.get_deployment( File "/root/serverless-benchmarks/sebs/sebs.py", line 111, in get_deployment deployment_config = Config.deserialize(config, self.cache_client, handlers) File "/root/serverless-benchmarks/sebs/faas/config.py", line 127, in deserialize return func(config[name] if name in config else config, cache, handlers) File "/root/serverless-benchmarks/sebs/gcp/config.py", line 177, in deserialize GCPConfig.initialize(config_obj, cached_config) File "/root/serverless-benchmarks/sebs/gcp/config.py", line 220, in initialize config._project_name = dct["project_name"] KeyError: 'project_name'

sebs.regression.GCPTestSequence.test_210.thumbnailer: fail sebs.regression.GCPTestSequence.test_210.thumbnailer: Traceback (most recent call last): File "/root/serverless-benchmarks/sebs/regression.py", line 38, in test deployment_client = self.get_deployment(benchmark_name) File "/root/serverless-benchmarks/sebs/regression.py", line 145, in get_deployment deployment_client = self.client.get_deployment( File "/root/serverless-benchmarks/sebs/sebs.py", line 111, in get_deployment deployment_config = Config.deserialize(config, self.cache_client, handlers) File "/root/serverless-benchmarks/sebs/faas/config.py", line 127, in deserialize return func(config[name] if name in config else config, cache, handlers) File "/root/serverless-benchmarks/sebs/gcp/config.py", line 177, in deserialize GCPConfig.initialize(config_obj, cached_config) File "/root/serverless-benchmarks/sebs/gcp/config.py", line 220, in initialize config._project_name = dct["project_name"] KeyError: 'project_name'

sebs.regression.GCPTestSequence.test_501.graph-pagerank: fail sebs.regression.GCPTestSequence.test_501.graph-pagerank: Traceback (most recent call last): File "/root/serverless-benchmarks/sebs/regression.py", line 38, in test deployment_client = self.get_deployment(benchmark_name) File "/root/serverless-benchmarks/sebs/regression.py", line 145, in get_deployment deployment_client = self.client.get_deployment( File "/root/serverless-benchmarks/sebs/sebs.py", line 111, in get_deployment deployment_config = Config.deserialize(config, self.cache_client, handlers) File "/root/serverless-benchmarks/sebs/faas/config.py", line 127, in deserialize return func(config[name] if name in config else config, cache, handlers) File "/root/serverless-benchmarks/sebs/gcp/config.py", line 177, in deserialize GCPConfig.initialize(config_obj, cached_config) File "/root/serverless-benchmarks/sebs/gcp/config.py", line 220, in initialize config._project_name = dct["project_name"] KeyError: 'project_name'

sebs.regression.GCPTestSequence.test_311.compression: fail sebs.regression.GCPTestSequence.test_311.compression: Traceback (most recent call last): File "/root/serverless-benchmarks/sebs/regression.py", line 38, in test deployment_client = self.get_deployment(benchmark_name) File "/root/serverless-benchmarks/sebs/regression.py", line 145, in get_deployment deployment_client = self.client.get_deployment( File "/root/serverless-benchmarks/sebs/sebs.py", line 111, in get_deployment deployment_config = Config.deserialize(config, self.cache_client, handlers) File "/root/serverless-benchmarks/sebs/faas/config.py", line 127, in deserialize return func(config[name] if name in config else config, cache, handlers) File "/root/serverless-benchmarks/sebs/gcp/config.py", line 177, in deserialize GCPConfig.initialize(config_obj, cached_config) File "/root/serverless-benchmarks/sebs/gcp/config.py", line 220, in initialize config._project_name = dct["project_name"] KeyError: 'project_name'

[17:53:24.297546] GCPResources-dd06 No cached resources for GCP found, using user configuration. [17:53:24.311840] GCPConfig-a3bc Loading cached config for GCP

sebs.regression.GCPTestSequence.test_504.dna-visualisation: fail sebs.regression.GCPTestSequence.test_504.dna-visualisation: Traceback (most recent call last): File "/root/serverless-benchmarks/sebs/regression.py", line 38, in test deployment_client = self.get_deployment(benchmark_name) File "/root/serverless-benchmarks/sebs/regression.py", line 145, in get_deployment deployment_client = self.client.get_deployment( File "/root/serverless-benchmarks/sebs/sebs.py", line 111, in get_deployment deployment_config = Config.deserialize(config, self.cache_client, handlers) File "/root/serverless-benchmarks/sebs/faas/config.py", line 127, in deserialize return func(config[name] if name in config else config, cache, handlers) File "/root/serverless-benchmarks/sebs/gcp/config.py", line 177, in deserialize GCPConfig.initialize(config_obj, cached_config) File "/root/serverless-benchmarks/sebs/gcp/config.py", line 220, in initialize config._project_name = dct["project_name"] KeyError: 'project_name'

sebs.regression.GCPTestSequence.test_220.video-processing: fail sebs.regression.GCPTestSequence.test_220.video-processing: Traceback (most recent call last): File "/root/serverless-benchmarks/sebs/regression.py", line 38, in test deployment_client = self.get_deployment(benchmark_name) File "/root/serverless-benchmarks/sebs/regression.py", line 145, in get_deployment deployment_client = self.client.get_deployment( File "/root/serverless-benchmarks/sebs/sebs.py", line 111, in get_deployment deployment_config = Config.deserialize(config, self.cache_client, handlers) File "/root/serverless-benchmarks/sebs/faas/config.py", line 127, in deserialize return func(config[name] if name in config else config, cache, handlers) File "/root/serverless-benchmarks/sebs/gcp/config.py", line 177, in deserialize GCPConfig.initialize(config_obj, cached_config) File "/root/serverless-benchmarks/sebs/gcp/config.py", line 220, in initialize config._project_name = dct["project_name"] KeyError: 'project_name'

sebs.regression.GCPTestSequence.test_411.image-recognition: fail sebs.regression.GCPTestSequence.test_411.image-recognition: Traceback (most recent call last): File "/root/serverless-benchmarks/sebs/regression.py", line 38, in test deployment_client = self.get_deployment(benchmark_name) File "/root/serverless-benchmarks/sebs/regression.py", line 145, in get_deployment deployment_client = self.client.get_deployment( File "/root/serverless-benchmarks/sebs/sebs.py", line 111, in get_deployment deployment_config = Config.deserialize(config, self.cache_client, handlers) File "/root/serverless-benchmarks/sebs/faas/config.py", line 127, in deserialize return func(config[name] if name in config else config, cache, handlers) File "/root/serverless-benchmarks/sebs/gcp/config.py", line 177, in deserialize GCPConfig.initialize(config_obj, cached_config) File "/root/serverless-benchmarks/sebs/gcp/config.py", line 220, in initialize config._project_name = dct["project_name"] KeyError: 'project_name'

sebs.regression.GCPTestSequence.test_120.uploader: fail sebs.regression.GCPTestSequence.test_120.uploader: Traceback (most recent call last): File "/root/serverless-benchmarks/sebs/regression.py", line 38, in test deployment_client = self.get_deployment(benchmark_name) File "/root/serverless-benchmarks/sebs/regression.py", line 145, in get_deployment deployment_client = self.client.get_deployment( File "/root/serverless-benchmarks/sebs/sebs.py", line 111, in get_deployment deployment_config = Config.deserialize(config, self.cache_client, handlers) File "/root/serverless-benchmarks/sebs/faas/config.py", line 127, in deserialize return func(config[name] if name in config else config, cache, handlers) File "/root/serverless-benchmarks/sebs/gcp/config.py", line 177, in deserialize GCPConfig.initialize(config_obj, cached_config) File "/root/serverless-benchmarks/sebs/gcp/config.py", line 220, in initialize config._project_name = dct["project_name"] KeyError: 'project_name'

[17:53:24.314749] GCPCredentials-247b Using cached credentials for GCP [17:53:24.315034] GCPResources-0360 No cached resources for GCP found, using user configuration. [17:53:24.315390] GCPConfig-fb86 Loading cached config for GCP

sebs.regression.GCPTestSequence.test_110.dynamic-html: fail sebs.regression.GCPTestSequence.test_110.dynamic-html: Traceback (most recent call last): File "/root/serverless-benchmarks/sebs/regression.py", line 38, in test deployment_client = self.get_deployment(benchmark_name) File "/root/serverless-benchmarks/sebs/regression.py", line 145, in get_deployment deployment_client = self.client.get_deployment( File "/root/serverless-benchmarks/sebs/sebs.py", line 111, in get_deployment deployment_config = Config.deserialize(config, self.cache_client, handlers) File "/root/serverless-benchmarks/sebs/faas/config.py", line 127, in deserialize return func(config[name] if name in config else config, cache, handlers) File "/root/serverless-benchmarks/sebs/gcp/config.py", line 177, in deserialize GCPConfig.initialize(config_obj, cached_config) File "/root/serverless-benchmarks/sebs/gcp/config.py", line 220, in initialize config._project_name = dct["project_name"] KeyError: 'project_name'

Succesfully executed 0 out of 10 functions Failures when executing 10 out of 10 functions

mcopik commented 1 year ago

@jchigu I cannot reproduce the issue. Please follow the template for bug reports (it appears automatically when you open a new issue), provide all necessary information, and format logging outputs correctly (use Markdown's fenced code block). Please attach your config.json and the gcp.json file from cache, after removing your credentials are personal information. It looks like one of these files is broken.

Is there a chance that you previously ran an operation with Google Cloud that crashed? This could lead to writing an incorrect cache.

Once you provide the requested information, I will be happy to reopen the issue and investigate.