search

spcl / serverless-benchmarks

SeBS: serverless benchmarking suite for automatic performance analysis of FaaS platforms.
https://mcopik.github.io/projects/sebs/
BSD 3-Clause "New" or "Revised" License
149 stars 67 forks source link

No Docker images present for local experiments #98

Closed wowu closed 2 years ago

wowu commented 2 years ago

I'm trying to run local experiments as described in https://github.com/spcl/serverless-benchmarks/blob/master/docs/usage.md#local

Running the following command:

./sebs.py local start 110.dynamic-html test out_benchmark.json --config config/local_deployment.json --deployments 1

Produced the following error:

RuntimeError: Docker pull of image build.local.python.3.7 failed!

In spcleth/serverless-benchmarks registry on Docker Hub there are indeed no images with local in their tag.

All previous commands from usage.md were run, and project's virtualenv is activated (with Python 3.10.0).

Using master branch at latest commit (3b57688873853a1ea74e10b02a9d89f3a4d679ac).

Traceback

ERROR:root:Docker pull of image build.local.python.3.7 failed!
Traceback (most recent call last):
  File "/Users/karol/Code/agh/serverless-benchmarks/python-venv/lib/python3.10/site-packages/docker/api/client.py", line 268, in _raise_for_status
    response.raise_for_status()
  File "/Users/karol/Code/agh/serverless-benchmarks/python-venv/lib/python3.10/site-packages/requests/models.py", line 1022, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: http+docker://localhost/v1.41/images/spcleth/serverless-benchmarks:build.local.python.3.7/json

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/karol/Code/agh/serverless-benchmarks/sebs/benchmark.py", line 347, in install_dependencies
    self._docker_client.images.get(repo_name + ":" + image_name)
  File "/Users/karol/Code/agh/serverless-benchmarks/python-venv/lib/python3.10/site-packages/docker/models/images.py", line 314, in get
    return self.prepare_model(self.client.api.inspect_image(name))
  File "/Users/karol/Code/agh/serverless-benchmarks/python-venv/lib/python3.10/site-packages/docker/utils/decorators.py", line 19, in wrapped
    return f(self, resource_id, *args, **kwargs)
  File "/Users/karol/Code/agh/serverless-benchmarks/python-venv/lib/python3.10/site-packages/docker/api/image.py", line 251, in inspect_image
    return self._result(
  File "/Users/karol/Code/agh/serverless-benchmarks/python-venv/lib/python3.10/site-packages/docker/api/client.py", line 274, in _result
    self._raise_for_status(response)
  File "/Users/karol/Code/agh/serverless-benchmarks/python-venv/lib/python3.10/site-packages/docker/api/client.py", line 270, in _raise_for_status
    raise create_api_error_from_http_exception(e)
  File "/Users/karol/Code/agh/serverless-benchmarks/python-venv/lib/python3.10/site-packages/docker/errors.py", line 31, in create_api_error_from_http_exception
    raise cls(e, response=response, explanation=explanation)
docker.errors.ImageNotFound: 404 Client Error for http+docker://localhost/v1.41/images/spcleth/serverless-benchmarks:build.local.python.3.7/json: Not Found ("no such image: spcleth/serverless-benchmarks:build.local.python.3.7: No such image: spcleth/serverless-benchmarks:build.local.python.3.7")

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/karol/Code/agh/serverless-benchmarks/python-venv/lib/python3.10/site-packages/docker/api/client.py", line 268, in _raise_for_status
    response.raise_for_status()
  File "/Users/karol/Code/agh/serverless-benchmarks/python-venv/lib/python3.10/site-packages/requests/models.py", line 1022, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: http+docker://localhost/v1.41/images/create?tag=build.local.python.3.7&fromImage=spcleth%2Fserverless-benchmarks

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/karol/Code/agh/serverless-benchmarks/sebs/benchmark.py", line 355, in install_dependencies
    self._docker_client.images.pull(repo_name, image_name)
  File "/Users/karol/Code/agh/serverless-benchmarks/python-venv/lib/python3.10/site-packages/docker/models/images.py", line 444, in pull
    pull_log = self.client.api.pull(
  File "/Users/karol/Code/agh/serverless-benchmarks/python-venv/lib/python3.10/site-packages/docker/api/image.py", line 428, in pull
    self._raise_for_status(response)
  File "/Users/karol/Code/agh/serverless-benchmarks/python-venv/lib/python3.10/site-packages/docker/api/client.py", line 270, in _raise_for_status
    raise create_api_error_from_http_exception(e)
  File "/Users/karol/Code/agh/serverless-benchmarks/python-venv/lib/python3.10/site-packages/docker/errors.py", line 31, in create_api_error_from_http_exception
    raise cls(e, response=response, explanation=explanation)
docker.errors.NotFound: 404 Client Error for http+docker://localhost/v1.41/images/create?tag=build.local.python.3.7&fromImage=spcleth%2Fserverless-benchmarks: Not Found ("manifest for spcleth/serverless-benchmarks:build.local.python.3.7 not found: manifest unknown: manifest unknown")

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/karol/Code/agh/serverless-benchmarks/./sebs.py", line 30, in __call__
    return self.main(*args, **kwargs)
  File "/Users/karol/Code/agh/serverless-benchmarks/python-venv/lib/python3.10/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/Users/karol/Code/agh/serverless-benchmarks/python-venv/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/karol/Code/agh/serverless-benchmarks/python-venv/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/karol/Code/agh/serverless-benchmarks/python-venv/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/karol/Code/agh/serverless-benchmarks/python-venv/lib/python3.10/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/Users/karol/Code/agh/serverless-benchmarks/./sebs.py", line 72, in wrapper
    return func(*args, **kwargs)
  File "/Users/karol/Code/agh/serverless-benchmarks/./sebs.py", line 410, in start
    func = deployment_client.get_function(
  File "/Users/karol/Code/agh/serverless-benchmarks/sebs/faas/system.py", line 157, in get_function
    rebuilt, _ = code_package.build(self.package_code)
  File "/Users/karol/Code/agh/serverless-benchmarks/sebs/benchmark.py", line 495, in build
    self.install_dependencies(self._output_dir)
  File "/Users/karol/Code/agh/serverless-benchmarks/sebs/benchmark.py", line 357, in install_dependencies
    raise RuntimeError("Docker pull of image {} failed!".format(image_name))
RuntimeError: Docker pull of image build.local.python.3.7 failed!

config/local_deployment.json

{
  "experiments": {
    "deployment": "openwhisk",
    "update_code": false,
    "update_storage": false,
    "download_results": false,
    "runtime": {
      "language": "python",
      "version": "3.7"
    },
    "type": "invocation-overhead",
    "perf-cost": {
      "benchmark": "110.dynamic-html",
      "experiments": [
        "cold",
        "warm",
        "burst",
        "sequential"
      ],
      "input-size": "test",
      "repetitions": 50,
      "concurrent-invocations": 50,
      "memory-sizes": [
        128,
        256
      ]
    },
    "network-ping-pong": {
      "invocations": 50,
      "repetitions": 1000,
      "threads": 1
    },
    "invocation-overhead": {
      "repetitions": 5,
      "N": 20,
      "type": "payload",
      "payload_begin": 1024,
      "payload_end": 6251000,
      "payload_points": 20,
      "code_begin": 1048576,
      "code_end": 261619712,
      "code_points": 20
    },
    "eviction-model": {
      "invocations": 1,
      "function_copy_idx": 0,
      "repetitions": 5,
      "sleep": 1
    }
  },
  "deployment": {
    "name": "aws",
    "aws": {
      "region": "us-east-1",
      "lambda-role": ""
    },
    "azure": {
      "region": "westeurope"
    },
    "gcp": {
      "region": "europe-west1",
      "project_name": "",
      "credentials": ""
    },
    "local": {
      "storage": {
        "address": "172.17.0.2:9000",
        "mapped_port": 9011,
        "access_key": "REDACTED",
        "secret_key": "REDACTED",
        "instance_id": "REDACTED",
        "input_buckets": [],
        "output_buckets": [],
        "type": "minio"
      }
    },
    "openwhisk": {
      "shutdownStorage": false,
      "removeCluster": false,
      "wskBypassSecurity": "true",
      "wskExec": "wsk",
      "experimentalManifest": false,
      "docker_registry": {
        "registry": "",
        "username": "",
        "password": ""
      },
      "storage": {
        "address": "",
        "mapped_port": -1,
        "access_key": "",
        "secret_key": "",
        "instance_id": "",
        "input_buckets": [],
        "output_buckets": [],
        "type": "minio"
      }
    }
  }
}
mcopik commented 2 years ago

@Wowu The problem has been fixed and the missing images are now available on our DockerHub. Thanks for raising the issue!

Please do not hesitate to contact us if you find other problems, we're happy to help!