search

aws / aws-lambda-runtime-interface-emulator

Apache License 2.0
921 stars 95 forks source link

Integration test suite is broken #47

Closed skwashd closed 2 years ago

skwashd commented 3 years ago

The tests aren't running properly because of duplicated container names. Further subsequent invocations create further failures because the tear down step is missing some of the containers.

Despite these failures being noted in the console, the test suite only catches one of the failures on the first run. On the second run there are more errors, but the suite passes.

First Invocation

$ make integ-tests-with-docker
go test ./...
?       go.amzn.com/cmd/aws-lambda-rie  [no test files]
ok      go.amzn.com/lambda/agents   (cached)
ok      go.amzn.com/lambda/appctx   (cached)
ok      go.amzn.com/lambda/core (cached)
ok      go.amzn.com/lambda/core/directinvoke    (cached)
?       go.amzn.com/lambda/core/statejson   [no test files]
?       go.amzn.com/lambda/extensions   [no test files]
?       go.amzn.com/lambda/fatalerror   [no test files]
?       go.amzn.com/lambda/interop  [no test files]
ok      go.amzn.com/lambda/logging  (cached)
ok      go.amzn.com/lambda/metering (cached)
ok      go.amzn.com/lambda/rapi (cached)
ok      go.amzn.com/lambda/rapi/handler (cached)
ok      go.amzn.com/lambda/rapi/middleware  (cached)
ok      go.amzn.com/lambda/rapi/model   (cached)
?       go.amzn.com/lambda/rapi/rendering   [no test files]
ok      go.amzn.com/lambda/rapid    (cached)
ok      go.amzn.com/lambda/rapidcore    (cached)
ok      go.amzn.com/lambda/rapidcore/env    (cached)
?       go.amzn.com/lambda/rapidcore/standalone [no test files]
?       go.amzn.com/lambda/rapidcore/telemetry  [no test files]
?       go.amzn.com/lambda/rapidcore/telemetry/logsapi  [no test files]
ok      go.amzn.com/lambda/runtimecmd   (cached)
ok      go.amzn.com/lambda/telemetry    (cached)
docker run --env GOPROXY=direct -v /path/to/aws-lambda-runtime-interface-emulator:/LambdaRuntimeLocal -w /LambdaRuntimeLocal golang:1.14 make compile-lambda-linux
CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -ldflags "-s -w" -o bin/aws-lambda-rie ./cmd/aws-lambda-rie
go: downloading github.com/google/uuid v1.1.2
go: downloading github.com/sirupsen/logrus v1.6.0
go: downloading github.com/jessevdk/go-flags v1.4.0
go: downloading github.com/go-chi/chi v4.1.2+incompatible
go: downloading github.com/go-chi/render v1.0.1
go: downloading golang.org/x/sys v0.0.0-20190422165155-953cdadca894
python3 -m venv .venv
.venv/bin/pip install --upgrade pip
Requirement already satisfied: pip in ./.venv/lib/python3.9/site-packages (21.1.3)
Collecting pip
  Using cached pip-21.2.4-py3-none-any.whl (1.6 MB)
Installing collected packages: pip
  Attempting uninstall: pip
    Found existing installation: pip 21.1.3
    Uninstalling pip-21.1.3:
      Successfully uninstalled pip-21.1.3
Successfully installed pip-21.2.4
.venv/bin/pip install requests
Collecting requests
  Using cached requests-2.26.0-py2.py3-none-any.whl (62 kB)
Collecting certifi>=2017.4.17
  Using cached certifi-2021.5.30-py2.py3-none-any.whl (145 kB)
Collecting urllib3<1.27,>=1.21.1
  Using cached urllib3-1.26.6-py2.py3-none-any.whl (138 kB)
Collecting idna<4,>=2.5
  Using cached idna-3.2-py3-none-any.whl (59 kB)
Collecting charset-normalizer~=2.0.0
  Using cached charset_normalizer-2.0.4-py3-none-any.whl (36 kB)
Installing collected packages: urllib3, idna, charset-normalizer, certifi, requests
Successfully installed certifi-2021.5.30 charset-normalizer-2.0.4 idna-3.2 requests-2.26.0 urllib3-1.26.6
.venv/bin/python3 test/integration/local_lambda/end-to-end-test.py
[+] Building 11.1s (8/8) FINISHED
 => [internal] load build definition from Dockerfile-allinone                                                                                                                                                                       0.0s
 => => transferring dockerfile: 217B                                                                                                                                                                                                0.0s
 => [internal] load .dockerignore                                                                                                                                                                                                   0.0s
 => => transferring context: 2B                                                                                                                                                                                                     0.0s
 => [internal] load metadata for public.ecr.aws/lambda/python:3.8                                                                                                                                                                  11.0s
 => [1/3] FROM public.ecr.aws/lambda/python:3.8@sha256:af98ec7625cc27f3efb189a753f57be48d4bedf4f558770157f3651e8bcb56a6                                                                                                             0.0s
 => [internal] load build context                                                                                                                                                                                                   0.0s
 => => transferring context: 1.71kB                                                                                                                                                                                                 0.0s
 => CACHED [2/3] WORKDIR /var/task                                                                                                                                                                                                  0.0s
 => CACHED [3/3] COPY ./ ./                                                                                                                                                                                                         0.0s
 => exporting to image                                                                                                                                                                                                              0.0s
 => => exporting layers                                                                                                                                                                                                             0.0s
 => => writing image sha256:e26aedd62dcec294fd7314bed8482bb9073623c1c2ff2a005eb6ac1bb6740d36                                                                                                                                        0.0s
 => => naming to docker.io/library/aws-lambda-local:testing                                                                                                                                                                         0.0s
0a1ac6973c0b3af7da123e9b4b83c15a94030dd64b51a224ad38a416bd082443
Ee33c68dd37c9e05bf78dc5a9ac4c717dd4dff152d8253c906c7ad39679796654
.635983bf9fe8d2eaef56cb90ed5a1716992fd883da68d62bacffc42e81f534ca
.707bea10bf31192a7b0e7a98fa07fa983607207a5c47461c11a5c66575b10b85
.61c45528518d5d528b8b49bc548de994c212504850f44ffb920f112ce33d4b19
.d8c0d8aed403fbe34af3199903fcd365a0583a4d84cf7f30e254ca13cedf9b97
.docker: Error response from daemon: Conflict. The container name "/testing" is already in use by container "d8c0d8aed403fbe34af3199903fcd365a0583a4d84cf7f30e254ca13cedf9b97". You have to remove (or rename) that container to be able to reuse that name.
See 'docker run --help'.
.b3fc2c0bdd914fc3d29efbc3992c2a213fb5cc08be97d5c5804144b45c4d5fb5
.docker: Error response from daemon: Conflict. The container name "/testing" is already in use by container "d8c0d8aed403fbe34af3199903fcd365a0583a4d84cf7f30e254ca13cedf9b97". You have to remove (or rename) that container to be able to reuse that name.
See 'docker run --help'.
.envvarcheck
testing
timeout
exception
Error response from daemon: conflict: unable to remove repository reference "aws-lambda-local:testing" (must force) - container 635983bf9fe8 is using its referenced image e26aedd62dce
[+] Building 0.4s (8/8) FINISHED
 => [internal] load build definition from Dockerfile-python36                                                                                                                                                                       0.0s
 => => transferring dockerfile: 217B                                                                                                                                                                                                0.0s
 => [internal] load .dockerignore                                                                                                                                                                                                   0.0s
 => => transferring context: 2B                                                                                                                                                                                                     0.0s
 => [internal] load metadata for public.ecr.aws/lambda/python:3.8                                                                                                                                                                   0.3s
 => [internal] load build context                                                                                                                                                                                                   0.0s
 => => transferring context: 107B                                                                                                                                                                                                   0.0s
 => [1/3] FROM public.ecr.aws/lambda/python:3.8@sha256:af98ec7625cc27f3efb189a753f57be48d4bedf4f558770157f3651e8bcb56a6                                                                                                             0.0s
 => CACHED [2/3] WORKDIR /var/task                                                                                                                                                                                                  0.0s
 => CACHED [3/3] COPY ./ ./                                                                                                                                                                                                         0.0s
 => exporting to image                                                                                                                                                                                                              0.0s
 => => exporting layers                                                                                                                                                                                                             0.0s
 => => writing image sha256:e26aedd62dcec294fd7314bed8482bb9073623c1c2ff2a005eb6ac1bb6740d36                                                                                                                                        0.0s
 => => naming to docker.io/library/aws-lambda-local:testing-py36                                                                                                                                                                    0.0s
cb043dfe61f2b6f6292495ec60baf90faeebe0bd55d0c0e7398dd3ceb79dfbfb
.b59248a9811888eec24fa94bf6990cbccddc2d0176337f65b5f591c022e0992a
.testing
assert-overwritten
Untagged: aws-lambda-local:testing-py36

======================================================================
ERROR: test_context_get_remaining_time_in_default_deadline (__main__.TestEndToEnd)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/path/to/aws-lambda-runtime-interface-emulator/.venv/lib/python3.9/site-packages/urllib3/connectionpool.py", line 699, in urlopen
    httplib_response = self._make_request(
  File "/path/to/aws-lambda-runtime-interface-emulator/.venv/lib/python3.9/site-packages/urllib3/connectionpool.py", line 445, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/path/to/aws-lambda-runtime-interface-emulator/.venv/lib/python3.9/site-packages/urllib3/connectionpool.py", line 440, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/local/Cellar/python@3.9/3.9.6/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1349, in getresponse
    response.begin()
  File "/usr/local/Cellar/python@3.9/3.9.6/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 316, in begin
    version, status, reason = self._read_status()
  File "/usr/local/Cellar/python@3.9/3.9.6/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 285, in _read_status
    raise RemoteDisconnected("Remote end closed connection without"
http.client.RemoteDisconnected: Remote end closed connection without response

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/path/to/aws-lambda-runtime-interface-emulator/.venv/lib/python3.9/site-packages/requests/adapters.py", line 439, in send
    resp = conn.urlopen(
  File "/path/to/aws-lambda-runtime-interface-emulator/.venv/lib/python3.9/site-packages/urllib3/connectionpool.py", line 755, in urlopen
    retries = retries.increment(
  File "/path/to/aws-lambda-runtime-interface-emulator/.venv/lib/python3.9/site-packages/urllib3/util/retry.py", line 532, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/path/to/aws-lambda-runtime-interface-emulator/.venv/lib/python3.9/site-packages/urllib3/packages/six.py", line 769, in reraise
    raise value.with_traceback(tb)
  File "/path/to/aws-lambda-runtime-interface-emulator/.venv/lib/python3.9/site-packages/urllib3/connectionpool.py", line 699, in urlopen
    httplib_response = self._make_request(
  File "/path/to/aws-lambda-runtime-interface-emulator/.venv/lib/python3.9/site-packages/urllib3/connectionpool.py", line 445, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/path/to/aws-lambda-runtime-interface-emulator/.venv/lib/python3.9/site-packages/urllib3/connectionpool.py", line 440, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/local/Cellar/python@3.9/3.9.6/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1349, in getresponse
    response.begin()
  File "/usr/local/Cellar/python@3.9/3.9.6/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 316, in begin
    version, status, reason = self._read_status()
  File "/usr/local/Cellar/python@3.9/3.9.6/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 285, in _read_status
    raise RemoteDisconnected("Remote end closed connection without"
urllib3.exceptions.ProtocolError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/path/to/aws-lambda-runtime-interface-emulator/test/integration/local_lambda/end-to-end-test.py", line 142, in test_context_get_remaining_time_in_default_deadline
    r = requests.post("http://localhost:9006/2015-03-31/functions/function/invocations", json={})
  File "/path/to/aws-lambda-runtime-interface-emulator/.venv/lib/python3.9/site-packages/requests/api.py", line 117, in post
    return request('post', url, data=data, json=json, **kwargs)
  File "/path/to/aws-lambda-runtime-interface-emulator/.venv/lib/python3.9/site-packages/requests/api.py", line 61, in request
    return session.request(method=method, url=url, **kwargs)
  File "/path/to/aws-lambda-runtime-interface-emulator/.venv/lib/python3.9/site-packages/requests/sessions.py", line 542, in request
    resp = self.send(prep, **send_kwargs)
  File "/path/to/aws-lambda-runtime-interface-emulator/.venv/lib/python3.9/site-packages/requests/sessions.py", line 655, in send
    r = adapter.send(request, **kwargs)
  File "/path/to/aws-lambda-runtime-interface-emulator/.venv/lib/python3.9/site-packages/requests/adapters.py", line 498, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))

----------------------------------------------------------------------
Ran 11 tests in 43.669s

FAILED (errors=1)
make: *** [integ-tests-with-docker] Error 1

Second Invocation

$ make integ-tests
python3 -m venv .venv
.venv/bin/pip install --upgrade pip
Requirement already satisfied: pip in ./.venv/lib/python3.9/site-packages (21.2.4)
.venv/bin/pip install requests
Requirement already satisfied: requests in ./.venv/lib/python3.9/site-packages (2.26.0)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in ./.venv/lib/python3.9/site-packages (from requests) (1.26.6)
Requirement already satisfied: charset-normalizer~=2.0.0 in ./.venv/lib/python3.9/site-packages (from requests) (2.0.4)
Requirement already satisfied: certifi>=2017.4.17 in ./.venv/lib/python3.9/site-packages (from requests) (2021.5.30)
Requirement already satisfied: idna<4,>=2.5 in ./.venv/lib/python3.9/site-packages (from requests) (3.2)
.venv/bin/python3 test/integration/local_lambda/end-to-end-test.py
[+] Building 1.1s (8/8) FINISHED
 => [internal] load build definition from Dockerfile-allinone                                                                                                                                                                       0.0s
 => => transferring dockerfile: 217B                                                                                                                                                                                                0.0s
 => [internal] load .dockerignore                                                                                                                                                                                                   0.0s
 => => transferring context: 2B                                                                                                                                                                                                     0.0s
 => [internal] load metadata for public.ecr.aws/lambda/python:3.8                                                                                                                                                                   1.0s
 => [1/3] FROM public.ecr.aws/lambda/python:3.8@sha256:af98ec7625cc27f3efb189a753f57be48d4bedf4f558770157f3651e8bcb56a6                                                                                                             0.0s
 => [internal] load build context                                                                                                                                                                                                   0.0s
 => => transferring context: 107B                                                                                                                                                                                                   0.0s
 => CACHED [2/3] WORKDIR /var/task                                                                                                                                                                                                  0.0s
 => CACHED [3/3] COPY ./ ./                                                                                                                                                                                                         0.0s
 => exporting to image                                                                                                                                                                                                              0.0s
 => => exporting layers                                                                                                                                                                                                             0.0s
 => => writing image sha256:e26aedd62dcec294fd7314bed8482bb9073623c1c2ff2a005eb6ac1bb6740d36                                                                                                                                        0.0s
 => => naming to docker.io/library/aws-lambda-local:testing                                                                                                                                                                         0.0s
docker: Error response from daemon: Conflict. The container name "/remainingtimedefault" is already in use by container "0a1ac6973c0b3af7da123e9b4b83c15a94030dd64b51a224ad38a416bd082443". You have to remove (or rename) that container to be able to reuse that name.
See 'docker run --help'.
.docker: Error response from daemon: Conflict. The container name "/remainingtimeten" is already in use by container "e33c68dd37c9e05bf78dc5a9ac4c717dd4dff152d8253c906c7ad39679796654". You have to remove (or rename) that container to be able to reuse that name.
See 'docker run --help'.
.docker: Error response from daemon: Conflict. The container name "/remainingtimethree" is already in use by container "635983bf9fe8d2eaef56cb90ed5a1716992fd883da68d62bacffc42e81f534ca". You have to remove (or rename) that container to be able to reuse that name.
See 'docker run --help'.
.6e0a5ba623f331fcd11891836d395c7a899813b45031e3079470c9d8b17236b4
.43040af058d655d5a5953e6c55dedf69f83b2f09ec7e45243757371c377a6779
.b640cd2ee547ab8745cefb0abaad543c58cfd7ee750220207234c08858f5b0ea
.docker: Error response from daemon: Conflict. The container name "/testing" is already in use by container "b640cd2ee547ab8745cefb0abaad543c58cfd7ee750220207234c08858f5b0ea". You have to remove (or rename) that container to be able to reuse that name.
See 'docker run --help'.
.fc8077999ffa686ccec9a5f940d1c0ccdb44bab8eb0f43da0db7d030fc3b976e
.docker: Error response from daemon: Conflict. The container name "/testing" is already in use by container "b640cd2ee547ab8745cefb0abaad543c58cfd7ee750220207234c08858f5b0ea". You have to remove (or rename) that container to be able to reuse that name.
See 'docker run --help'.
.envvarcheck
testing
timeout
exception
Error response from daemon: conflict: unable to remove repository reference "aws-lambda-local:testing" (must force) - container 635983bf9fe8 is using its referenced image e26aedd62dce
[+] Building 0.4s (8/8) FINISHED
 => [internal] load build definition from Dockerfile-python36                                                                                                                                                                       0.0s
 => => transferring dockerfile: 217B                                                                                                                                                                                                0.0s
 => [internal] load .dockerignore                                                                                                                                                                                                   0.0s
 => => transferring context: 2B                                                                                                                                                                                                     0.0s
 => [internal] load metadata for public.ecr.aws/lambda/python:3.8                                                                                                                                                                   0.3s
 => [1/3] FROM public.ecr.aws/lambda/python:3.8@sha256:af98ec7625cc27f3efb189a753f57be48d4bedf4f558770157f3651e8bcb56a6                                                                                                             0.0s
 => [internal] load build context                                                                                                                                                                                                   0.0s
 => => transferring context: 107B                                                                                                                                                                                                   0.0s
 => CACHED [2/3] WORKDIR /var/task                                                                                                                                                                                                  0.0s
 => CACHED [3/3] COPY ./ ./                                                                                                                                                                                                         0.0s
 => exporting to image                                                                                                                                                                                                              0.0s
 => => exporting layers                                                                                                                                                                                                             0.0s
 => => writing image sha256:e26aedd62dcec294fd7314bed8482bb9073623c1c2ff2a005eb6ac1bb6740d36                                                                                                                                        0.0s
 => => naming to docker.io/library/aws-lambda-local:testing-py36                                                                                                                                                                    0.0s
45bb379b15c6edec3bfc81336e389c1e5224c5ee221219fcdd6706068c6168dc
.1bf6581903f9aa1a0852c73f5f6788fa5856de21034136fe701cc8ff7521ae3f
.testing
assert-overwritten
Untagged: aws-lambda-local:testing-py36

----------------------------------------------------------------------
Ran 11 tests in 35.430s

OK
valerena commented 2 years ago

Fixed by the linked PR. Let us know if you're still experiencing these issues.