Fix event loop error

[michaelsaki]

🐛 Summary

When this lambda was made, we first tried to implement it using asyncio.run(process_urls(cve_json_urls, write_db)). This led to an error on the subsequent invocations, RuntimeError('Event loop is closed')

We were able to get a temporary fix in by using asyncio.get_event_loop().run_until_complete(process_urls(cve_json_urls, write_db)). This isn't an optimal solution as was discussed in the PR for this work.

See also here and here.

To reproduce

Steps to reproduce the behavior:

1. Remove the SSM Parameter functionality and pass in a locally setup MongoDB URI.
2. Build the lambda locally on a container.
3. Open a terminal into the container to show the live logs.
4. Invoke the lambda using cloudwatch. Should look something like this curl -XPOST "http://localhost:9000/2015-03-31/functions/function/invocations" -d @cloudwatch.json
5. After the lambda finishes running to completion, do a second invocation to catch the error in the logs.

Expected behavior

The expected behavior is the lambda should successfully run to completion upon any subsequent invocation.

Any helpful log output or screenshots

Paste the results here:

END RequestId: 7f941e39-0005-49c7-aa73-4a1fd6c7f67a
REPORT RequestId: 7f941e39-0005-49c7-aa73-4a1fd6c7f67a  Init Duration: 0.20 ms     Duration: 566786.75 ms  Billed Duration: 566787 ms Memory Size: 10240 MB   Max Memory Used: 10240 MB
START RequestId: a38e2e56-6a44-4f87-a78b-a00b5a589c74 Version: $LATEST
[ERROR] 2023-08-30T15:43:52.850Z        a42566a8-8bfe-4343-803a-cd1cb45abe4c       Problem encountered while processing the CVEs JSON
[ERROR] 2023-08-30T15:43:52.851Z        a42566a8-8bfe-4343-803a-cd1cb45abe4c       Event loop is closed
Traceback (most recent call last):
  File "/var/task/lambda_handler.py", line 261, in handler
    asyncio.run(process_urls(cve_json_urls, write_db))
  File "/var/lang/lib/python3.9/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/var/lang/lib/python3.9/asyncio/base_events.py", line 647, in run_until_complete
    return future.result()
  File "/var/task/lambda_handler.py", line 153, in process_urls
    await init_beanie(database=motor_client[db], document_models=[CVEDoc])
  File "/var/task/beanie/odm/utils/init.py", line 735, in init_beanie
    await Initializer(
  File "/var/task/beanie/odm/utils/init.py", line 121, in __await__
    yield from self.init_class(model).__await__()
  File "/var/task/beanie/odm/utils/init.py", line 702, in init_class
    await self.init_document(cls)
  File "/var/task/beanie/odm/utils/init.py", line 527, in init_document
    build_info = await self.database.command({"buildInfo": 1})
  File "/var/task/motor/metaprogramming.py", line 73, in method
    return framework.run_on_executor(
  File "/var/task/motor/frameworks/asyncio/__init__.py", line 85, in run_on_executor
    return loop.run_in_executor(_EXECUTOR, functools.partial(fn, *args, **kwargs))
  File "/var/lang/lib/python3.9/asyncio/base_events.py", line 806, in run_in_executor
    self._check_closed()
  File "/var/lang/lib/python3.9/asyncio/base_events.py", line 515, in _check_closed
    raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closedEND RequestId: a42566a8-8bfe-4343-803a-cd1cb45abe4c
REPORT RequestId: a42566a8-8bfe-4343-803a-cd1cb45abe4c  Duration: 15.35 ms Billed Duration: 16 ms  Memory Size: 10240 MB      Max Memory Used: 10240 MB
^Z
[1]  + 89732 suspended  debug.sh

Add any screenshots of the problem here.

[jsf9k]

@michaelsaki - Are you saying that the log output above appears if you remove asyncio.get_event_loop().run_until_complete(process_urls(cve_json_urls, write_db)) and replace it with asyncio.run(process_urls(cve_json_urls, write_db))? I just want to make sure I understand the problem.

[michaelsaki]

@jsf9k Yes that is correct. If you use asyncio.run(process_urls(cve_json_urls, write_db)) it will run only once and then crash on any subsequent runs.