ModuleNotFoundError: Unable to import module in Lambda with Serverless SDK

Description

We are encountering a ModuleNotFoundError during the execution of an AWS Lambda function configured through Serverless. The error suggests that the function's handler module cannot be located or is not recognized as a Python package. This issue arose upon redeploying the function without any changes to the path or configuration, which previously worked in past deployments.

Error Details

Timestamp: 2024-05-13T12:24:28.530Z
File: /opt/sls-sdk-python/exec_wrapper.py

Error Message:

[ERROR] Runtime.ImportModuleError: Unable to import module 'platform/agents/mainAgent/lambda_function': No module named 'platform.agents'; 'platform' is not a package
Traceback (most recent call last):
File "/opt/sls-sdk-python/exec_wrapper.py", line 86, in main
_set_handler()
File "/opt/sls-sdk-python/exec_wrapper.py", line 70, in _set_handler
if not _module_exists(handler_module_dir, handler_module_basename):
File "/opt/sls-sdk-python/exec_wrapper.py", line 26, in _module_exists
return importlib.util.find_spec(module_name) is not None
File "/var/lang/lib/python3.9/importlib/util.py", line 94, in find_spec
parent = __import__(parent_name, fromlist=['__path__'])
ModuleNotFoundError: No module named 'platform.agents'; 'platform' is not a package

Serverless.yml

platformMainAgent:
    description: Handle various type of text requests
    role: platformMainAgentRole
    handler: platform/agents/mainAgent/lambda_function.handler
    architecture: arm64
    timeout: 60
    runtime: python3.9
  events:
      - schedule:
          rate: ${self:custom.rates.${self:provider.stage}}
          enabled: true
          inputTransformer:
            inputTemplate: '{}'
    layers:
      - { Ref: OpenaiLambdaLayer }
      - { Ref: TiktokenLambdaLayer }

Expected Behavior

The Lambda function should successfully locate and import the specified handler module without errors, as it did in previous deployments.

Actual Behavior

The function fails to start due to an inability to import the specified module, indicating a potential issue in how the Serverless SDK handles module paths or packages, especially since there were no changes made to the configuration or paths that previously worked.

Steps to Reproduce

Reproduction does not seem to be possible. Serverless support writes the following:

We have recently been seeing sporadic issues with AWS Lambda and importing of Lambda layers, and not specifically those created Serverless Dashboard. Unfortunately, the incidences have been very low volume (2-3 users affected so far we have encountered) and we have not been able to replicate ourselves.

Additional Information

This issue is particularly perplexing as it occurred without any modifications to the Lambda configuration or the file structure. It suggests a potential underlying issue with how the Serverless framework or AWS Lambda handles redeployments or updates to the runtime environment. Insights or recommendations on resolving this sudden error would be highly appreciated.

What we tried

Moving the function into a different folder, redeploying with serverless.yml adjusted accordingly
Changing runtime to python 3.8, deploying, then changing it back to 3.9, deploying
Deploying in multiple stages with various configurations
Rolling back everything to how we deployed a week ago

-> nothing works

serverless / dashboard