Scheduled Lambda timeout error 504 after execution

[10Bude10]

Bug Report

Current Behavior A few seconds after a scheduled lambda is successfully executed an error occurs and will be displayed in console: Failed to execute scheduled function: [emitStatistics] Error: Error: [504] - Lambda timeout. It seems that serverless-offline can't handle the scheduled lambda in offline mode. After deploying everything works fine. There's a stackoverflow article with the same described problem: https://stackoverflow.com/questions/76415087/serverless-offline-lambda-timeout-504

• file: serverless.yml

plugins:

• serverless-offline

provider: runtime: nodejs16.x stage: offline

functions: emitStatistics: handler: src/handler/stats/handler.emitStatistics events:

• schedule: rate(1 hour)

• file: handler.js

  export function emitStatistics(event: APIGatewayProxyEvent, context: Context) {
  invokeWithDatabase(event, context, (conn) => handler(event, conn).emitAll());
  }

Expected behavior/code No error occurs after scheduled lambda is executed

Environment

• serverless version: 3.33.0
• serverless-offline version: 12.0.4
• node.js version: 16.20.0
• OS: macOS 13.4.1

Additional context/Screenshots

[johnfilo-kmart]

@10Bude10 did you manage to fix this, I am seeing the same error locally after successful schedule execution

[dnalborczyk]

are you possibly running a long(er) running handler? the default lambda timeout is 6 seconds. the lambda might (or might not) run to completion, even if you see the lambda timeout error. that's expected, as the real lambda would behave similar, depending on how long it's running and when it's timing out.

you could try running your scheduler with a higher timeout, or alternatively, with the noTimeout option. if you still think it's a bug, which is possible, it would be helpful if you could provide a repro repository.

[johnfilo-kmart]

Ah, ok I see. Yeah, this handler runs for around 3-4 minutes.

Thanks for the tip. I will try add the timeout: parameter to this handler and see if that resolves it.

[johnfilo-kmart]

That seems to have fixed the 504 timeout error. However, I still see that scheduler-offline triggers my cronHanlder just fine at the right time, and it runs fine, but right after it finishes it seems to trigger it a second time.

I print out the function name of my handler as the very first thing, which is <proj_name>-dev-cronHandler, and it looks like this:

2023-10-05 10:25:01,908 56418-4379182464 root [cron.py:14] : INFO: Function <proj_name>-dev-cronHandler: STARTED

But when it triggers a second time immediately after the first run has finished, the handler name changes to Fake and it prints out this:

2023-10-05 11:32:09,423 57567-4337517952 root [cron.py:14] : INFO: Function Fake: STARTED

This is the code in the handler:

function_name = context.function_name logger.info("Function " + function_name + ": STARTED")

I've no idea why it's doing that only when I run it locally. It was this second schedule trigger that was causing the 504 timeout error btw

[dnalborczyk]

@johnfilo-kmart could provide a small repro repository which shows the problem? seems to be specific to python.

[johnfilo-kmart]

sure, here you go. See README for steps to setup and reproduce https://github.com/johnfilo-kmart/sls-schedule-fake-handler

[johnfilo-kmart]

one thing I have noticed is if I uninstall serverless-offline-scheduler and run it then I only get the Fake handler log events, and it only runs once without the 504 timeout error. mmm interesting

[dnalborczyk]

thank you @johnfilo-kmart ! I didn't know you were also using serverless-offline-scheduler in addition to serverless-offline. serverless-offline has the scheduler functionality built-in now, so it's essentially redundant. I imagine that both plugins run the same code. you are essentially running it twice. sorry for the confusion, the README is also not really clear about this. the serverless-offline-scheduler plugin is also unmaintained: https://github.com/ajmath/serverless-offline-scheduler .

If you remove serverless-offline-scheduler from the serverless.yml your handler should be running as expected, and run only once per interval.

the problem I'm still seeing tho is the Fake naming stuff, that should ideally be fixed.

[dnalborczyk]

btw, context.callbackWaitsForEmptyEventLoop = False (https://github.com/johnfilo-kmart/sls-schedule-fake-handler/blob/e492c032c05be8f1f77bab18444b55d032470a55/cron.py#L8C5-L8C43) shouldn't provide any functionality as far as I know when used with Python. I'm surprised it's even accessible on the context for Python environments.

[johnfilo-kmart]

Ok. This makes sense. I'll remove that plugin.

Yeah, the Fake handler name isn't ideal though it is only running the once now and without the 504 timeout error so that's better!

Appreciate your time on this.

[johnfilo-kmart]

Ah. Yeah, I had added that to see if it made any difference to this twice running timeout behaviour.

Ok. I'll remove that also. Thanks.