How to reduce 504s

[trieloff]

Some ideas:

• increase parallelism (fetch 404 and redirects earlier, now that runtime limits are less of an issue)
• check execution deadline, e.g. if 25 seconds have passed, log the remaining requests, so that we can optimize them
• tell Fastly to re-fetch if no response is ready after 29 seconds – there is a good chance that the second request is faster as the cold starts have been completed
• if there is enough time left, retry 504 in dispatch
• stop resolving git refs (saves 1-2 seconds) #430

[trieloff]

Inner CDN response times (microseconds) for 504s for the default strain (Adobe I/O)

504s occur largely randomly and unpredictably. This indicates that the runtime is flaky.

Inner CDN response times (microseconds) for 504s for the universal strain (AWS)

Note the changed axis scales.

504s are clustered around the lambda integration deadline of 30 seconds. This indicates that the action is often just too slow (maybe due to cold starts)

[trieloff]

Link to viz: https://helix.coralogix.com/goto/83122cae55abec264f7344412a430d7f