NASA-IMPACT / csdap-cumulus

SmallSat Cumulus Deployment
Other
1 stars 1 forks source link

FormatProviderPaths output exceeds maximum number of bytes service limit #296

Closed chuckwondo closed 10 months ago

chuckwondo commented 10 months ago

While testing the new "fully" scalable granule discovery workflow, running the WV04_Pan_L1B___1 rule in CBA Prod (discovery only -- no queuing) hit a snag on the FormatProviderPaths lambda function:

The state/task 'arn:aws:lambda:us-west-2:********5047:function:cumulus-prod-FormatProviderPaths'
returned a result with a size exceeding the maximum number of bytes service limit.

Unfortunately, the 256K limit on step function messages appears to apply to the entirety of a message even when the message is a list of inputs for distinct Map step iterations, rather than a limit per iteration input (i.e., per list element).