search

danielgmyers / flux-swf-client

Apache License 2.0
8 stars 3 forks source link

Truncate success result text if it's too long #22

Open danielgmyers opened 3 years ago

danielgmyers commented 3 years ago

We already truncate the retry text if it's too long, we need to do the same for the success case's result text.

https://github.com/awslabs/flux-swf-client/blob/main/flux/src/main/java/software/amazon/aws/clients/swf/flux/poller/ActivityTaskPoller.java#L245

danielgmyers commented 3 years ago

Looks like there's a reason I didn't do this in the first place. The success message we pass here is actually the encoded json with the step output metadata; if we just truncate it, then the json is no longer valid.

Probably a better solution is to limit the aggregate size of the user-provided attributes in some fashion. We'd need to save some space for the Flux-generated input and output attributes, and account for the overhead from the json encoding.

This field can be 32768 characters long.

danielgmyers commented 2 years ago

Another option would be to split the overflow text into markers, similar to how we split partition metadata across multiple markers, except we probably would have to split the raw text and then reconstruct it in the proper order.

danielgmyers commented 1 year ago

Going to put this off longer; I don't see a way to add markers or any other data long with the activity result.