Limit error size return from api to avoid size exceed error

[longquanzheng]

{
  "message": "Failure exceeds size limit.",
  "source": "",
  "stackTrace": "",
  "encodedAttributes": null,
  "cause": {
    "message": "",
    "source": "GoSDK",
    "stackTrace": "",
    "encodedAttributes": null,
    "cause": null,
    "applicationFailureInfo": {
      "type": "STATE_API_FAIL_MAX_OUT_RETRY_ERROR_TYPE",
      "nonRetryable": false,
      "details": null
    }
  },
  "serverFailureInfo": {
    "nonRetryable": false
  }
}

We should truncate the error before returning back to activity result.

based on a convo with Temporal, there is a limit of 4K for activity failure. So we need to truncate it to 3K(to be safer because our error also have other stuff). The error message is composed in https://github.com/indeedeng/iwf/blob/064da989953c4102ec866e6e83de62b523a006c4/service/interpreter/activityImpl.go#L111 using "Sprintf"

See thread: https://temporalio.slack.com/archives/CTDTU3J4T/p1685675923648969

[longquanzheng]

This is not a problem anymore within Indeed because we have an internal controller wrapper to truncate the error already, but we should still fix it in the server to prevent this and fixing it for the community

It's still a problem for internal Indeed because limiting in the SDK is not guaranteed to work because of other things...