Default alert manager templates get overwritten by the content of `message` field

[maulikpandey]

According to AWS documentation we can use templates to produce JSON based output from alert manager. But in practice the output of templates get overwritten by the content of the message field of the sns_configs.

We need the message field content for integrating with Pagerduty and Slack. We are currently over coming this limitation by using the templating language to create blocks in the message field.

[ampabhi-aws]

Hey @maulikpandey,

Thank you for posting this feature request! I wanted to double click on a few items:

(1) Do you need JSON to be embedded as a sub piece of your message field? Usually, we find that customers want to send the payload as entirely JSON or want to customize the YAML body.

(2) If we provided a JSON function rather than a message override in the documentation, would that solve the issue? How would you use the new function?

[maulikpandey]

(1) Entire payload as a JSON. (2) A JSON function that tells Alert Manager to output a JSON payload (instead of YAML) is sufficient for my use case.

[ampabhi-aws]

For anyone else who comes across this issue, you can structure the entire payload as JSON through the template snippet in the AWS Documentation.

We are discussing ways of improving the toJSON capabilities as can be seen in this Prometheus community discussion: https://github.com/prometheus/alertmanager/issues/2678.

[mjrlee]

@maulikpandey did you ever come up with a template that works for pagerduty? If so, would you be willing to share it? It would be nice to get it in to the official documentation if possible.