more readable slack messages

[laroque]

It would be awesome if we could format messages sent to slack. in particular, I'm thinking separating the error message from the traceback (the latter in either a code block ` or in a quote block >>>)... That sounds hard since the receiver is just grabbing a string from a message payload, we'd probably have to rework the alert message payloads (keeping in mind that we want those to be more generic so that they are usable for other notification systems like twitter/sms/email/etc.). Depending on that thought, this may require a dripline use protocol change.

Super low priority, but something to maybe think about if someone earns themself a diversion project.

[wcpettus]

By my count, we dump the traceback on Slack in 4 places:

• in dragonfly:open_spimescape_portal, ln 77
• in dripline:service when a service is crashing, ln 435
• in dripline:endpoint when a handle_request fails, ln 197
• in dripline:scheduler when a process_schedule fails, ln 180

The traceback exists in the code in several other places, but is left at info or debug level

I would propose de-escalating all of the above cases to logger.info; there is already an associated message saying the brief error going to slack. The traceback is demonstrably ill-suited to slack, and the error messages should be sufficiently descriptive to give a sense of the error.