No feedback for job completion

[dchaley]

When the DeepCell job finishes, there's no feedback for the user. How should we let the user know when the job is complete?

[dchaley]

Our preferred feedback was via email, nice & simple. But this requires an SMTP server. SendGrid has a free plan w/ 100 daily messages, but we need to run 350 inputs… Google Workspace accounts also have email servers but our target environment doesn't use Workspace.

Our next thought was to invoke a webhook as discussed here: https://cloud.google.com/architecture/sending-notifications-for-google-cloud-events

Teams

Our target environment uses Teams. Slack also supports creating webhooks to send messages to channels.

I used a Teams account (paid, but I'm not sure which plan) I have to create a workflow using the template: "Post to a channel when a webhook request is received". The documentation is slightly wrong, you need to go to the "Workflows" section (there is no "Manage" menu option).

It produces a URL which can be POSTed to to trigger the notification. We couldn't find documentation about the POST body format, we found this post giving us a format:

{
    "type": "message",
    "attachments": [
        {
            "contentType": "application/vnd.microsoft.card.adaptive",
            "contentUrl": null,
            "content": {
                "$schema": "http://adaptivecards.io/schemas/adaptive-card.json",
                "type": "AdaptiveCard",
                "version": "1.2",
                "body": [
                    {
                        "type": "TextBlock",
                        "text": "Sample Text!"
                    }
                ]
            }
        }
    ]
}

This resulted in the following post to the channel!

Not sure how to get rid of the "get template" prompt.

But, presumably we can add more content to this, such as links to the job, or at least the result TIFF/npz.

Slack

Meanwhile, free Slack accounts don't support creating webhooks. In previous lives, I've created webhooks so I know it's possible.

Our own usage

We don't use paid Teams or Slack for our own work but we do have Workspace accounts available.

We don't "need" notifications, it's just nice to not refresh the page in loops. It's also nice to get output URIs without fishing through the logs.

---

Next steps:

• Verify that users in target environment can create webhooks like this, and have channels they can post to.

[dchaley]

Hi @bnovotny , are you interested in notifications to a Teams channel when jobs complete?

How does your qupath process currently understand that a job is done? (with or without deepcell, for that matter)

[bnovotny]

Hi @dchaley! I think a notification would be nice, and it would be great to have the output URI. Ideally we would get a notification only when the last job completes or when a job fails.

Currently the segmentation and qupath project creation are separate commands, and we just wait for them to finish. It would be great to be able to automatically kick off the QuPath project creation when the last segmentation mask is finished. Do you think that's a possibility with the way things are set up (each image running in a separate job)? Thanks!

[dchaley]

Thanks @bnovotny – ok cool! We can definitely add a notification when the last step of the job completes (or fails). Although, when you say "the last job" do you mean the last in a batch? Or do you mean the last step of a multistep job. When you say "last segmentation mask" I get the impression you're segmenting a series of images within a dataset, is that right?

I don't really understand the Qupath stuff: what does it mean to "kick off project creation"? At a high level, we can do anything that involves writing to cloud storage, and/or calling a webhook, and/or running a script, and/or possibly other mechanisms. Is this something easy to spec out here and/or do you want to join a pairing to go over it?

[bnovotny]

Just to preface: I totally understand if this isn't possible!

I do mean the last job in a batch, when segmenting a series of images within a dataset.

Kicking off project creation would mean running the groovy script that points to the OMETIFF and SEGMASK folders to create a QuPath "project". This script, to be precise: createNewProject.groovy. This could be run with the QuPath CLI.

I would be happy to go over this in a call. It's kind of hard to explain in short messages; sorry about any confusion!

[dchaley]

Great thanks @bnovotny – Weihao and I took a look at the script, it should be "possible" to do "something" 😅 but we need to understand what this is doing better. Added you to our Monday session– does that time work for you?

One big takeaway we have beyond the QuPath project script is the notion of a "batch" or "project" isn't something we've been using. Is it as simple as a list of input images … is it a dataset location where we run segmentation on every/some image … etc.

Looking forward to it!

[bnovotny]

Thanks for the invitation, that works! I have a public dataset/project that I'm hoping will help demonstrate. See you then and have a great weekend!

[dchaley]

I'm going to close this out. We have #283 to add the Teams notification for completion, which is blocked on #294 and/or #292 .