Webhook template for gotify

[1RandomDev]

Is there already a webhook template for Gotify? If not I'd like to create my own one but I just can't find any information about what the file specified in webhook.template_path has to contain.

[davidnewhall]

The template stuff doesn't have any documentation. There are five working templates right here: https://github.com/davidnewhall/unpackerr/blob/master/pkg/unpackerr/webhooks_templates.go#L48-L221

Copy the data between the ticks (``) into a file and you now have a template file. Adjust it to your needs. When you get it working, lets discuss more. :) if you need help, lemme know!

Use the -w cli arg to test your changes. unpackerr -w 2 for instance will fire a test webhook.

[krystiancharubin]

I tried making a quick template, but seems like gotify does not like multiline strings, or they are not getting generated correctly. Tried using the text attribute from Telegram template for the message but get the same error.

2022/04/23 01:12:37.896993 webhook.go:119: [DEBUG] Webhook Payload: {
  "priority": 2,
  "title": "Unpackerr",
  "message": "Hello
  World",
  "extras": {
    "client::display": {
      "contentType": "text/markdown"
    }
  }
}
2022/04/23 01:12:37.897049 webhook.go:120: [ERROR] Webhook (/this/is/a/path = extracting): Gotify: invalid HTTP status reply (400 Bad Request): {"error":"Bad Request","errorCode":400,"errorDescription":"invalid character '\\n' in string literal"}

[davidnewhall]

Share your template so I can help provide ideas. Might just need to use \n like this: https://github.com/davidnewhall/unpackerr/blob/master/pkg/unpackerr/webhooks_templates.go#L181

[krystiancharubin]

Yeah using Hello\nWorld works just harder to make a more complex template since it all will have to be on one line.

[davidnewhall]

Got any experience with Go? You could probably make it replace newlines with \n :)

[davidnewhall]

What you're experiencing is a JSON problem. Not anything specific to my code or this app. If you wrote a JSON blob and fed it into curl you'd have to do the same thing. EDIT: as mentioned in the previous comment, the app could "help" by "fixing" the JSON if it has newlines in it...

[krystiancharubin]

I've dabbled with go before, and yeah you are absolutely right that its a JSON problem. Just trying to see if there is an easy way to get around this with Go. Gotify allows sending notifications using url parameters, how would I make a template to generate url params like WebhookTemplatePushover. Do I just need to set content_type to something else.

Edit: I looked though the code and found you are using application/x-www-form-urlencoded for Pushover, got it to work with Gotify.

[krystiancharubin]

The following works, based on Pushover template. Wanted to do some markdown to add styling but thats only supported with json payloads in Gotify

UN_WEBHOOK_0_URL=https://gotify.example.com/message?token=REDACTED

title={{nickname}}: {{formencode .Event.Desc}}&message=App: {{.App}}
Name: {{formencode (index .IDs "title")}}
Path: {{formencode .Path}}
{{ if .Data -}}
{{ if .Data.Elapsed.Duration}}Time: {{.Data.Elapsed}}
{{end}}{{ if .Data.Archives}}RARs: {{len .Data.Archives}}
{{end}}{{ if .Data.Files}}Files: {{len .Data.Files}}
{{end}}{{ if .Data.Bytes}}Bytes: {{humanbytes .Data.Bytes}}
{{end}}{{ if and (gt .Event 1) (lt .Event 5)}}Queue: {{.Data.Queue}}
{{end}}{{ if .Data.Error}}
ERROR: {{formencode .Data.Error}}
{{end}}{{end -}}

[davidnewhall]

Lookin good. Can ya share a screenshot?

[krystiancharubin]

[krystiancharubin]

In the Telegram template the text attribute is defined over multiple lines, is that generating invalid JSON or does Go do something special since the template is defined inside the file? https://github.com/davidnewhall/unpackerr/blob/280a04fe716b850084f65a3a3708c6856e4f5eb1/pkg/unpackerr/webhooks_templates.go#L75-L90

[davidnewhall]

In the Telegram template the text attribute is defined over multiple line

the - at the end (or beginning) of a template item {{ }} removes all trailing (or leading) spaces and newlines. You can use this to your advantage to cut up long strings. EDIT: Read more about it here: https://pkg.go.dev/text/template

[krystiancharubin]

<b><a href=\"https://github.com/davidnewhall/unpackerr/releases\">Unpackerr<a>: {{.Event.Desc}}</b> 
     \n<b>Title</b>: {{rawencode (index .IDs "title") -}} 
     \n<b>App</b>: {{.App -}} 
     \n\n<b>Path</b>: <code>{{rawencode .Path}}</code>

This bit is not within {{ }} and is split across multiple lines, with explicit \n

[davidnewhall]

Good question. It's producing invalid JSON. I'll have to test this more, but I suspect telegram is "Accepting and fixing" it.

2022/04/23 23:19:10.681473 webhook.go:123: [DEBUG] Webhook Payload: {
  "chat_id": "Unpackerr",
  "parse_mode": "HTML",
  "text": "<b><a href=\"https://github.com/davidnewhall/unpackerr/releases\">Unpackerr<a>: Extracting</b>
    \n<b>Title</b>: Some Cool Title Name Here\n<b>App</b>: Starr\n\n<b>Path</b>: <code>/this/is/a/path</code>
  \n
    \n <b>Archives</b>: 2\n <b>Queue</b>: 0"}

[davidnewhall]

Are you good with your template at this point? Would you like me to bake it into the app?

[krystiancharubin]

Yeah template is as good as its going to get. Would be great if you can bake it in.

[davidnewhall]

kk, will do! I'm using what you put into this comment, so if you need to change it, edit that comment. You'll get to see and comment on the pull request before I merge it in. https://github.com/davidnewhall/unpackerr/issues/192#issuecomment-1107408255

Thank you for this!

[krystiancharubin]

Sounds good, have you considered using something like https://github.com/containrrr/shoutrrr to streamline the notifications?

[davidnewhall]

I have not, but something like is probably not a bad idea. I didn't expect these notifications to be so in demand when I wrote them. The original intent was for notifiarr/discord, and I added a few more as folks requested. it would be nice if it were a bit more autonomous. Will look into this more!

[davidnewhall]

@krystiancharubin If you're using docker, can you try this tag? golift/unpackerr:unstable The gotify template is baked in; see this commit (this commit also has some unrelated cleanup in the example config file). If you're not using Docker, let me know which OS and I'll drop a binary in here to try out. I also re-read your messages above and if I'm understanding correctly, you could not get the JSON template to work? If you have or can point me to a working payload, I can probably templatize it. Wouldn't mind spending some time on this (with you) to make it look excellent. It will probably be a while before I switch my template stuff to a library (if ever), so we can definitely make this look good before we finalize it.

[krystiancharubin]

Seems like wrong template is picked up, tried forcing the gotify template using the new parameter and still same thing.

[[webhook]]
 url    = "https://gotify.example.com/message?token= REDACTED"
 name   = "Gotify"    # Set this to hide the URL in logs.
 silent = false # do not log success (less log spam)
 events = [0]   # list of event ids to include, 0 == all.
# Advanced Optional Webhook Configuration
 nickname      = "Unpackerr"    # Used in Discord and Slack templates as bot name, in Telegram as chat_id.
 channel       = ""    # Also passed into templates. Used in Slack templates for destination channel.
 exclude       = []    # list of apps to exclude, ie. ["radarr", "lidarr"]
#  template_path = "/config/template/gotify.go.tmpl"    # Override internal webhook template for discord.com or other hooks.
 template = "gotify" # Override automatic template detection. Values: notifiarr, discord, telegram, gotify, pushover, slack
 ignore_ssl    = false # Set this to true to ignore the SSL certificate on the server.
 timeout       = "10s" # You can adjust how long to wait for a server response.
#  content_type  = "application/x-www-form-urlencoded" # If your custom template uses another MIME type, set this.

2022/05/01 11:16:50.366096 webhook.go:237:  => Webhook Config: 1 URL: Gotify, timeout: 10s, ignore ssl: false, silent: false, events: all, channel: , nickname: Unpackerr
2022/05/01 11:16:50.366171 cmdhook.go:124:  => Command Hook Configs: 0 cmds
2022/05/01 11:16:50.366260 webhook_samples.go:14: Sending sample webhooks and exiting! (-w 1 passed)
2022/05/01 11:16:50.541414 webhook.go:126: [DEBUG] Webhook Payload: {
  "path": "/this/is/a/path",
  "app": "Folder",
  "ids": {
    "downloadId": "some-id-goes-here-1651418210",
"otherId": "another-id-here-like-imdb",
"title": "Some Cool Title Name Here"
  },
  "unpackerr_eventtype": "queued",
  "time": "2022-05-01 11:16:50.366352742 -0400 EDT m=+0.014549907",
    "go_version": "go1.18.1",
  "os": "linux",
  "arch": "amd64",
  "version": "0.10.0b",
  "revision": "421",
  "branch": "dn2_updates2 (9859bdc)",
  "started": "2022-05-01 11:16:50.353099123 -0400 EDT m=+0.001296340"
}
2022/05/01 11:16:50.541482 webhook.go:127: [ERROR] Webhook (/this/is/a/path = queued): Gotify: invalid HTTP status reply (400 Bad Request): {"error":"Bad Request","errorCode":400,"errorDescription":"Field 'message' is required"}
2022/05/01 11:16:50.541510 webhook.go:128: [DEBUG] Webhook Response: 

[krystiancharubin]

I also re-read your messages above and if I'm understanding correctly, you could not get the JSON template to work? If you have or can point me to a working payload, I can probably templatize it. Wouldn't mind spending some time on this (with you) to make it look excellent. It will probably be a while before I switch my template stuff to a library (if ever), so we can definitely make this look good before we finalize it.

I was thinking of using a few features gotify provides in the json api. https://gotify.net/docs/msgextras We can also add a click url and an image to the notification and potentially some styling with markdown

{
    "title": "Unpackerr: Imported",
    "message": "*App:* Starr   \nName: Some Cool Title Name Here   \nPath: /this/is/a/path   \nTime: 12.981776ms   \nRARs: 2",
    "extras": {
        "client::display": {
            "contentType": "text/markdown"
        },
        "client::notification": {
            "click": {
                "url": "https://www.imdb.com/title/tt1160419"
            },
            "bigImageUrl": "https://m.media-amazon.com/images/M/MV5BN2FjNmEyNWMtYzM0ZS00NjIyLTg5YzYtYThlMGVjNzE1OGViXkEyXkFqcGdeQXVyMTkxNjUyNQ@@._V1_FMjpg_UX509_.jpg"
        }
    }
}

[davidnewhall]

Thanks for the info! Will do some more debugging on my end to figure out why you're getting the wrong template. I'll also build a second template to try. Might be later this evening as I'm heading to the in-laws today.

[davidnewhall]

I tested this locally and that template works. All I can assume is that I forgot to docker push and I never put the new changes into docker.

Davids-iMac:unpackerr david$ docker push golift/unpackerr:unstable
The push refers to repository [docker.io/golift/unpackerr]
b20ab3395eba: Layer already exists
2a1db186d965: Layer already exists
1841c82f7432: Layer already exists
unstable: digest: sha256:88f39d7fb98c402b920a3b3864488c777d69f3d998d9892ff59c849ed389f965 size: 948

They're there now. golift/unpackerr:unstable

[davidnewhall]

Can you try this template? (put it in a file and pass template_path)

{
  "title": "{{if nickname}}{{nickname}}{{else}}Unpackerr{{end}}: {{.Event.Desc}}",
  "message": "*App*: {{.App -}}
    \n*Name*: {{rawencode (index .IDs "title") -}}
    \n*Path*: {{rawencode .Path -}}
    {{ if .Data.Elapsed.Duration }}\n*Elapsed*: {{.Data.Elapsed}}{{end -}}
    {{ if .Data.Archives }}\n*RARs*: {{len .Data.Archives}}{{end -}}
    {{ if .Data.Files }}\n*Files*: {{len .Data.Files}}{{end -}}
    {{ if .Data.Bytes }}\n*Bytes*: {{humanbytes .Data.Bytes}}{{end -}}
    {{ if and (gt .Event 1) (lt .Event 5) }}\n*Queue*: {{.Data.Queue}}{{end -}}
    {{ if .Data.Error}}\n\n*ERROR*:\n` + "```" + `\n{{rawencode .Data.Error}}\n` + "```" + `{{end}}",
  "extras": {
    "client::display": {
      "contentType": "text/markdown"
    },
    "client::notification": {
      "click": {
          "url": ""
      },
      "bigImageUrl": ""
    }
  }
}

Is there a way to change the gopher icon on the message? If not, I may try to embed an unpackerr logo somewhere else. If you can try this template and show me a screenshot, it may give me ideas. I'm not real sure what would go well as a large image or a clickable url. Other template just link back to the repo, but it's mostly in the footer, or somewhere unobtrusive. Unfortunately, we don't have any "real data" about the item being extracted - no links or imbd/tvdb IDs.

[krystiancharubin]

Yeah in Gotify you configure apps and can set an icon. Since we are interacting with Starr do we have ids to those media? Tried the JSON template but getting some errors.

2022/05/02 10:36:44.332772 webhook.go:230:  => Webhook Config: 1 URL: Gotify, timeout: 10s, ignore ssl: false, silent: false, template: /config/template/gotify.go.tmpl, content_type: application/json, events: all, channel: , nickname: Unpackerr
2022/05/02 10:36:44.332814 cmdhook.go:124:  => Command Hook Configs: 0 cmds
2022/05/02 10:36:44.332875 webhook_samples.go:14: Sending sample webhooks and exiting! (-w 7 passed)
2022/05/02 10:36:45.231866 webhook.go:119: [DEBUG] Webhook Payload: {
  "title": "Unpackerr: Delete Failed",
  "message": "*App*: Starr\n*Name*: Some Cool Title Name Here\n*Path*: /this/is/a/path\n*RARs*: 2\n*Files*: 2\n\n*ERROR*:\n` + "```" + `\nunable to delete files\n` + "```" + `",
  "extras": {
    "client::display": {
      "contentType": "text/markdown"
    },
    "client::notification": {
      "click": {
          "url": ""
      },
      "bigImageUrl": ""
    }
  }
}
2022/05/02 10:36:45.232173 webhook.go:120: [ERROR] Webhook (/this/is/a/path = deletefailed): Gotify: invalid HTTP status reply (400 Bad Request): {"error":"Bad Request","errorCode":400,"errorDescription":"invalid character '`' after object key:value pair"}
2022/05/02 10:36:45.232201 webhook.go:121: [DEBUG] Webhook Response: 

[davidnewhall]

Ahh, maybe those ticks don't work in json very well either. Change this line

    {{ if .Data.Error}}\n\n*ERROR*:\n` + "```" + `\n{{rawencode .Data.Error}}\n` + "```" + `{{end}}",

to

    {{ if .Data.Error}}\n\n*ERROR*:\n{{rawencode .Data.Error}}\n{{end}}",

---

Since we are interacting with Starr do we have ids to those media?

Those IDs are not available because nothing in unpackerr uses them. They would be "different" for each of the four starr apps, and you'd have to account for that if you wanted to turn them into links, etc. And the Folder feature doesn't interact with the starr apps at all, so that data wouldn't be anywhere.

[krystiancharubin]

Got it to look good on web, but on android the spacing is different now. Also since we don't have data for the additional feature maybe we go back to the form data template.

{
  "title": "{{if nickname}}{{nickname}}{{else}}Unpackerr{{end}}: {{.Event.Desc}}",
  "message": "**App**: {{.App -}}
    \n\n**Name**: {{rawencode (index .IDs "title") -}}
    \n\n**Path**: {{rawencode .Path -}}
    {{ if .Data.Elapsed.Duration }}\n\n**Elapsed**: {{.Data.Elapsed}}{{end -}}
    {{ if .Data.Archives }}\n\n**RARs**: {{len .Data.Archives}}{{end -}}
    {{ if .Data.Files }}\n\n**Files**: {{len .Data.Files}}{{end -}}
    {{ if .Data.Bytes }}\n\n**Bytes**: {{humanbytes .Data.Bytes}}{{end -}}
    {{ if and (gt .Event 1) (lt .Event 5) }}\n\n**Queue**: {{.Data.Queue}}{{end -}}
    {{ if .Data.Error}}\n\n**ERROR**: {{rawencode .Data.Error}}{{end}}",
  "extras": {
    "client::display": {
      "contentType": "text/markdown"
    },
  "client::notification": {
      "click": {
          "url": "https://github.com/davidnewhall/unpackerr"
      },
      "bigImageUrl": ""
    }
  }
}

[davidnewhall]

Did you have to remove the empty "url" and "bigImageUrl" keys to make this work? I was going to try to leave them there in case we find something to put into them later. It looks like you replaced the single \n with double \n. Android is recognizing the double newline, but web is not; that's interesting. I wonder if you have any other payload example that look correct on both?

I'd prefer to use json+markdown because we can bold things, and I can add a link back to the unpackerr repo (I'll probably make the title a link).

EDIT: presumably, this is why they render differently:

[krystiancharubin]

No you can leave those in even if they are blank. By default clicking on a notification opens Gotify on Andoird. If url is empty nothing happens. Similarly with bigImageUrl you can display an image when notification is expanded, this too can be left empty. I was looking though the Gotify documentation and web/android use different flavors of markdown. Normally you should be able to add multiple spaces at the end of the line to break into a new one. Seems, problem is likely that go templates remove all the extra spaces.

[davidnewhall]

I don't think we have any extra spaces at the end of the lines? We can try adding them. I'm probably going to install gotify now so I can mess with this and figure it out. I don't have an android though, so not sure what I'll do there. I'll probably break this down into a couple curl commands for you to try so we can make this look right on both platforms.

[davidnewhall]

@krystiancharubin Can you tell me how this looks on Android? I'm not sure I can test that.

{
  "title": "{{if nickname}}{{nickname}}{{else}}Unpackerr{{end}}: {{.Event.Desc}}",
  "message": "**App**: {{.App}}  \n**Name**: {{rawencode (index .IDs "title")}}  \n**Path**: {{rawencode .Path -}}
    {{ if .Data.Elapsed.Duration }}  \n**Elapsed**: {{.Data.Elapsed}}{{end -}}
    {{ if .Data.Archives }}  \n**RARs**: {{len .Data.Archives}}{{end -}}
    {{ if .Data.Files }}  \n**Files**: {{len .Data.Files}}{{end -}}
    {{ if .Data.Bytes }}  \n**Bytes**: {{humanbytes .Data.Bytes}}{{end -}}
    {{ if and (gt .Event 1) (lt .Event 5) }}  \n**Queue**: {{.Data.Queue}}{{end -}}
    {{ if .Data.Error}}  \n**ERROR**: \n~~~\n{{rawencode .Data.Error}}\n~~~{{end}}",
  "extras": {
    "client::display": {
      "contentType": "text/markdown"
    },
  "client::notification": {
      "click": {
          "url": ""
      },
      "bigImageUrl": ""
    }
  }
}

Found the "double space" solution here: https://github.com/gotify/android/issues/65

[krystiancharubin]

Notification

App

[davidnewhall]

Nice. I can possibly get rid of one of the newlines in there before the error.

[davidnewhall]

Try changing this line

    {{ if .Data.Error}}  \n**ERROR**: \n~~~\n{{rawencode .Data.Error}}\n~~~{{end}}",

to

    {{ if .Data.Error}}  \n**ERROR**: ~~~\n{{rawencode .Data.Error}}\n~~~{{end}}",

Just removing one \n. Not sure where else that extra line would come from, and I don't have high hopes that's it..

[davidnewhall]

nope, that does not work. This is probably as good as it gets. You okay with it?

[krystiancharubin]

Yeah I think it's fine as is.