What did you do?
I wrote my Pushover user key and app token to files using vim (which adds a newline to the end of each file by default) and set user_key_file and token_file in alertmanager.yml to point to those files.
What did you expect to see?
Alertmanager should have successfully sent alerts to Pushover, with a notification appearing on my phone.
What did you see instead? Under which circumstances?
When Alertmanager tried sending a notification to Pushover, Pushover returned an HTTP 400 error from its API and said that my application token was invalid.
Workaround
Either write the secrets to each file without any newlines using echo -n secret > secret_file.txt or after writing the files with newlines strip the newline character from each file using truncate --size=-1 secret_file.txt.
Environment
System information:
Linux 6.5.6-4k-00001-g6e5115d7f640 ppc64le
Alertmanager version:
alertmanager, version 0.26.0 (branch: HEAD, revision: d7b4f0c7322e7151d6e3b1e31cbc15361e295d8d)
build user: root@9623f4d745ac
build date: 20230824-11:10:34
go version: go1.20.7
platform: linux/ppc64le
tags: netgo
Prometheus version:
prometheus, version 2.48.1 (branch: HEAD, revision: 63894216648f0d6be310c9d16fb48293c45c9310)
build user: root@bd93f1a7309d
build date: 20231208-23:33:20
go version: go1.21.5
platform: linux/ppc64le
tags: netgo,builtinassets,stringlabels
ts=2023-12-18T04:35:54.049Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="pushover/pushover[0]: notify retry canceled due to unrecoverable error after 1 attempts: unexpected status code 400: {\"token\":\"invalid\",\"errors\":[\"application token is invalid, see https://pushover.net/api\"],\"status\":0,\"request\":\"9eaadfe6-562e-4eae-9ed9-979374db4c11\"}"
I had the same issue with version="(version=0.26.0, branch=HEAD, revision=d7b4f0c7322e7151d6e3b1e31cbc15361e295d8d)". The workaround mentioned by @cyrozap works but ugh.
What did you do? I wrote my Pushover user key and app token to files using vim (which adds a newline to the end of each file by default) and set
user_key_file
andtoken_file
inalertmanager.yml
to point to those files.What did you expect to see? Alertmanager should have successfully sent alerts to Pushover, with a notification appearing on my phone.
What did you see instead? Under which circumstances? When Alertmanager tried sending a notification to Pushover, Pushover returned an HTTP 400 error from its API and said that my application token was invalid.
Workaround Either write the secrets to each file without any newlines using
echo -n secret > secret_file.txt
or after writing the files with newlines strip the newline character from each file usingtruncate --size=-1 secret_file.txt
.Environment
Alertmanager configuration file:
Logs: