Can't send messages from Home Assistant after linking device

[TheGroundZero]

I first had issues with my phone number being locked by a pin in Signal (#22).

I managed to use signal-cli link -n HomeAssistant to get the data for a QR code, which I converted and scanned with my mobile phone's Signal app. (see)
signal-cli -u +<phone> receive now works and I can send messages via signal-cli -u +<phone> send -m "<message>" <recipient>

I don't seem to need to verify my number anymore

root@abcdef-signal-messenger:/# signal-cli -u +<phone> verify -p <pin> <verification>
User registration is already verified

However, I can't send notifications from Home Assistant

notify:
  - name: Signal_me
    platform: signal_messenger
    url: "http://192.168.10.105:8080" # service not reachable at 127.0.0.1?
    number: "+<phone>"
    recipients:
      - "+<phone>"

Developer tools:

Service: notify.signal_me
Data: message: "Lorum ipsum"

Add-on logs

[GIN] 2020/10/27 - 16:37:35 | 200 |      55.424µs |  192.168.10.105 | GET      "/v1/about"
[GIN] 2020/10/27 - 16:37:39 | 400 |  4.665510344s |  192.168.10.105 | POST     "/v2/send"

HA logs

Logger: homeassistant.components.websocket_api.http.connection.139716746907808
Source: components/signal_messenger/notify.py:76
Integration: Home Assistant WebSocket API (documentation, issues)
First occurred: 15:23:18 (6 occurrences)
Last logged: 15:37:39
Failed to get sender certificate: org.whispersystems.signalservice.api.push.exceptions.AuthorizationFailedException: Authorization failed! Failed to send message: Authorization failed!

Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/components/websocket_api/commands.py", line 138, in handle_call_service
    await hass.services.async_call(
  File "/usr/src/homeassistant/homeassistant/core.py", line 1335, in async_call
    task.result()
  File "/usr/src/homeassistant/homeassistant/core.py", line 1370, in _execute_service
    await handler.func(service_call)
  File "/usr/src/homeassistant/homeassistant/components/notify/__init__.py", line 138, in _async_notify_message_service
    await self.async_send_message(**kwargs)
  File "/usr/src/homeassistant/homeassistant/components/notify/__init__.py", line 117, in async_send_message
    await self.hass.async_add_job(partial(self.send_message, message, **kwargs))  # type: ignore
  File "/usr/local/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/src/homeassistant/homeassistant/components/signal_messenger/notify.py", line 79, in send_message
    raise ex
  File "/usr/src/homeassistant/homeassistant/components/signal_messenger/notify.py", line 76, in send_message
    self._signal_cli_rest_api.send_message(message, self._recp_nrs, filenames)
  File "/usr/local/lib/python3.8/site-packages/pysignalclirestapi/api.py", line 83, in send_message
    raise exc
  File "/usr/local/lib/python3.8/site-packages/pysignalclirestapi/api.py", line 79, in send_message
    raise SignalCliRestApiError(json_resp["error"])
pysignalclirestapi.api.SignalCliRestApiError: Failed to get sender certificate: org.whispersystems.signalservice.api.push.exceptions.AuthorizationFailedException: Authorization failed!
Failed to send message: Authorization failed!

Logger: homeassistant.components.signal_messenger.notify
Source: components/signal_messenger/notify.py:78
Integration: signal_messenger (documentation, issues)
First occurred: 15:23:18 (6 occurrences)
Last logged: 15:37:39
Failed to get sender certificate: org.whispersystems.signalservice.api.push.exceptions.AuthorizationFailedException: Authorization failed! Failed to send message: Authorization failed! 

[bbernhard]

Could you maybe try to send a message via curl instead of Home Assistant (to rule out any issues with Home Assistant).

It should look like this:

curl -X POST -H "Content-Type: application/json" -d '{"message": "Hello World!", "number": "<yourRegisteredSignalNumber>", "recipients": ["<recipient>"]}' 'http://192.168.10.105:8080'

signal-cli -u +<phone> receive now works and I can send messages via signal-cli -u +<phone> send -m "<message>" <recipient>

Are you sure that signal-cli uses the signal config file at the correct location? Could you maybe run the command again with --config /home/.local/share/signal-cli/, just to make sure that the signal-cli process picks up the correct signal config file. i.e:

signal-cli --config /home/.local/share/signal-cli/ -u +<phone> send -m "<message>" <recipient>.

It would be great, if you could execute the above command from within the running docker container.

[TheGroundZero]

Shouldn't there be an API endpoint?

curl -X POST -H "Content-Type: application/json" -d '{"message": "Hello World!", "number": "<phone>", "recipients": ["<phone>"]}' 'http://192.168.10.105:8080'
404 page not found

Are you sure that signal-cli uses the signal config file at the correct location?

Can I copy the config created by linking my device to where the Signal add-on will read it?

Could you maybe run the command again with --config /home/.local/share/signal-cli/, just to make sure that the signal-cli process picks up the correct signal config file.

# Open terminal to host
Login: root
Password:

# Move to Docker Supervisor?
ha > login

# docker ps
# docker exec -it 1315902c /bin/bash
root@1315902c-signal-messenger:/# signal-cli --config /home/.local/share/signal-cli/ -u "<phone>" send -m "Hello world!" "<phone>"
Failed to get sender certificate: org.whispersystems.signalservice.api.push.exceptions.AuthorizationFailedException: Authorization failed!
Failed to send message: Authorization failed!

:(

[bbernhard]

Shouldn't there be an API endpoint?

oh, sorry, I accidentally posted the wrong API endpoint. It should be this one:

curl -X POST -H "Content-Type: application/json" -d '{"message": "Hello World!", "number": "<yourRegisteredSignalNumber>", "recipients": ["<recipient>"]}' 'http://192.168.10.105:8080/v2/send'

But you will most probably see the same error here, as the one you get with signal-cli (as the API is just a simple wrapper around the commandline tool).

Can I copy the config created by linking my device to where the Signal add-on will read it?

In the first post you've written that receive and send outside of the docker container works, right? So I guess there's signal config somewhere on your host system that works. If you can find the location of that signal config file you can copy it to the right folder, so that docker container container picks it up. If you are using the sample docker-compose.yml file

version: "3"
services:
  signal-cli-rest-api:
    image: bbernhard/signal-cli-rest-api:latest
    ports:
      - "8080:8080" #map docker port 8080 to host port 8080.
    volumes:
      - "./signal-cli-config:/home/.local/share/signal-cli" #map "signal-cli-config" folder on host system into docker container. the folder contains the password and cryptographic keys when a new number is registered

there should be a signal-cli-config folder in the same folder where the docker-compose.yml file is stored. You need to copy your signal config into that folder. So it should look like that:

signal-cli-config
  data
    +4923312121
    +4923312121.d

The signal-cli's config is stored inside the data folder, which itself is inside the signal-cli-config folder.

In case you can't find the signal-cli config on the host system, you could do the linking again in your docker container.

Just connect again with docker exec -it abcdef /bin/bash to your docker container and run the "link" command again. But make sure to always use the --config /home/.local/share/signal-cli/ parameter for every command you are executing, to make sure that signal-cli persists the changes to the correct config file.

[TheGroundZero]

I ran all commands from within the Signal-Messenger Docker container.
Installed this via the Home Assistant add-on (https://github.com/haberda/hassio_addons/tree/master/signal), so I have no control over the docker-compose.yml.

EDIT: realized I have 2 phone numbers on this phone and that Signal is configured on the 2nd number.
Send via CLI now works, but REST API still fails ...

Retried using link by passing the --config parameter.

$ signal-cli --config /home/.local/share/signal-cli/ link -n HomeAssistant
tsdevice:/?uuid=xxxxx&pub_key=xxxxxxxxxx
# Copy string above and convert to QR code at https://www.the-qrcode-generator.com/
# Scan QR-code with Signal app "Linked devices"
Link request error: null
$ signal-cli --config /home/.local/share/signal-cli/ -u +<my no.> send -m "TEST" +<my no.>
1604346018945
# Message appears in the "Notes to self" chat

Via REST API

$ curl -X POST -H "Content-Type: application/json" -d '{"message": "Hello World!", "number": "+<my no.>", "recipients": ["+<my no.>"]}' 'http://192.168.10.105:8080/v2/send'
{"error":"Failed to get sender certificate: org.whispersystems.signalservice.api.push.exceptions.AuthorizationFailedException: Authorization failed!\nFailed to send message: Authorization failed!\n"}

Testing receive via CLI

$ signal-cli --config /home/.local/share/signal-cli/ -u +<my no.> receive
Envelope from: +<my no.> (device: 1)
Timestamp: <current time>
Sender: +<my no.> (device: 1)
Received a sync message
Received sync message with block list
Blocked numbers:

[...]

[bbernhard]

EDIT: realized I have 2 phone numbers on this phone and that Signal is configured on the 2nd number. Send via CLI now works, but REST API still fails ...

That's really strange. The REST API wrapper basically does exactly the same. Just to be sure: You are using the same number in the REST API call, as with the commandline tool, right? And you executed the signal-cli command within the docker container, right? (just to rule out that it was invoked directly on the host system where maybe a different signal-cli config is stored at /home/.local/share/signal-cli)

[TheGroundZero]

EDIT: realized I have 2 phone numbers on this phone and that Signal is configured on the 2nd number. Send via CLI now works, but REST API still fails ...

That's really strange. The REST API wrapper basically does exactly the same. Just to be sure: You are using the same number in the REST API call, as with the commandline tool, right? And you executed the signal-cli command within the docker container, right? (just to rule out that it was invoked directly on the host system where maybe a different signal-cli config is stored at /home/.local/share/signal-cli)

I did run each command again using the correct phone no.

I'm fairly sure I'm working from within the container.
I have no experience in Docker containers, so I'm learning as I go :) That's why I added the commands I executed, to verify that I actually am within the container.

This is opening a shell within the container, correct?

# docker ps
# docker exec -it [container_id] /bin/bash

Is the wrapper using a certain token that is not set or set differently when using link instead of register?

[bbernhard]

This is opening a shell within the container, correct?

yep, that's correct :)

Is the wrapper using a certain token that is not set or set differently when using link instead of register?

not really, that should be handled entirely by signal-cli. At the moment I do not have a clue why it works from within the docker container by manually invoking signal-cli but not with the wrapper.

I've found an issue in the signal-cli repository, which sounds similar to yours (https://github.com/AsamK/signal-cli/issues/122). But in your case it does work from the commandline. So I guess that's a red herring then.

There's one more thing I can think of that you could try:

• clone this repository with git clone https://github.com/bbernhard/signal-cli-rest-api.git
• above this line here, add the following line log.Info("cmd = ", cmd) to log which command arguments are passed to signal-cli
• adapt your docker-compose.yml file and replace image: bbernhard/signal-cli-rest-api:latest with build: "." to build the image locally instead of pulling from dockerhub.
• run docker-compose build to build the image and docker-compose down followed by docker-compose up to start the docker container in the foreground.

When you now execute the curl send request again, you should see the arguments that are passed to signal-cli

e.g something like that:

cmd = [--config /home/.local/share/signal-cli/ -u +43yyyyyy send -m Hello World! +43xxxxx]"

Maybe that gives us a hint what's wrong in your setup.

[TheGroundZero]

There's one more thing I can think of that you could try:

* clone this repository with `git clone https://github.com/bbernhard/signal-cli-rest-api.git`

* above this line [here](https://github.com/bbernhard/signal-cli-rest-api/blob/master/src/api/api.go#L166), add the following line `log.Info("cmd = ", cmd)` to log which command arguments are passed to `signal-cli`

* adapt your `docker-compose.yml` file and replace `image: bbernhard/signal-cli-rest-api:latest` with `build: "."` to build the image locally instead of pulling from dockerhub.

* run `docker-compose build` to build the image and `docker-compose down` followed by `docker-compose up` to start the docker container in the foreground.

When you now execute the curl send request again, you should see the arguments that are passed to signal-cli

e.g something like that:

cmd = [--config /home/.local/share/signal-cli/ -u +43yyyyyy send -m Hello World! +43xxxxx]"

Maybe that gives us a hint what's wrong in your setup.

I'm still quite confused on working with Docker and the setup of Hassio. I'm running a Hassio/HomeAssistant VM within Proxmox. I assume this VM is basically a light OS with Docker installed? This Docker instance then has a container for Hassio. Does the signal-cli run within a container on the same level, or is this another Docker instance running within the Hassio-container?

I'm opening the console and performing the steps via the Proxmox console viewer, since SSH'ing to Hassio brings me inside the Hassio container(?).

Do I clone the repo inside the signal-container, within Hassio, at the host, ...?

[nikkolade]

I believe I just had this issue myself. For some reason my account that was working in the past had gone to dysfunctional. Therefore I now linked the docker image to my phone and tested message sending from the docker using the signal-cli command directly and that started to work. However, the http api didn't work initially at all and just returned the following error: {"error":"Authorization failed, was the number registered elsewhere?\n"} After comparing the signal-cli config file to the signal-cli-rest config file, I just copied my updated config file from "/root/.local/share/signal-cli/data/ into "/home/.local/share/signal-cli/data" and everything started working just fine.

[bbernhard]

@nikkolade thanks for sharing your experience! As I do not use signal-cli with linked devices I do not have much experience with it. So any help/tips/success stories from other users are really appreciated.

@TheGroundZero Unfortunately, I do not have much experience with HASSIO nor Proxmox. :/

If possible, I would suggest to first get the docker container running on a bare metal machine, before adding any additional virtualization layers. Ideally, the virtualization layer shouldn't make any difference, but I think it makes it easier to find the root cause of your problem by removing any additional complexity that's not absolutely needed. Once you have the docker container running, you can easily copy the signal-cli config file to your VM.

[papoms]

I just ran into a similar issue and realized that linking does not work after you already registered the number with signal-cli.

More specifically signal-cli refuses to overwrite the config file for an existing "user" e.g. Number in /signal-cli-config/data/{{number}}.

When trying to register or link an existing number you first have to manually delete the profile in /signal-cli-config/data/{{number}}.

This is the output from within the Container, trying to manually link it after already registering the number.

$ signal-cli --config /home/.local/share/signal-cli/ link -n TestDeviceName
tsdevice:/?uuid=***&pub_key=***
The user +49*********** already exists
Delete "/home/.local/share/signal-cli//data/+49*************" before trying again.

Reproduce: 1 - use /v1/register/{{ number }} to register your number 2 - verify with token using /v1/register/{{ number }}/verify/{{token}} 3 - run /v1/qrcodelink and try to link the container 4 - confirm problem by manually invoking signal-cli --config /home/.local/share/signal-cli/ link -n TestDeviceName

Top of my head Solutions:

• Delete /data/{{number}} either automatically when registering or linking or offer an endpoint to do so
• Propose a command line flag to overwrite existing profile in signal-cli see https://github.com/AsamK/signal-cli/blob/bc47c0d5d6ade54db2997f7d6ed5f7154781fcc7/src/main/java/org/asamk/signal/manager/ProvisioningManager.java#L90 Thanks for this great tool!

[bbernhard]

Many thanks @papoms for the detailed analysis. As I do not use linked devices, any help & feedback on that topic is always highly appreciated!

In general I would prefer option 2, as it sounds like the "cleaner" solution - at least in my opinion. But I guess with the recent ToS changes of Whatsapp, the maintainer of signal-cli probably will receive quite a lot of feature requests already. And I do not have the time to create a PR upstream. But option 1 is for sure doable - I think that could be added to one of the next releases :+1:

[babajun12]

Maybe a tip for some unexperienced useres like me: Had the same issue, just re-registered the phone No. But be careful with your config-file location!: Using curl -X POST -H "Content-Type: application/json" 'http://localhost:8080/v1/register/+1******'' ..stores data into /home/.local/share/signal-cli/data But if you use signal-cli from inside the container like dokcer@signal:/# signalsignal-cli -u +4***** register data is stored into root/.local/share/signal-cli/data This means if you configure signal to your phone No within the container, you will receive "authentication failed" with the API wrapper afterwards (bcs data is stored in /root/..data/ , API wrapper reads /home/..../data/) or use --config /home/.local/share/signal-cli/data option

Sorry if technicaly not correct expressioned ;-)

[bbernhard]

Had the same issue, just re-registered the phone No. But be careful with your config-file location!: Using curl -X POST -H "Content-Type: application/json" 'http://localhost:8080/v1/register/+1******'' ..stores data into /home/.local/share/signal-cli/data

If you want to invoke signal-cli directly (because you want to use/enable some functionality that's not yet exposed via API), always specify the config file location via: signal-cli --config /home/.local/share/signal-cli/. That ensures that you are working on the correct config file.