Incompatible with REST API for signal (needed for compliance with WCAG & Americans with Disabilities Act)

[jaanli]

Hi! I need to use screen reading for my disability (see the ADA for more details or WCAG), and for translation.

For this I use this Signal REST API for most content:

https://github.com/bbernhard/signal-cli-rest-api

Can you please help me understand what pull request would be needed in order to fix the issue and enable local LLMs to be used for screen reading and summarization to bring this into compliance?

Thank you! Jaan

P.S. We have some resources @onefact for supporting this type of work as well, please let me know if that may be of interest - that is because @signalapp is one of the few HIPAA-compliant consumer oriented solutions we use for the hospitals and clinics we work with to share protected health information in compliance with federal law.

[tulir]

You forgot to specify what the issue is

[jaanli]

Ah sorry, the issue is that we can't use local large language models with the REST API via @beeper, but can with the virtual private servers we use via @aws (e.g. Lightsail boxes: https://aws.amazon.com/lightsail/).

Is that helpful or would you need further details?

[tulir]

I'm not sure how that's related to the bridge, but in general all Beeper issues should be reported via the in-app "Report a problem" button rather than github

[jaanli]

Ah sorry for being unclear @tulir , the bridge issue is the REST API - we cannot integrate it with https://github.com/bbernhard/signal-cli-rest-api in order to send and receive messages from large language models running on AWS.

Does that make more sense? I'm happy to send over a UML diagram or other format (Figma?) if it is easier to see our architecture.

[tulir]

The bridge works just fine with signal-cli as the primary device. When logging in via commands, the bot will send the raw linking URI (sgnl://...) in addition to the QR image, which can be passed to signal-cli's addDevice command. POST /v1/devices/{number} with uri in the body appears to be the equivalent method in that API.

If you meant something else than linking, it's probably out of scope for the bridge. The bridge is only for bridging messages between Matrix and Signal. Sending and receiving messages has to be done on Matrix or Signal via their respective APIs, not in the bridge.

[jaanli]

Ah got it, sorry I thought the bridge was for bridging messages between Matrix, Signal, and large language models.

I think I was mistaken so I will close the issue shortly - my bad!

Would a new bridge be needed to bridge messages between Matrix and large language model servers?

(Whether language models are used locally, in WebAssembly e.g. @WasmEdge - https://github.com/LlamaEdge/LlamaEdge - or in the cloud, like on @aws Lightsail)

[tulir]

LLM integrations would probably either be bots (like https://github.com/matrixgpt/matrix-chatgpt-bot) or something integrated into clients (like https://github.com/beeper/widget-summarizer)

[jaanli]

LLM integrations would probably either be bots (like https://github.com/matrixgpt/matrix-chatgpt-bot) or something integrated into clients (like https://github.com/beeper/widget-summarizer)

Got it, thank you so much!

I think indeed I got confused between the words:

• matrix I thought was a chat server that I use for E2E encryption
• mautrix is not the same thing
• Beeper is not the same thing

Somehow these are connected, and it’s very confusing for me :(

any help? If there are any diagrams (we use Figma and mermaid.live or diagrams.net for this) that show the legal entities involved?

No worries of not, think it would help my brain get less muddled about this.

[maltee1]

• Matrix is a federated messaging protocol (matrix.org). Multiple server implementations (synapse being the most popular one) and thousands of instances all communicate with each other.
• Mautrix is a brand name for bridges between matrix and other networks (signal in this case).
• Beeper is a commercial multi-protocol messaging client & service based on matrix and mautrix bridges, with some proprietary software on top.

There are plenty of resources explaining this in detail, I suggest you start with matrix (make sure not to confuse it with mathematics or the movie).

[jaanli]

• Matrix is a federated messaging protocol (matrix.org). Multiple server implementations (synapse being the most popular one) and thousands of instances all communicate with each other.
• Mautrix is a brand name for bridges between matrix and other networks (signal in this case).
• Beeper is a commercial multi-protocol messaging client & service based on matrix and mautrix bridges, with some proprietary software on top.

There are plenty of resources explaining this in detail, I suggest you start with matrix (make sure not to confuse it with mathematics or the movie).

Thanks so much! I am more of a visual learner. Is there a diagram? I saw columbiafacts.org recently (seems like a protest thing, but also seems open source), or I guess a mermaid diagram could work too.. just helps to understand why people working at beeper are also working at mautrix which is not matrix (the edit distance is once character which leads to brand conflation and brand behavioral identity diffusion sadly- I suggest a name change?)

[jaanli]

The bridge works just fine with signal-cli as the primary device. When logging in via commands, the bot will send the raw linking URI (sgnl://...) in addition to the QR image, which can be passed to signal-cli's addDevice command. POST /v1/devices/{number} with uri in the body appears to be the equivalent method in that API.

If you meant something else than linking, it's probably out of scope for the bridge. The bridge is only for bridging messages between Matrix and Signal. Sending and receiving messages has to be done on Matrix or Signal via their respective APIs, not in the bridge.

Thank you so much @tulir ! That is exactly what we need - we are building physical devices to help people create bridges via the API & bridge system you described.

We will keep you posted if we run into any issues.