search

immanuelfodor / matrix-encrypted-webhooks

Send any JSON and form data through webhooks to Matrix end-to-end encrypted (E2EE) rooms with HTTP POST requests
MIT License
17 stars 5 forks source link

Pushing Images / Files #2

Open superchase9000 opened 3 years ago

superchase9000 commented 3 years ago

Is there any current or planned support for pushing files?

immanuelfodor commented 3 years ago

Great idea! I haven't got any use case for it yet that's why it wasn't implemented but it can be definitely done: https://matrix-nio.readthedocs.io/en/latest/examples.html#sending-an-image

The only thing need to be figured out is how to get the file and its stats from the post message without creating a temp file (if it's possible, not to spam the container's filesystem nor have the responsibility of cleanup).

immanuelfodor commented 3 years ago

Two links for future reference:

superchase9000 commented 3 years ago

@immanuelfodor Nice! Thanks for looking into this. This is wonderful project.

neilwoodwards commented 2 years ago

Fantastic project well done. Is there a way to send json strings without quotations surrounding it?

immanuelfodor commented 2 years ago

Thanks!

Could you please provide an example what would you want to achieve? My first intuition is that the JSON should be parseable by Python. Second guess is that curl can be parameterized with a data file, so the JSON could be passed this way without extra quotes.

neilwoodwards commented 2 years ago

Could you please provide an example what would you want to achieve? My first intuition is that the JSON should be parseable by Python. Second guess is that curl can be parameterized with a data file, so the JSON could be passed this way without extra quotes.

When I send curl -d "world" http://localhost:8000/post/token it returns world: '' Rather, I just want a single string with not quotations or colons like world.

Also, when I send a webhook from iOS (Shortcuts) it appears as message: coming to you from shortcuts instead of coming to you from shortcuts.

Just trying to send strings so that its just the string when it appears in element.io

My .env file has the following:

MESSAGE_FORMAT=yaml USE_MARKDOWN=False ALLOW_UNICODE=True DISPLAY_APP_NAME=False

immanuelfodor commented 2 years ago

Oh, I see what you mean by extra quotes. It's just a coincidence that it is a valid YAML according to the Python YAML parser. The built-in parser appends the extra quotes as the world key doesn't have a value, hence the world: ''. It would be interesting to see what happens if you set the message format env to raw (I can't try this right now) as you want to send in simple strings.

immanuelfodor commented 2 years ago

As for the iOS Shortcuts, it probably sends a JSON like this: 

{
    "message": "coming to you from shortcuts"
}

Or a form-data payload like this: 

message=coming to you from shortcuts

So I think this one is normal (feature, not bug), the gateway doesn't alter the payload it receives by stripping out elements (keys or values). What you want to achieve here is some custom functionality based on sender token or payload structure (to remove the message key).

neilwoodwards commented 2 years ago 
message=coming to you from shortcuts

So I think this one is normal (feature, not bug), the gateway doesn't alter the payload it receives by stripping out elements (keys or values). What you want to achieve here is some custom functionality based on sender token or payload structure (to remove the message key).

Ah ok. That explains it. Thank you. Do you have anything you can link me regarding to stripping out the JSON elements?

immanuelfodor commented 2 years ago

Oh, it's just plain old Python, you can alter the message anywhere between the payload decode and the send message function call: https://github.com/immanuelfodor/matrix-encrypted-webhooks/blob/main/src/WebhookServer.py#L55

If you'll use the gateway for only Shortcuts, you can replace the message formatting with data = data.message here, so the send function will get the value string only instead of the full string JSON dump: https://github.com/immanuelfodor/matrix-encrypted-webhooks/blob/4fda6c0011c94f2cdf865fd77c0d298deaf83ab9/src/WebhookServer.py#L78

neilwoodwards commented 1 year ago

Receives this (which is fine): 2022-11-13 10:32:35,216 | INFO | module:root | Received raw data: "Test"

But then prints this out: 2022-11-13 10:32:35,428 | INFO | module:root | @user in Room | Test\n...\n

Where is this new information coming from?

immanuelfodor commented 1 year ago

The second log line is the message sent to Matrix by the bridge received back in the bridge as a new Matrix message event. The bridge receives a webhook -> the bridge logs the payload -> the bridge sends it as a message to the configured room -> Matrix sends the message event (back) to the bridge -> the bridge logs the message event. The last two steps also happen when a regular user sends a message in the room, and it could provide a capability later to manage the bridge from the room via messages/commands.

Do you see the \n...\n part of the message in Matrix as well? So does your message appear as 

Test
...

in your Matrix client as well?

neilwoodwards commented 1 year ago

Do you see the \n...\n part of the message in Matrix as well? So does your message appear as

Test
...

in your Matrix client as well?

Yep. It comes up as that. See here: https://ibb.co/hV5GJw4

Is there a way so it's just Test?

immanuelfodor commented 1 year ago

Well, I tested it, and I can't see any extra ... around the message.

$ curl -d "world" https://example.com/post/TOKEN
{"success": true}

See the image for the Matrix output:

image

Even the logs are fine:

...
2022-12-16 11:30:26,470 | INFO | module:root | We synced, token: XXXXXXXXX_XXXXXXXXXX_XXXXXXXXXXX
2022-12-16 11:30:27,242 | INFO | module:root | We synced, token: XXXXXXXXX_XXXXXXXXXX_XXXXXXXXXXX
2022-12-16 11:30:29,194 | INFO | module:root | Received raw data: world
2022-12-16 11:30:29,217 | INFO | module:aiohttp.access | 10.42.1.40 [16/Dec/2022:11:30:29 +0000] "POST /post/TOKEN HTTP/1.1" 200 180 "-" "curl/7.74.0"
2022-12-16 11:30:29,243 | INFO | module:root | @WEBHOOK_USER in ROOM | world

The settings I used were the same as yours:

  MESSAGE_FORMAT: "raw"
  USE_MARKDOWN: "False"
  DISPLAY_APP_NAME: "False"
  ALLOW_UNICODE: "True"
neilwoodwards commented 1 year ago

Thanks @immanuelfodor

What about sending JSON webhooks? When I send:

{ "message": "Hello World" }

I receive the following:

image

The settings I am using for this is:

MESSAGE_FORMAT: "json" USE_MARKDOWN: "False" DISPLAY_APP_NAME: "False" ALLOW_UNICODE: "True"

Is there a way to only extract the Hello World?

immanuelfodor commented 1 year ago

Yes, the answer is at https://github.com/immanuelfodor/matrix-encrypted-webhooks/issues/2#issuecomment-1172304818 :smile:

immanuelfodor commented 1 year ago

Note to self:

Regarding the original idea of pushing images and files, Apprise is considering a Matrix notification channel overhaul at https://github.com/caronc/apprise/issues/795 . When they implement this, the custom E2E client written in this repo could be replaced by Apprise, just the same way we use that in the RocketChat gateway (https://github.com/immanuelfodor/rocketchat-push-gateway).

We depend on two issues of Apprise to be completed before it can be integrated: https://github.com/caronc/apprise/issues/356 for attachment support and https://github.com/caronc/apprise/issues/305 for E2E support. When this happens, the Matrix webhook bridge's API can mainly stay the same and the project could become a wrapper of Apprise to provide the necessary (containerized) REST API to accept webhooks.