vranki / hemppa

Generic modular bot for Matrix (and via it irc, telegram, slack, etc..)
GNU General Public License v3.0
154 stars 37 forks source link
hacktoberfest matrix matrix-bot

Hemppa - generic modular Matrix bot

This bot is meant to be super easy platform to write Matrix bot functionality in Python. It uses matrix-nio library https://github.com/poljar/matrix-nio/ for Matrix communications.

Zero configuration except Matrix account info is needed. Everything else can be done with bot commands.

Type !help in room with this bot running to list active modules.

If you don't want some modules, just delete the files from modules directory or disable them with !bot disable command.

End-to-end encryption is currently not supported by bot but should be doable. Bot won't respond to commands in e2ee rooms. PR for enabling e2ee is welcome!

Support room: #hemppa:hacklab.fi - https://matrix.to/#/#hemppa:hacklab.fi

Hint: RSS Bridge

RSS Bridge is awesome project that creates RSS feeds for sites that don't have them: https://github.com/RSS-Bridge/rss-bridge

If you want bot to notify on new posts on a service, check out if RSS Bridge supports it! You can use the stock Matrix RSS bot to subscribe to feeds created by RSS bridge.

Module list

Bot

Bot management commands.

The following must be done as the bot owner:

Help

Prints help on existing modules.

Alias

Add or remove aliases for a module.

Echo

Simple example module that just echoes what user said.

Metar

Aviation weather metar service access.

TAF

Aviation weather TAF service access.

NOTAM

Aviation NOTAM data access. Currently supports only Finnish airports - implement other countries where data is available.

Teamup

Can access Teamup ( https://teamup.com/ ) calendar. Teamup has nice API and is easier to set up than Google so prefer it if possible. This bot polls the calendar every 5 minutes and notifies the room of any changes.

Howto:

Commands:

Google Calendar

Can access a google calendar in a room. This is a bit pain to set up, sorry.

To set up, you'll need to generate oauth2 credentials.json file - see https://console.developers.google.com/apis/credentials

Run the bot on local machine as OAuth2 wants to open localhost url in your browser. I haven't found out an easy way to do this on server.

There is a empty credentials.json file in the bot directory. Replace it with yours. When credentials.json is present, you must authenticate the bot to access calendar. There will be a link in console like this:

Please visit this URL to authorize this application: https://accounts.google.com/o/oauth2/auth?response_type=code&client_id=907....

Open the link and authenticate as needed. A new file token.pickle will be created in the directory and bot will read it in future. Save the token.pickle and ship it with the bot to your server.

Now the bot should be usable.

Use !googlecal add [calendar id] to add new calendar to a room. The bot lists availble calendar ID's on startup and you can find them in google calendar.

Commands:

NOTE: disabled by default

Cron

Can schedule things to be done.

Commands:

Examples:

Location

Can search OpenStreetMaps for locations and send Matrix location events from them. Translates Matrix location events into OSM links.

Commands:

Example:

Room

This module is for interacting with the room that the commands are being executed on.

Note on tombstone: If using alias, bot must be present in target room. This is the preferred way. If using id, make sure it's correct, as it's not validated! Tombstoning requires power level for room upgrade. Make sure bot has it in the room.

Welcome to Room

When configured in a room, the bot will monitor a room for new users and send new users a welcome message 1:1. It will then notify bot owners of the new user. It will also, optionally, notify of user departure.

Commands:

Welcome to Server

As a server admin, the bot will monitor new user creation on the server and send the welcome message to new users 1:1. It will then notify bot owners of the new user.

Commands:

Slow polling services

These have the same usage - you can add one or more accounts to a room and bot polls the accounts. New posts are sent to room. Polls only randomly every 30 to 60 minutes to keep traffic at minimum.

Commands:

Prefix with selected service, for example "!ig add accountname" or "!teamup list"

Instagram

Polls instagram account(s). Uses instagram scraper library without any authentication or api key.

See: https://github.com/realsirjoe/instagram-scraper/

NOTE: disabled by default

Matrix Messaging API (mxma)

This is a simple API to ask bot to send messages in Matrix using JSON file from external service.

You'll need an API endpoint (webserver) that contains a message queue. It must respond with following JSON to a HTTP GET request:

{
   "messages":[
      {
         "to": "@example:matrix.org",
         "title": "Room Title",
         "message": "Hello from Hemppa"
      },
      {
         "to": "@another:matrix.user",
         "title": "Room 2 Title",
         "message": "Second message"
      }
   ]
}

Normally you want to clear the messages when the endpoint is GETted or the messages will repeat every time bot updates itself.

These messages are sent to given Matrix users in private message with given room title. Messages are sent "best effort" - if sending fails, it will be logged to bot output log.

Then just:

mxma requires all commands to be run as bot owner.

SpaceAPI

Polls the status of Hack- and Makerspaces that provide an endpoint that conforms to the SpaceAPI protocol and notifies about changes of the opening status.

To add a new endpoint simply use !spaceapi add https://hackspace.example.org/status

For Admins: A template and I18N can be configured via settings of the module. Use !bot export spacepi, then change the settings and import again with !bot import spacepi SETTINGS.

Url

Watches all messages in a room and if a url is found tries to fetch it and spit out the title if found.

Defaults to off and needs to be activated on every room you want this.

You can choose to send titles as notices (as in Matrix spec) or normal messages (IRC users might prefer this). This is a global setting currently. You can set a blacklist to ignore URLs containing words from the blacklist.

Commands:

Example:

NOTE: Disabled by default, i.e. you also need to enable it before activating it

Cmd

Can be used to pre-configure shell commands run by bot. This is easy way to add security issues to your bot so be careful.

Pre-defined commands can be set only by bot owner, but anyone can run them. It's your responsibility as owner to make sure you don't allow running anything dangerous.

Commands have 5 second timeout so don't try to run long processes.

Environ variables seen by commands:

Commands:

Example:

Astronomy Picture of the Day

Upload and send latest astronomy picture of the day to the room. See https://apod.nasa.gov/apod/astropix.html

Command:

API Key:

The module uses a demo API Key which can be replaced by your own api key by setting the environment variable APOD_API_KEY or by setting the api key as a bot owner with command !apod apikey [apikey].

You can create one at https://api.nasa.gov/#signUp

XKCD

Fetch comic strips from XKCD (A webcomic of romance, sarcasm, math, and language) and post them to the room.

Command:

Inspirobot

Query a randomly generated inspirational poster from https://inspirobot.me/ upload and send to the room.

Command:

User Status

This is a substitute for matrix' (element's?) missing user status feature. Save a custom (status) message for users and allows to query them.

Commands:

Wolfram Alpha

Make queries to Wolfram Alpha

You'll need to get an appid from https://products.wolframalpha.com/simple-api/documentation/

Examples:

Commands:

OGN Field Log (FLOG)

Open Glider Network maintains a unified tracking platform for gliders, drones and other aircraft.

Read more about OGN at https://www.glidernet.org/

FLOG module supports showing field logs for OGN receivers and can display live field log in a room. It can also show latest known location of an aircraft using !sar command.

It uses FlightBook instance at https://flightbook.glidernet.org/

FlightBook sources and documentation at https://gitlab.com/lemoidului/ogn-flightbook

Real life field log output looks something like:

Flights at Chateau Arnoux St Auban (LFMX) 2021-04-09:
13:36-18:01 04:24 F-CLDG JS-3 18M FM 2291m
13:46-··:··       F-CIFF Arcus T FP
13:57-17:13 03:15 JS-1 C21 72 2636m
14:08-17:30 03:22 ZS-GCC JS-1 C21 FD 2754m
18:29-··:··       F-CLDG JS-3 18M FM

Commands and examples:

(You must be room admin for all commands)

NOTE: disabled by default

Jitsi

If enabled, Jitsi calls created with Matrix clients will be sent as text messages to rooms, allowing non-matrix users to join them.

Mastodon

Send toots to Mastodon. You can login to Mastodon with the bot and toot with it.

Normal login is personal - it's mapped to your Matrix ID.

Room login overrides personal login and allows room admins to toot to a specified account.

You can limit usage to bot owners only or make it public.

Note: You can subscribe to Mastodon users and hashtags with stock RSS feed integration. Mastodon generates the feeds automatically!

Commands:

Example commands:

Relay bridge

Bridges two or more Matrix rooms together via relaybot.

Note: Room ID is not same as room alias! Rooms can exist without aliases so ID's are more flexible. Room id is usually in format !123LotOfRandomChars:server.org

To get room ID, it's in Element Web room settings | Advanced | Internal room ID

Before bridging the bot must be present on both rooms.

Commands:

File uploads, joins, leaves or other special events are not (yet) handled. Contributions welcome.

Relaybots are stupid. Please prefer real Matrix bridges to this. Sometimes there's no alternative.

Printing

With this module you can set up a room to print any uploaded files on a specified printer. The printer must be visible to bot via CUPS.

Commands (all can be done by bot owner only):

The module sends the files to CUPS for printing so please see CUPS documentation on what works and what doesn't.

Tested formats: PDF, JPG, PNG

SVG files are printed as text currently, avoid printing them.

This module is disabled by default.

Giphy

Can be used to post a picture from giphy given a query string.

API Key:

The module has no API Key set up by default. You have to provide an api key by using the relevant command.

Read the documentation to create one at https://developers.giphy.com/docs/api

Commands:

Example:

Gfycat

NOTE: This module is not working at the moment - gfycat now needs API key

Can be used to post a picture from Gfycat given a query string.

Commands:

Example:

Tautulli

Can be used to fetch recently added information from Tautulli or receive Tautulli recently added notification webhook

Commands:

Tautulli instance and API Key:

The module work with an instance of Tautulli accessible on URL defined by env variable TAUTULLI_URL In order to load art pictures you need to define an instance of Plex Media Server with its token as well by the two env variables PLEX_MEDIA_SERVER_URL and PLEX_MEDIA_SERVER_TOKEN You have to provide an api key by using the relevant command.

Environ variables seen by command:

Docker environment:

Since the module needs access to the source of the running Tautulli instance volumes on both Docker (hemppa and Tautulli) should be defined and being visible each other. When running on Docker the env variables seen by command should be defined for the bot instance.

Example:

Github based asset management

This module was written for asset (machines, tasks and todo) management by mis-using GitHub issues and labels. It has been designed to be used with hackerspace environment but can be extended to any purpose.

Github project setup

Hackerspace machines, todo and stuff. domains={"koneet":["#BFDADC","#0CBBF0","#0CBBF0","#E15D19","#ED49CF"],"tilat":["#0E8A16","#1E8A16"]}

Make sure you type the description on one line - this is a silly Github limitation.

Usage

Repository name must be in format TampereHacklab/Inventaario - you can use this as a example to see how the labels work.

PeerTube search

Searches PeerTube instances for videos. Uses Sepia Serch at https://sepiasearch.org/ by default, but you can set any single instance to search on.

Usage

User management

Admin commands to manage users and some utilities.

You can classify users based on MXID to get stats on where users come from.

Usage

Example:

RASP (Gliding Weather forecast)

Currently only Finnish RASP supported. Uses data from http://ennuste.ilmailuliitto.fi/

Usage

Day and hour can be omitted - fetches today's forecast if not set. PR welcome for supporting other data sources.

Mumble

Show information about a configured mumble server, including version, how many users are connected, and ping time.

Usage

Nitter

Reads Twitter links from room, replaces domain with nitter, removes query parameters and posts link to room.

Usage

Wikipedia

Searches Wikipedia for a given query and returns the first result summary and link.

Usage

Dice Roll

Rolls dice in XdY format.

For more syntax help, see https://d20.readthedocs.io/en/latest/start.html#dice-syntax.

Bot setup

Running locally

Run something like (tested on Ubuntu):

sudo apt install python3-pip libcups2-dev libatlas-base-dev gcc
sudo pip3 install pipenv
pipenv shell
pipenv install --pre                       (this takes ages, go make a sandwich)
MATRIX_USER="@user:matrix.org" MATRIX_ACCESS_TOKEN="MDAxOGxvYlotofcharacters53CgYAYFgo" MATRIX_SERVER="https://matrix.org" JOIN_ON_INVITE=True BOT_OWNERS=@botowner:matrix.org
 python3 bot.py

NOTE: The Pipfile does not define the python version as it is always strict and causes problems. See https://github.com/pypa/pipenv/issues/1050 . Python 3.7 and 3.8 should both work fine.

Running locally with systemd unit file (Raspberry Pi etc)

First modify run.sh and make sure Hemppa starts successfully with it.

There's a systemd unit file you can use. It assumes hemppa is installed to /opt/hemppa and is run as user pi in group pi. Note: this is probably not the smartest way to do this, feel free to do a PR for better way.

If your user is not pi, modify hemppa.service first.

sudo ln -s `pwd` /opt/hemppa
sudo ln -s `pwd`/hemppa.service /lib/systemd/system
sudo systemctl daemon-reload
sudo systemctl enable hemppa.service
sudo systemctl start hemppa.service
sudo systemctl status hemppa.service

Status should show "Active: active (running)"

Running with Docker

Create .env file and set variables:

MATRIX_USER=@user:matrix.org
MATRIX_ACCESS_TOKEN=MDAxOGxvYlotofcharacters53CgYAYFgo
MATRIX_SERVER=https://matrix.org
JOIN_ON_INVITE=True
BOT_OWNERS=@user1:matrix.org,@user2:matrix.org
DEBUG=False
TZ=America/New_York

Note: without quotes!

Just run:

docker-compose up

Env variables

MATRIX_USER is the full MXID (not just username) of the Matrix user. MATRIX_ACCESS_TOKEN and MATRIX_SERVER should be url to the user's server (non-delegated server). Set JOIN_ON_INVITE (default true) to false if you don't want the bot automatically joining rooms.

You can get access token by logging in with Element Android and looking from Settings / Help & About.

BOT_OWNERS is a comma-separated list of matrix id's for the owners of the bot. Some commands require sender to be bot owner. Typically set your own id into it.

OWNERS_ONLY is an optional variable once defined only the owners can operate the bot (this is a form of whitelisting)

INVITE_WHITELIST (default empty) is an optional comma-separated list of matrix id's to restrict the acceptance of invites into rooms of the bot to users or servers. It supports wild cards: Example value: @user:matrix.org,@*:myserver.org

LEAVE_EMPTY_ROOMS (default true) if this is set to false, the bot will stay in empty rooms

ATTENTION: Don't include bot itself in BOT_OWNERS if cron or any other module that can cause bot to send custom commands is used, as it could potentially be used to run owner commands as the bot itself.

To enable debugging for the root logger set DEBUG=True.

TZ takes any valid TZ database name value and sets the bot server to the appropriate zone.

Module API

Just write a python file with desired command name and place it in modules. See current modules for examples. No need to register it anywhere else.

Simple skeleton for a bot module:


class MatrixModule(BotModule):

    async def matrix_message(self, bot, room, event):
        args = event.body.split()
        args.pop(0)

        # Echo what they said back
        self.logger.debug(f"room: {room.name} sender: {event.sender} wants an echo")
        await bot.send_text(room, ' '.join(args), event=None)

    def help(self):
        return 'Echoes back what user has said'

Functions

You only need to implement the ones you need. See existing bots for examples.

Bot API

class Bot:
    async def send_msg(self, mxid, roomname, message):
        """

        :param mxid: A Matrix user id to send the message to
        :param roomname: A Matrix room id to send the message to
        :param message: Text to be sent as message
        :return bool: Success upon sending the message
        """

    async def send_text(self, room, body, event=None, msgtype="m.notice", bot_ignore=False):
        """

        :param room: A MatrixRoom the text should be send to
        :param body: Textual content of the message
        :param event: The event to reply to
        :param msgtype: The message type for the room https://matrix.org/docs/spec/client_server/latest#m-room-message-msgtypes
        :param bot_ignore: Flag to mark the message to be ignored by the bot
        :return:
        """

    async def send_html(self, room, html, plaintext, event=None, msgtype="m.notice", bot_ignore=False):
        """

        :param room: A MatrixRoom the html should be send to
        :param html: Html content of the message
        :param plaintext: Plaintext content of the message
        :param event: The event to reply to
        :param msgtype: The message type for the room https://matrix.org/docs/spec/client_server/latest#m-room-message-msgtypes
        :param bot_ignore: Flag to mark the message to be ignored by the bot
        :return:
        """

    async def send_image(self, room, url, body, event=None, mimetype=None, width=None, height=None, size=None):
        """

        :param room: A MatrixRoom the image should be send to
        :param url: A MXC-Uri https://matrix.org/docs/spec/client_server/r0.6.0#mxc-uri
        :param body: A textual representation of the image
        :param event: The event to reply to
        :param mimetype: The mimetype of the image
        :param width: Width in pixel of the image
        :param height: Height in pixel of the image
        :param size: Size in bytes of the image
        :return:
        """

    async def upload_image(self, url, blob=False, blob_content_type="image/png"):
        """

        :param url: Url of binary content of the image to upload
        :param blob: Flag to indicate if the first param is an url or a binary content
        :param blob_content_type: Content type of the image in case of binary content
        :return: A MXC-Uri https://matrix.org/docs/spec/client_server/r0.6.0#mxc-uri, Content type, Width, Height, Image size in bytes
        """

    async def upload_and_send_image(self, room, url, event=None, text=None, blob=False, blob_content_type="image/png"):
        """

        :param room: A MatrixRoom the image should be send to after uploading
        :param url: Url of binary content of the image to upload
        :param event: The event to reply to
        :param text: A textual representation of the image
        :param blob: Flag to indicate if the second param is an url or a binary content
        :param blob_content_type: Content type of the image in case of binary content
        :return:
        """
    async def send_location(self, room, body, latitude, longitude, event=None bot_ignore=False):
        """

        :param room: A MatrixRoom the html should be send to
        :param html: Html content of the message
        :param body: Plaintext content of the message
        :param latitude: Latitude in WGS84 coordinates (float)
        :param longitude: Longitude in WGS84 coordinates (float)
        :param event: The event to reply to
        :param bot_ignore: Flag to mark the message to be ignored by the bot
        :return:
        """

Logging

Uses python logging facility to print information to the console. Customize it to your needs editing config/logging.yml. See logging.config documentation for further information.

Use self.logger in your module to print information to the console.

Module settings are stored in Matrix account data.

Ignoring text messages

If you want to send a m.text message that bot should always ignore, set "org.vranki.hemppa.ignore" property in the event. Bot will ignore events with this set. Set the bot_ignore parameter to True in sender functions to acheive this.

If you write a module that installs a custom message handler, use bot.should_ignore_event(event) to check if event should be ignored.

Aliasing modules

A module can declare its own aliases with the add_module_aliases command. You probably want to call it during matrix_start:

class MatrixModule(BotModule):

    def matrix_start(self, bot):
        super().matrix_start(bot)
        self.add_module_aliases(bot, ['newname', 'anothername'])

Then you can call this module with its original name, !newname, or !another-name. (Like module names, Hemppa ignores non-alphanumeric characters in aliases.)

Contributing

If you write a new module, please make a PR if it's something useful for others.