.. list-table:: :widths: auto :align: center
It's a small bit of Python glue between great projects, including:
Fallback fonts:
The background image is from Sharon McCutcheon.
Send @colorcodebot_ the code you want highlighted,
as a forwarded or original direct message.
Or, add it to your group
and it will send an image of any monospace content sent in the chat.
.. image:: https://user-images.githubusercontent.com/1787385/174667742-32e414b4-e4f4-41f8-ae38-d6d64c1075f2.png :alt: Screenshot of the bot in action :align: right
As a convenience, you can get to a direct chat with it from any other chat,
by typing @colorcodebot and tapping the button that pops up.
A button returning you (with a shiny new image)
to your original chat will be presented after you send the code.
The bot should run anywhere with Python, fontconfig, highlight, Silicon, and the ability to install TensorFlow. Or anywhere that can run a container image.
Depending on your hardware, you may see faster syntax guessing (from guesslang_)
by installing cuda and cudnn packages.
This is not done for the currently hosted container images,
which is the result of ./mk/ctnr.sh -d prod --push run by a GitHub Action.
Outside of the core Python app,
sops is used for secrets,
buildah for container building,
GitHub Actions for automated container image builds and other CI tasks,
and wheezy.template and yamlpath are extremely handy for
defining+rendering service definitions and other dev/ops maneuvers.
Most of the mk/ and start/ scripts are POSIX,
but mk/svcs.zsh requires Zsh,
and mk/ctnr.sh calls mk/svcs.zsh.
Please do send a message_ or open an issue with any questions.
Organization
An abbreviated file tree overview:
.. code:: shell
colorcodebot/
├──app/ # core app that gets deployed
│ ├──requirements.in # loosely versioned reqs for the bot
│ └──sops/ # encrypted deployment-specific data
├──vars.<deployment>.yml # unencrypted deployment-specific data
├──start/ # scripts that help start the bot
├──mk/ # scripts that make things
├──templates/ # used by mk/ scripts to generate files
└──ops-requirements.in # loosely versioned reqs for mk/ and start/ scripts
The following are generated by ``mk/`` scripts:
.. code:: shell
colorcodebot/
├──app/
│ ├──requirements.txt # mk/reqs.sh - lockfile for the bot
│ ├──svcs/ # mk/svcs.zsh - supervised process definitions for s6 [untracked]
│ └──theme_previews.yml # mk/file_ids.sh - {theme_name: image_id} [untracked]
└──ops-requirements.txt # mk/reqs.sh - lockfile for mk/ and start/ scripts
When building a container image with ``mk/ctnr.sh``,
``app`` becomes ``/home/colorcodebot``.
If you want to use the container images already built from this repo,
you'll probably write or mount over:
- ``/home/colorcodebot/theme_previews.yml``
- ``/home/colorcodebot/svcs``
- ``/home/colorcodebot/sops``
- ``/home/colorcodebot/.sops.yaml``
Getting StartedTo run colorcodebot.py, the environment variable TG_API_KEY must be set,
with a token from @botfather_.
.. code:: console
$ python3 -m venv app/venv $ . ./app/venv/bin/activate $ python -m pip install -r app/requirements.txt $ TG_API_KEY='...' ./app/colorcodebot.py
After chatting with the bot, check the logs for your chat_id.
Pass this as an additional environment variable ADMIN_CHAT_ID to get:
Deployments, Secrets, and Scripts
Encrypted Variables
^^^^^^^^^^^^^^^^^^^
Configure Sops
""""""""""""""
Create one or more age_ keys to use with sops_:
.. code:: console
$ mkdir -p ~/.config/sops/age
$ printf '%s\n' '' '# --- colorcodebot ---' >>~/.config/sops/age/keys.txt
$ age-keygen >>~/.config/sops/age/keys.txt
Public key: age1r50agxl277e24h4ammj0kvpqh224ut8ds67qc2d537dq0uy74shq98dh97
And use that public key in ``.sops.yaml`` to match your desired deployments.
Write colorcodebot Variables
""""""""""""""""""""""""""""
Overwrite ``app/sops/colorcodebot.<deployment>.yml`` with
.. code:: yaml
TG_API_KEY: <put-the-real-token-here>
(and optionally ``ADMIN_CHAT_ID``) and encrypt it with
.. code:: console
$ sops -e -i app/sops/colorcodebot.<deployment>.yaml
.. You can set ``host`` and ``port`` in ``app/sops/papertrail.<deployment>.yml``
.. the same way, if using that service.
Load colorcodebot Variables
"""""""""""""""""""""""""""
.. code:: console
$ ./start/local.sh -h
Start the bot locally, without process supervision or other svcs
Args: [-d <deployment>=dev]
You can use ``start/local.sh`` to:
- ensure Python lockfile is updated
- ensure a virtual environment exists
- ensure the venv has all Python dependencies installed
- ensure the venv is activated if one is not already
- update or create ``app/theme_previews.yml`` if file IDs are present in ``vars.<deployment>.yml``
- load decrypted values from ``app/sops/colorcodebot.<deployment>.yml`` into environment variables
- launch the bot (unsupervised, no other services)
You can do just those last two (as seen in the script) with
.. code:: console
$ sops exec-env "app/sops/colorcodebot.${deployment}.yml" app/colorcodebot.py
Unencrypted Variables
^^^^^^^^^^^^^^^^^^^^^
A deployment's unencrypted variables are defined by ``vars.<name>.yml``.
There are two top-level keys:
``theme_previews``
mapping of theme names to Telegram file IDs; see `Generating Theme Previews`_
used by: ``mk/file_ids.sh``, ``mk/ctnr.sh``
``svcs``
list of mappings that each define a long-running supervised service
(the bot and optionally a log sender for Papertrail_)
used by: ``mk/svcs.sh``, ``mk/ctnr.sh``
The deployments ``dev`` and ``prod`` are both intended to run inside a container,
built by ``mk/ctnr.sh``.
Note the difference between the ``svc`` definitions
of ``vars.dev.yml`` and ``vars.prod.yml``:
.. code:: diff
--- vars.dev.yml 2021-06-28 11:13:46.347838948 -0400
+++ vars.prod.yml 2021-07-12 14:22:07.638842356 -0400
@@ -4,7 +4,7 @@
exec: >-
sops exec-env
- sops/colorcodebot.dev.yml
+ sops/colorcodebot.prod.yml
"s6-setuidgid colorcodebot ./venv/bin/python
./colorcodebot.py"
@@ -16,7 +16,7 @@
exec: >-
sops exec-file --filename log_files.yml
- ../log_files.dev.yml
+ ../log_files.prod.yml
"remote_syslog -D -c {}"
@@ -24,7 +24,7 @@
sops_templates:
- src: papertrail.log_files.yml.wz
- dest: log_files.dev.yml
+ dest: log_files.prod.yml
- differences:
+ which encrypted variables get set in the environment of the bot process
+ which encrypted config file is created for and read by the remote logger
Now let's compare ``vars.dev.yml`` to ``vars.local.yml``:
.. code:: diff
--- vars.dev.yml 2021-06-28 11:13:46.347838948 -0400
+++ vars.local.yml 2021-07-12 13:57:00.414719676 -0400
@@ -6,14 +6,15 @@
- "s6-setuidgid colorcodebot ./venv/bin/python
+ "./venv/bin/python
./colorcodebot.py"
folder:
run: ../../
log: ../../../logs/colorcodebot
+ cgroups: /sys/fs/cgroup/user.slice/user-1000.slice/user@1000.service/app.slice/svcs
- name: papertrail
- enabled: true
+ enabled: false
@@ -22,6 +23,7 @@
folder:
run: log
log: ../../../logs/papertrail
+ cgroups: /sys/fs/cgroup/user.slice/user-1000.slice/user@1000.service/app.slice/svcs
- similarities:
+ which encrypted configs are used
- differences:
+ ``local``: no user changing (no ``s6-setuidgid``)
+ ``local``: overrides the default cgroup path used by services with a systemd-flavored one
+ ``local``: disables optional Papertrail remote logging service
Modify one of these to your liking, or copy to ``vars.<name>.yml`` with your own deployment name, e.g.:
.. code:: console
$ cp vars.local.yml "vars.$(hostname).yml"
Generating Theme PreviewsOUTDATED!
TODO: Fill in the missing steps in this section
now that the /previews command has been removed
highlight_ has many themes, so we picked a subset.
For the user to choose a theme, we need to generate preview images, and save their file IDs.
Start by creating app/theme_previews.yml either manually or with ./mk/file_ids.sh
.. code:: console
$ ./mk/file_ids.sh -h
Generate theme_previews.yml, with data from vars.
For now the value of each entry can be garbage, what's important is that the keys are the names of the themes you wish to offer.
Send the /previews command to the bot, and the file IDs you need
will show up in the log as preview images are generated and sent your way.
Enter those into vars.<deployment>.yml,
then generate app/theme_previews.yml for local deployment with mk/file_ids.sh,
which is automatically called by start/local.sh and mk/ctnr.sh.
.. _@botfather: https://t.me/botfather .. _a demo video: https://user-images.githubusercontent.com/1787385/123204250-ae9a0380-d485-11eb-981d-3302220aad58.mp4 .. _age: https://github.com/FiloSottile/age .. buildah: https://github.com/containers/buildah .. @colorcodebot: https://t.me/colorcodebot .. _guesslang: https://github.com/yoeo/guesslang .. _highlight: http://www.andre-simon.de/doku/highlight/highlight.html .. _Iosevka: https://github.com/be5invis/Iosevka .. _Papertrail: https://www.papertrail.com .. _pyTelegramBotAPI: https://github.com/eternnoir/pyTelegramBotAPI .. _send a message: https://t.me/andykluger .. _Silicon: https://github.com/Aloxaf/silicon .. _sops: https://github.com/mozilla/sops .. _wheezy.template: https://github.com/akornatskyy/wheezy.template .. _yamlpath: https://github.com/wwkimball/yamlpath
.. |actions-ctnr| image:: https://github.com/AndydeCleyre/colorcodebot/actions/workflows/ci.yml/badge.svg?branch=develop :alt: Automated Container Build Status :target: https://github.com/AndydeCleyre/colorcodebot/actions/workflows/ci.yml
.. |actions-fmt| image:: https://github.com/AndydeCleyre/colorcodebot/actions/workflows/fmt.yml/badge.svg?branch=develop :alt: Format and Lint Status :target: https://github.com/AndydeCleyre/colorcodebot/actions/workflows/fmt.yml
.. |actions-reqs| image:: https://github.com/AndydeCleyre/colorcodebot/actions/workflows/reqs.yml/badge.svg?branch=develop :alt: Automated Python Requirements Bump Status :target: https://github.com/AndydeCleyre/colorcodebot/actions/workflows/reqs.yml
.. |quay| image:: https://img.shields.io/badge/Quay.io-andykluger%2Fcolorcodebot--prod--archlinux-blue?logo=redhat :alt: Container Image Repository :target: https://quay.io/repository/andykluger/colorcodebot-prod-archlinux?tab=tags
.. |telegram| image:: https://img.shields.io/badge/Telegram-%40colorcodebot-blue?logo=telegram :alt: Telegram user @colorcodebot :target: https://t.me/colorcodebot