[Feature request] Dedicated options to specify state.db and the configuration folder

StayPirate commented 4 months ago

Context

I was looking for a way to easily manage rss2email's configuration files (feeds.txt and templates) via git. For that I created a companion container which shares a common volume with the rss2email's container and its only task is to keep the volume content in sync with a remote git repository.

This companion container is just a simple bash script polling a remote git repository and pull data in case it has changed.

In my case, the remote repository contains rss2email's configuration data. Data is synced in a volume that is also mounted by the rss2email container, this way rss2email sees any change sent to the remote repository. And this will allow the user (me) to conveniently manage templates and feeds via git.

Real case example

I created this docker compose stack where the two containers are configured to share a common volume named rss2email_config:

services:

  rss2email_git-local-sync:
    [...]
    volumes:
      - "rss2email_config:/repo"

[...]

  rss2email:
    [...]
    volumes:
      - "rss2email_state:/app/.rss2email/state"
      - "rss2email_config:/app/.rss2email/config:ro"

Limitation

Unfortunately, rss2email expects to find (or attempt to write) the state.db file in the same location. This is sub-optimal for my setup, because I'd like to mount the common volume in read-only mode on the rss2email's container.

Feature request

In order to make my setup work I need some extra configuration flexibility from rss2email. Would be possible to add two new cmdline options, one to specify a custom path for the configuration folder (feeds.txt and templates) and the other to specify a custom pathname for stated.db?

I think something like the following might work:

rss2email \
    --config /app/.rss2email/config/rss2email \
    --state /app/.rss2email/state/state.db \
    daemon user@email.com

skx commented 4 months ago

I guess the problem here is simple enough to describe:

We have one directory containing two different things:
- Templates and configuration files that might want to be versioned, via git or similar.
- A state-database that nobody cares about, except it must be writeable.

I wonder if there's an indirect solution here without modifying anything - if we assume you have two volumes:

"config" - you could mount that at /app/.rss2email/config
"state" - You could mount that at /var/rss2email

Then create a symlink in config to point to the state. That should let you keep the two parts isolated and still work - though I admit I've not tested if a symlink/hardlink will work I kinda assume it would.

I'm suggesting this partly because it's a quick "fix", and partly because I'm loathe to make this too configurable:

If we allow a different config-path then we have to update all the commands, run, list, add, delete, etc, etc.
If we allow a different state-path then I have to do a lot of testing, and worry about what would happen if I miss a spot - presumably everything becomes unseen.

Having done a couple of migration-updates in the past they're always way more annoying than I would expect in advance and if I can avoid one that might be best.

(There are times when I think I should only support "rss2email run|cron" and drop the other commands, but that would probably make a lot of people annoyed with me!)

TLDR; Can you use symlinks? If not we might have to come up with a plan, I think moving either alone would suffice, but both are more involved than I'd like so there's not an easy choice about which should be moved. Everything uses state.Directory() as things stand.

StayPirate commented 4 months ago

Hi Steve, I understand your point of view and find your symlink suggestion very effective. As you can see from here I added the symlink to the root of the repo where I'm storing rss2email's configuration files (feed.txt, templates/). I mounted this repo in the rss2email container at ~/.rss2email, and it worked properly.

My current setup can also be seen from the docker-compose file.

Thanks again and feel free to close this issue if you want.

skx commented 4 months ago

I think with the working symlink solution I will close this. I can understand why people could want to separate the state and the config, but I think it would be a challenge to implement cleanly right now.

If it comes up in the future I guess I can reassess my reluctance, but since you're happy with the workaround for the moment I'll ignore the issue :)

Thanks again for such a well-reasoned report, seeing your issues and pull-requests is always a pleasure.

skx / rss2email