(Feature request) Separated TRILIUM_DATA_DIR for document, backup and config

[stevefan1999-personal]

Describe feature

I want to separate backups and config file away to their own variables. I can therefore mount my S3 storage class into a specific location and back up on the go. Plus, I would like to generate the config by a config map.

Additional Information

LiteFS does not support storing other kinds of file:

LiteFS handles user-created files based on their suffix:

Journal files have a -journal suffix. WAL files have a -wal suffix. Shared memory files have a -shm suffix.

All other user-created files are treated as SQLite databases. Additionally, there are special read-only files made available by LiteFS as an easy way to communicate state to the end user.

Focus on the bold part, this would clearly be wrong if we have placed backup and config file into the same location for LiteFS

[stevefan1999-personal]

https://github.com/zadam/trilium/blob/1b24276a4a2ab8e5b1814c8fe6ca222b9ab29de4/src/services/data_dir.js#L62-L67 Those are the places where the paths are controlled

[stevefan1999-personal]

Here's the design:

We will preserve the previous behavior of using the omnibus data directory

But we will add independent environmental variables for each file/directory path to specify their own location. TRILIUM_BACKUP_DIR, TRILIUM_LOG_DIR, etc.

In theory they should be mutually exclusive (meaning you should not mix TRILIUM_DATA_DIR and independent environmental variables together), but we will just allow it for now until later, and warn the user about not recommending to mix them together.

Independent environmental variables take over precedence above TRILIUM_DATA_DIR.

Examples

Given the following scenarios:

$TRILIUM_DATA_DIR = "/data"
$TRILIUM_BACKUP_DIR = ""
$TRILIUM_LOG_DIR = ""

Then the missing paths will be imputed as:

$TRILIUM_DATA_DIR = "/data"
$TRILIUM_BACKUP_DIR = "$TRILIUM_DATA_DIR/backup" # imputed
$TRILIUM_LOG_DIR = "$TRILIUM_DATA_DIR/log" # imputed

---

If you do not have a data dir but independent variables:

$TRILIUM_DATA_DIR = ""
$TRILIUM_BACKUP_DIR = "/mnt/rclone/s3/trilium-backup"
$TRILIUM_LOG_DIR = "/var/log/trilium"

The app should work as is.

---

If you have missed one path however, and data path isn't set:

$TRILIUM_DATA_DIR = ""
$TRILIUM_BACKUP_DIR = "/mnt/rclone/s3/trilium-backup"
$TRILIUM_LOG_DIR = ""

The app should panic and expect to display an error dialog to remind the user to fill in remaining fields.

---

However, if one of the independent environmental variables are not empty:

$TRILIUM_DATA_DIR = "/data"
$TRILIUM_BACKUP_DIR = "/mnt/rclone/s3/trilium-backup"
$TRILIUM_LOG_DIR = "" 

You should expect to see:

$TRILIUM_DATA_DIR = "/data"
$TRILIUM_BACKUP_DIR = "/mnt/rclone/s3/trilium-backup"
$TRILIUM_LOG_DIR = "$TRILIUM_DATA_DIR/log" # imputed

And warn the user of this to give them a notice in case they confused about mixing the whereabouts of the independent files on different locations.

[zadam]

Hi, why not just use symlinks?

[stevefan1999-personal]

@zadam It might not be suitable to use symlink in a containerized environment because we basically have two sets of ACL to manage. Plus another issue is that LiteFS does not support regular files other than SQLite stuffs, but symlink is still considered a regular file (both soft links and hard links are as well)

[zadam]

Ok, will be released in 0.58 soon: https://github.com/zadam/trilium/wiki/Data-directory#fine-grained-directorypath-location