Access repository without config file

[happysalada]

Currently restic only supports restoring a backup on the same machine it was created. That is quite inconvenient because all your backups are lost in case you loose the machine. Does rustic support restoring a backup on a different machine than that of the one where the backup was created ? If not, would you consider it ?

[aawsome]

Rustic currently only supports restore to "local storage", which means onto the filesystem. But this can also be some remote files system that has been locally mounted.

Restore to a remote destination is not yet supported by rustic. But this doesn't mean that restoring on a different machine is not supported. If you have a repository which can be accessed by a different machine (this can be a cloud storage or a storage using the REST backend or even a locally mounted or copied repository), then you can run rustic on this machine and restore to its local discs. This allows to restore in case you loose a complete machine.

In fact, rustic is perfectly capable of using a repsitory with multiple machines each backing up concurrently into the same repository - and all snapshots within this repository can be restored to any of the machines.

BTW: This all also applies to restic.

[happysalada]

hey, thanks a lot for the quick answer. I was using restic, and here is what I ran into, albeit that I stored my backup on an s3 machine. https://forum.restic.net/t/unable-to-open-config-file-can-i-restore-it/1648 trying to restore it just gives me that error "Fatal: unable to open config file: Stat: Invalid Argument: Authorization Is there a repository at the following location?" I haven't found a solution to this, and I still have the repo, so I would be happy to try different things to confirm this is working. My guess is that it's not. Did you ever try restoring a rustic s3 backup on a different machine ? (just wondering, I'm happy to try as well, but this is the use case I'm looking for).

(and just to make sure, this is the command I am using to restore the backup

restic restore -r "s3:46d381004464b9e0c88b3bb5a0f0f100.r2.cloudflarestorage.com/back
up/gitea" --target "/var/lib/gitea" latest

with the S3_CREDENTIALS env var set to a path that contains the credentials)

[aawsome]

Just read the topic at the restic form, thanks for the link. So it seem in your case that for some reason the repository lost file(s). In order to confirm this: Can you configure rclone to access the repository and send the list the files which are containted? (You don't need to list all files which are similar, but in fact the directories containing something.

If it is only "just" the config file which is missing, I'll happily make a rustic repair config command to re-create one which will then allow you to restore from that repo. That said, chances are that if one file disappeared in the storage, there might be others, so you could have more problems with that repository.

[aawsome]

Did you ever try restoring a rustic s3 backup on a different machine ? (just wondering, I'm happy to try as well, but this is the use case I'm looking for).

I myself didn't try this, but I did this with other cloud storages and I know about other people successfully doing this with S3. So it's definitively not a topic about restoring to a different machine, but about a - for whatever reason - corrupted repository and the possibility to still salve data from this. Rustic does offer possibilities for this and I'm willing to enhance them such that all kind of corruptions can be handled - even though there might be cases where there is nothing to do. In your case, however, we should first try all posibilities before giving up.

[happysalada]

I just tried setting up rclone, and I'm sure I have the right values, but still

❯ rclone size r2_gitea:/
Total objects: 0 (0)
Total size: 0 B (0 Byte)

The one thing that I couldn't find how to configure with rclone, is that I used the "BACKUP_PASSWORD" environment variable, maybe that is the reason why no file shows ? I can see from the r2 ui that the files are there. I'm going to dowload the files using the wrangler cli and upload them. (I can also see that the folders include a config file, so I'm not sure why restic fails).

[happysalada]

I've managed to restore the backup. Here is what I found out.

• To use rclone, the "s3_url" should not include the bucket
• if restic is missing the env_var RESTIC_PASSWORD, the error message will be that the config file is missing (in case you encrypted the backup with a password). In restic the RESTIC_REPOSITORY has to include the bucket in the URL.
• If you are trying to restore a backup made by a service running as DynamicUser from systemd, you won't be able to restore the backup because the user that has write permission to your directory is not accessible.

I have one last question

• Reading the full config) it seems like the way it's supposed to be used is that you do one backup and it contains several sources. Would the way to run several backup be just to include several configuration files ? I'm asking in the context of making a nixos module to run rustic. The use case for making different backup, is that you might want a different strategy for your database backups than your photos backup for example. It looked as if you would have one backup with several sources in the config.

[happysalada]

Im going to close this issue for now. When you have a moment im curious as to how rustic is supposed to be used. Nixos is still lacking a systemd service for it.

[aawsome]

Sorry for the late reply @happysalada

• it seems like the way it's supposed to be used is that you do one backup and it contains several sources. Would the way to run several backup be just to include several configuration files ?

Actually it is the other way round: You usually specify multiple backup sources (with individual backup options, if you like) which will then each result in one separate snapshot. If you run rustic backup without a source, it will read the config file and produce all the snapshots.

However, you can (like in restic) specify multiple paths within one source. In this case, you will get a single snapshot containing just this path. But then also some backup options can just be specified for the whole snapshot.

When you have a moment im curious as to how rustic is supposed to be used. Nixos is still lacking a systemd service for it.

Is this the same question I answered above or do you want to get more information about how to run rustic? I'm not familiar with Nixos, so I can't say anything about it, sorry.

[happysalada]

I think it makes sense. So you have a single repository, but if you define multiple backup.sources, it will generate a snapshot for each source. nixos is just a configurable os, you have to make a compatibility layer between the configuration language and the toml language. here is how a restic configuration looks like

{ config, pkgs, ... }:

let
  s3_url = "46d381004464b9e0c88b3bb5a0f0f100.r2.cloudflarestorage.com/backup";
  defaults = {
    initialize = true;
    timerConfig = {
      OnCalendar = "daily";
      Persistent = true;
    };
    passwordFile = config.age.secrets.BACKUP_PASSWORD.path;
    environmentFile = config.age.secrets.S3_CREDENTIALS.path;
    pruneOpts = [
      "--keep-daily 30"
      "--keep-weekly 52"
      "--keep-monthly 120"
    ];
  };
in
{
  services.restic = {
    backups = {
      postgresql = defaults // {
        paths = ["/root/pg_backup.sql"];
        repository = "s3:${s3_url}/postgresql";
        backupPrepareCommand = ''
          ${pkgs.sudo}/bin/sudo -u postgres ${pkgs.postgresql_15}/bin/pg_dumpall > /root/pg_backup.sql
        '';
      };
      # surrealdb = defaults // {
      #   paths = ["/root/backup.surql"];
      #   repository = "s3:${s3_url}/surrealdb";
      #   # TODO figure out how to inject the correct user and password.
      #   backupPrepareCommand = ''
      #     surreal backup http://127.0.0.1:${toString config.services.surrealdb.port}  /root/backup.surql --user ___ --pass ___ 
      #   '';
      # };
      bitwarden_rs = defaults // {
        paths = ["/var/lib/bitwarden_rs"];
        repository = "s3:${s3_url}/bitwarden_rs";
      };
      gitea = defaults // {
        paths = ["/var/lib/gitea"];
        repository = "s3:${s3_url}/gitea";
      };
      rustus = defaults // {
        paths = ["/var/lib/rustus"];
        repository = "s3:${s3_url}/rustus";
      };
    };
  };

  age.secrets =  {
    BACKUP_PASSWORD = {
      file = ../secrets/restic.backup.password.age;
    };
    S3_CREDENTIALS = {
      file = ../secrets/restic.s3.credentials.age;
    };
  };
}

it's a bit long, but you get the idea, you define all these backup targets, and nixos will run a separate systemd process with restic for each of those. I was wondering if it was the same for rustic, but it sounds like, you only need to run a single systemd process for a backup with multiple sources. I thought all the sources got aggregated into one snapshot, and that people might want different snapshots for different files, but it sounds like what rustic is doing makes the most sense. I'm going to have a go at this when I have a moment and will report here if I find anything weird.