search

NixOS / nixpkgs

Nix Packages collection & NixOS
MIT License
17.3k stars 13.54k forks source link

Can't restore Mealie backup #325391

Closed poperigby closed 2 weeks ago

poperigby commented 2 months ago

Describe the bug

I'm attempting to restore a backup of Mealie using the built-in restore tool, but I'm running into an error:

ERROR    2024-07-07T18:20:53 - Path doesn't exist: '/nix/store/c3z2jp21qqr97j2bssmz89d7x9krkjf9-python3.11-mealie-1.9.0/lib/python3.11/site-packages/alembic'.  Please use the 'init' command to create a new scripts folder.
Traceback (most recent call last):
  File "/nix/store/c3z2jp21qqr97j2bssmz89d7x9krkjf9-python3.11-mealie-1.9.0/lib/python3.11/site-packages/mealie/routes/admin/admin_backups.py", line 105, in import_one
    backup.restore(file)
  File "/nix/store/c3z2jp21qqr97j2bssmz89d7x9krkjf9-python3.11-mealie-1.9.0/lib/python3.11/site-packages/mealie/services/backups_v2/backup_v2.py", line 101, in restore
    self.db_exporter.restore(database_json)
  File "/nix/store/c3z2jp21qqr97j2bssmz89d7x9krkjf9-python3.11-mealie-1.9.0/lib/python3.11/site-packages/mealie/services/backups_v2/alchemy_exporter.py", line 158, in restore
    command.upgrade(alembic_cfg, alembic_version)
  File "/nix/store/7s0lpzck4khnaxbbvg5vj49cpwz5pkgr-python3.11-alembic-1.13.1/lib/python3.11/site-packages/alembic/command.py", line 383, in upgrade
    script = ScriptDirectory.from_config(config)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/7s0lpzck4khnaxbbvg5vj49cpwz5pkgr-python3.11-alembic-1.13.1/lib/python3.11/site-packages/alembic/script/base.py", line 226, in from_config
    return ScriptDirectory(
           ^^^^^^^^^^^^^^^^
  File "/nix/store/7s0lpzck4khnaxbbvg5vj49cpwz5pkgr-python3.11-alembic-1.13.1/lib/python3.11/site-packages/alembic/script/base.py", line 107, in __init__
    raise util.CommandError(

Steps To Reproduce

Steps to reproduce the behavior:

  1. Use this Mealie configuration:
services.mealie = {
  enable = true;
  package = pkgs.mealie.overrideAttrs (o: {
    patches = (o.patches or [ ]) ++ [
      (pkgs.fetchpatch {
        url = "https://github.com/mealie-recipes/mealie/commit/445754c5d844ccf098f3678bc4f3cc9642bdaad6.patch";
        hash = "sha256-ZdATmSYxhGSjoyrni+b5b8a30xQPlUeyp3VAc8OBmDY=";
        revert = true;
      })
    ];
  });
};
  1. Sign in as an administrator
  2. Attempt to restore a backup

Expected behavior

The restore succeeds

Notify maintainers

@litchipi

Metadata

Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.

 - system: `"x86_64-linux"`
 - host os: `Linux 6.6.34, NixOS, 24.11 (Vicuna), 24.11.20240703.9f4128e`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.23.1`
 - nixpkgs: `/nix/store/dk2rpyb6ndvfbf19bkb2plcz5y3k8i5v-source`

Add a :+1: reaction to issues you find important.

eclairevoyant commented 2 months ago

Do you have a simple config to repro? Seems like services.mealie.enable = true; wasn't enough

edit: nvm, my issue was due to https://github.com/mealie-recipes/mealie/issues/3852

poperigby commented 2 months ago

@eclairevoyant where you able to reproduce after dealing with that issue? I don't have anything in my config that could be causing this, but I'll post it anyway: https://codeberg.org/PopeRigby/nixos/src/commit/706444f10ff494483954c3d6ef838da56e9201b4/systems/x86_64-linux/haddock/services/mealie.nix

eclairevoyant commented 2 months ago

Yes, I could repro after patching the package to get around the /run/secrets issue. I only set .enable and .package.

  services.mealie = {
    enable = true;
    package = pkgs.mealie.overrideAttrs (o: {
      patches = (o.patches or [ ]) ++ [
        (pkgs.fetchpatch {
          url = "https://github.com/mealie-recipes/mealie/commit/445754c5d844ccf098f3678bc4f3cc9642bdaad6.patch";
          hash = "sha256-ZdATmSYxhGSjoyrni+b5b8a30xQPlUeyp3VAc8OBmDY=";
          revert = true;
        })
      ];
    });
  };
litchipi commented 1 month ago

I don't understand how it's able to work even when the /run/secret patch revert is not applied...

Anyway, found a problem and created a PR in upstream: https://github.com/mealie-recipes/mealie/pull/3882

Not really sure it's the problem, but was a low hanging fruit. Could you add https://patch-diff.githubusercontent.com/raw/mealie-recipes/mealie/pull/3882.patch to your patches and try again ?

poperigby commented 1 month ago

I tried that patch and it sadly still didn't work. I confirmed the variable was set with systemctl show mealie:

ALEMBIC_CONFIG_FILE=/nix/store/ks7di8rg60p6v4liw7fxrrgp4k0m4rfv-python3.11-mealie-1.9.0/config/alembic.ini

I also wiped Mealie's data directory and database and recreated them before starting.

poperigby commented 1 month ago

@litchipi do you have any other ideas?

litchipi commented 1 month ago

@poperigby It's caused because of the ugly PROJECT_DIR = Path(__file__).parent.parent.parent lines. I'm working on a PR that will substitute all of these for the installation path in the nix store, this should do the trick hopefully.

litchipi commented 3 weeks ago

@poperigby Could you try now, using the master branch for nixpkgs ?

poperigby commented 3 weeks ago

@litchipi It still doesn't work, same error. I have no idea what's going on here.

litchipi commented 3 weeks ago

@poperigby Is it the same error ?

poperigby commented 3 weeks ago

Yeah, it still seems to be the same error:

Aug 14 19:41:01 haddock mealie[2543954]: INFO     2024-08-14T19:41:01 - initializing backup restore
Aug 14 19:41:01 haddock mealie[2543954]: INFO     2024-08-14T19:41:01 - dropping all database tables
Aug 14 19:41:02 haddock mealie[2543954]: INFO     2024-08-14T19:41:02 - importing database tables
Aug 14 19:41:02 haddock mealie[2543954]: ERROR    2024-08-14T19:41:02 - Path doesn't exist: '/nix/store/z9a8pij846xx0yh29ygc6lk2m9iq6ch1-mealie-1.11.0/alembic'.  Please use the 'init' command to create a new scripts folder.
Aug 14 19:41:02 haddock mealie[2543954]: Traceback (most recent call last):
Aug 14 19:41:02 haddock mealie[2543954]:   File "/nix/store/z9a8pij846xx0yh29ygc6lk2m9iq6ch1-mealie-1.11.0/lib/python3.12/site-packages/mealie/routes/admin/admin_backups.py", line 105, in import_one
Aug 14 19:41:02 haddock mealie[2543954]:     backup.restore(file)
Aug 14 19:41:02 haddock mealie[2543954]:   File "/nix/store/z9a8pij846xx0yh29ygc6lk2m9iq6ch1-mealie-1.11.0/lib/python3.12/site-packages/mealie/services/backups_v2/backup_v2.py", line 101, in restore
Aug 14 19:41:02 haddock mealie[2543954]:     self.db_exporter.restore(database_json)
Aug 14 19:41:02 haddock mealie[2543954]:   File "/nix/store/z9a8pij846xx0yh29ygc6lk2m9iq6ch1-mealie-1.11.0/lib/python3.12/site-packages/mealie/services/backups_v2/alchemy_exporter.py", line 164, in restore
Aug 14 19:41:02 haddock mealie[2543954]:     command.upgrade(alembic_cfg, alembic_version)
Aug 14 19:41:02 haddock mealie[2543954]:   File "/nix/store/78sps7nkzmii8k6l9plwqj2c9lcs0qiz-python3.12-alembic-1.13.1/lib/python3.12/site-packages/alembic/command.py", line 383, in upgrade
Aug 14 19:41:02 haddock mealie[2543954]:     script = ScriptDirectory.from_config(config)
Aug 14 19:41:02 haddock mealie[2543954]:              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Aug 14 19:41:02 haddock mealie[2543954]:   File "/nix/store/78sps7nkzmii8k6l9plwqj2c9lcs0qiz-python3.12-alembic-1.13.1/lib/python3.12/site-packages/alembic/script/base.py", line 226, in from_config
Aug 14 19:41:02 haddock mealie[2543954]:     return ScriptDirectory(
Aug 14 19:41:02 haddock mealie[2543954]:            ^^^^^^^^^^^^^^^^
Aug 14 19:41:02 haddock mealie[2543954]:   File "/nix/store/78sps7nkzmii8k6l9plwqj2c9lcs0qiz-python3.12-alembic-1.13.1/lib/python3.12/site-packages/alembic/script/base.py", line 107, in __init__
Aug 14 19:41:02 haddock mealie[2543954]:     raise util.CommandError(
Aug 14 19:41:02 haddock mealie[2543954]: alembic.util.exc.CommandError: Path doesn't exist: '/nix/store/z9a8pij846xx0yh29ygc6lk2m9iq6ch1-mealie-1.11.0/alembic'.  Please use the 'init' command to create a new scripts folder.
Aug 14 19:41:02 haddock mealie[2543954]: ERROR    2024-08-14T19:41:02 - Path doesn't exist: '/nix/store/z9a8pij846xx0yh29ygc6lk2m9iq6ch1-mealie-1.11.0/alembic'.  Please use the 'init' command to create a new scripts folder.
Aug 14 19:41:02 haddock mealie[2543954]: Traceback (most recent call last):
Aug 14 19:41:02 haddock mealie[2543954]:   File "/nix/store/z9a8pij846xx0yh29ygc6lk2m9iq6ch1-mealie-1.11.0/lib/python3.12/site-packages/mealie/routes/admin/admin_backups.py", line 105, in import_one
Aug 14 19:41:02 haddock mealie[2543954]:     backup.restore(file)
Aug 14 19:41:02 haddock mealie[2543954]:   File "/nix/store/z9a8pij846xx0yh29ygc6lk2m9iq6ch1-mealie-1.11.0/lib/python3.12/site-packages/mealie/services/backups_v2/backup_v2.py", line 101, in restore
Aug 14 19:41:02 haddock mealie[2543954]:     self.db_exporter.restore(database_json)
Aug 14 19:41:02 haddock mealie[2543954]:   File "/nix/store/z9a8pij846xx0yh29ygc6lk2m9iq6ch1-mealie-1.11.0/lib/python3.12/site-packages/mealie/services/backups_v2/alchemy_exporter.py", line 164, in restore
Aug 14 19:41:02 haddock mealie[2543954]:     command.upgrade(alembic_cfg, alembic_version)
Aug 14 19:41:02 haddock mealie[2543954]:   File "/nix/store/78sps7nkzmii8k6l9plwqj2c9lcs0qiz-python3.12-alembic-1.13.1/lib/python3.12/site-packages/alembic/command.py", line 383, in upgrade
Aug 14 19:41:02 haddock mealie[2543954]:     script = ScriptDirectory.from_config(config)
Aug 14 19:41:02 haddock mealie[2543954]:              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Aug 14 19:41:02 haddock mealie[2543954]:   File "/nix/store/78sps7nkzmii8k6l9plwqj2c9lcs0qiz-python3.12-alembic-1.13.1/lib/python3.12/site-packages/alembic/script/base.py", line 226, in from_config
Aug 14 19:41:02 haddock mealie[2543954]:     return ScriptDirectory(
Aug 14 19:41:02 haddock mealie[2543954]:            ^^^^^^^^^^^^^^^^
Aug 14 19:41:02 haddock mealie[2543954]:   File "/nix/store/78sps7nkzmii8k6l9plwqj2c9lcs0qiz-python3.12-alembic-1.13.1/lib/python3.12/site-packages/alembic/script/base.py", line 107, in __init__
Aug 14 19:41:02 haddock mealie[2543954]:     raise util.CommandError(
Aug 14 19:41:02 haddock mealie[2543954]: alembic.util.exc.CommandError: Path doesn't exist: '/nix/store/z9a8pij846xx0yh29ygc6lk2m9iq6ch1-mealie-1.11.0/alembic'.  Please use the 'init' command to create a new scripts folder.
Aug 14 19:41:02 haddock mealie[2543954]: INFO     2024-08-14T19:41:02 - [192.168.1.1:0] 500 Internal Server Error "POST /api/admin/backups/mealie_2024.06.10.22.49.23.zip/restore HTTP/1.1"

Performing this operation actually borks the Mealie install too. Presumably because it's dropping the database before performing the backup :grimacing:.

litchipi commented 2 weeks ago

Alright I got something.

On the NixOS package, we configure the alembic.ini configuration to the folder where the scripts are located This config is ignored in the backup_v2/alchemy_exporter.py script, as it uses a blank config to setup the restoration. Substituting to the good path should fix the alchemy scripts path error

Try with:

package = pkgs.mealie.overrideAttrs (o: {
    postPatch = o.postPatch + ''
        substituteInPlace mealie/services/backups_v2/alchemy_exporter.py \
            --replace-fail '"script_location", path.join(PROJECT_DIR, "alembic")' '"script_location", "${o.src}/alembic"'
    '';
});

Let me know if that fixes the issue, I may open a PR in upstream for this if that resolves the issue

poperigby commented 2 weeks ago

@litchipi That totally worked! Thanks so much!

poperigby commented 2 weeks ago

@litchipi I'll close this now. I think an upstream PR would be a good idea :slightly_smiling_face:.