search

manticoresoftware / manticoresearch-backup

Repository for Manticore Search backup scripts
GNU General Public License v3.0
3 stars 1 forks source link

Can't restore outdated backup cause folder non exist #83

Closed djklim87 closed 10 months ago

djklim87 commented 11 months ago

I faced with interesting behavior.

If I created a dump with the old version of Buddy, I can't restore it because I haven't desired path.

For example in my dump has files inside /usr/share/manticore/modules/manticore-buddy/vendor/composer/composer/doc/... folder In newer versions, this folder doesn't exist. When I'll try to restore this backup I get an error 

2023-10-10 16:58:22 [Error] Failed to create directory – "/usr/share/manticore/modules/manticore-buddy/vendor/composer/composer/doc/fixtures/repo-composer-with-providers/p/bar"

I guess it's because we don't create folders, we just move files from the archive recursively.

djklim87 commented 11 months ago

backup-20231009160538.zip

sanikolaev commented 11 months ago

versions.json in this case:

{"manticore":"6.0.5 f77ce0e65@230524 dev","columnar":"0.0.0","secondary":"0.0.0"}
sanikolaev commented 11 months ago

Decided that we won't allow to restore in a version different from the one the backup was made in. There should be flag --force or such to force restore.

sanikolaev commented 10 months ago

Decided on call that we're going to follow this logic:

donhardman commented 10 months ago

Implemented in this pull: https://github.com/manticoresoftware/manticoresearch-backup/pull/85

djklim87 commented 10 months ago

Implemented in this pull: #85

And how I can test it? Maybe should you merge this into master and build deb/rpm package?

sanikolaev commented 10 months ago

I believe it's been waiting for my review.

https://github.com/manticoresoftware/manticoresearch-backup/pull/85 didn't pass the tests. Reassigning to @donhardman to look into that.

djklim87 commented 10 months ago

No it's still not working

2023-10-31T12:26:06.151872211Z + manticore-backup --version
2023-10-31T12:26:06.189049352Z Copyright (c) 2023, Manticore Software LTD (https://manticoresearch.com)
2023-10-31T12:26:06.189094711Z 
2023-10-31T12:26:06.200092228Z Manticore Backup version: 1.1.1

2023-10-31T12:26:06.206261426Z + manticore-backup --force --backup-dir=/tmp/ --restore=backup
2023-10-31T12:26:06.237071852Z Copyright (c) 2023, Manticore Software LTD (https://manticoresearch.com)
2023-10-31T12:26:06.237125927Z 
2023-10-31T12:26:06.247807871Z Manticore config file: 
2023-10-31T12:26:06.247846228Z Backup dir: /tmp/
2023-10-31T12:26:06.262314505Z 2023-10-31 12:26:06 [Info] Stored versions: {"manticore":"6.0.5 f77ce0e65@230524 dev","columnar":"0.0.0","secondary":"0.0.0"}
2023-10-31T12:26:06.262356709Z 2023-10-31 12:26:06 [Info] Current versions: {"manticore":"6.2.13 3e7105823@231031 dev","columnar":"2.2.5 b8be4eb@230928","secondary":"2.2.5 b8be4eb@230928"}
2023-10-31T12:26:06.262386221Z 2023-10-31 12:26:06 [Info] Warning: You try to restore backup of the different version. Use --force to bypass this.
2023-10-31T12:26:06.262396419Z 2023-10-31 12:26:06 [Info] Starting to restore...
2023-10-31T12:26:06.286697452Z 
2023-10-31T12:26:06.286742380Z Manticore config
2023-10-31T12:26:06.286753981Z   endpoint =  http://127.0.0.1:9308
2023-10-31T12:26:06.489484274Z 2023-10-31 12:26:06 [Info] Restoring config files...
2023-10-31T12:26:06.490296417Z 2023-10-31 12:26:06 [Info]   config files - OK
2023-10-31T12:26:06.490340594Z 2023-10-31 12:26:06 [Info] Restoring state files...
2023-10-31T12:26:06.603427020Z 2023-10-31 12:26:06 [Error] Failed to create directory – "/usr/share/manticore/modules/manticore-buddy/vendor/composer/composer/doc/fixtures/repo-composer-with-providers/p/bar"
donhardman commented 10 months ago

As per log I can see that it works fine. The issue is one of two:

  1. The target directory ALREADY exists. We need to restore on empty instance and the requirements are to have all restored files to be removed: /usr/share/manticore/modules/manticore-buddy
  2. No writing permission for user who try to restore there: /usr/share/manticore/modules/manticore-buddy

I need more information to check it out. Like permissions on directories, exactly steps to reproduce and so on.

djklim87 commented 10 months ago

Yes maybe you right. I tried to remove this files, start again and got new error:

2023-10-31T14:19:52.460561446Z + ls /usr/share/manticore/modules/manticore-buddy/vendor/
2023-10-31T14:19:52.463271950Z bin    evenement  justinrainbow  php-ds  react        seld
2023-10-31T14:19:52.463308998Z composer  fig         manticoresoftware  psr ringcentral  symfony

2023-10-31T14:19:52.463634979Z + manticore-backup --version
2023-10-31T14:19:52.504617802Z Copyright (c) 2023, Manticore Software LTD (https://manticoresearch.com)
2023-10-31T14:19:52.504674005Z 
2023-10-31T14:19:52.515813027Z Manticore Backup version: 1.1.1

2023-10-31T14:19:52.524603331Z + manticore-backup --force --backup-dir=/ --restore=docker-entrypoint-initdb.d
2023-10-31T14:19:52.565568030Z Copyright (c) 2023, Manticore Software LTD (https://manticoresearch.com)
2023-10-31T14:19:52.565641696Z 
2023-10-31T14:19:52.577415487Z Manticore config file: 
2023-10-31T14:19:52.577460920Z Backup dir: /
2023-10-31T14:19:52.598381462Z 2023-10-31 14:19:52 [Info] Stored versions: {"manticore":"6.0.5 f77ce0e65@230524 dev","columnar":"0.0.0","secondary":"0.0.0"}
2023-10-31T14:19:52.598465056Z 2023-10-31 14:19:52 [Info] Current versions: {"manticore":"6.2.13 3e7105823@231031 dev","columnar":"2.2.5 b8be4eb@230928","secondary":"2.2.5 b8be4eb@230928"}
2023-10-31T14:19:52.598492827Z 2023-10-31 14:19:52 [Info] Warning: You try to restore backup of the different version. Use --force to bypass this.
2023-10-31T14:19:52.598512193Z 2023-10-31 14:19:52 [Info] Starting to restore...
2023-10-31T14:19:52.628040172Z 
2023-10-31T14:19:52.628094583Z Manticore config
2023-10-31T14:19:52.628106164Z   endpoint =  http://127.0.0.1:9308
2023-10-31T14:19:52.942103941Z 2023-10-31 14:19:52 [Info] Restoring config files...
2023-10-31T14:19:52.943324840Z 2023-10-31 14:19:52 [Info]   config files - OK
2023-10-31T14:19:52.943360616Z 2023-10-31 14:19:52 [Info] Restoring state files...
2023-10-31T14:19:53.047612690Z 2023-10-31 14:19:53 [Error] Failed to create directory – "usr/share/manticore/modules/manticore-buddy/vendor/psr/log"

it looks like now it is trying to create a new path not from the system root folder

sanikolaev commented 10 months ago

@djklim87 how do I reproduce this from scratch? I can't reproduce it like this:

Install release:

root@b3e00313b77e:/# wget https://repo.manticoresearch.com/manticore-repo.noarch.deb
sudo dpkg -i manticore-repo.noarch.deb
sudo apt update
sudo apt install manticore manticore-extra

Searchd version:

root@b3e00313b77e:/# searchd -v
Manticore 6.2.12 dc5144d35@230822 (columnar 2.2.4 5aec342@230822) (secondary 2.2.4 5aec342@230822)
...

Create a test table and backup:

root@b3e00313b77e:/# systemctl start manticore
root@b3e00313b77e:/# mysql -P9306 -h0 -e "create table t"
root@b3e00313b77e:/# manticore-backup --backup-dir=/tmp/
Copyright (c) 2023, Manticore Software LTD (https://manticoresearch.com)

Manticore config file: /etc/manticoresearch/manticore.conf
Tables to backup: all tables
Backup dir: /tmp/

Manticore config
  endpoint =  http://127.0.0.1:9312

Manticore versions:
  manticore: 6.2.12 dc5144d35@230822
  columnar: 2.2.4 5aec342@230822
  secondary: 2.2.4 5aec342@230822

Manticore versions:
  manticore: 6.2.12 dc5144d35@230822
  columnar: 2.2.4 5aec342@230822
  secondary: 2.2.4 5aec342@230822
2023-10-31 16:03:11 [Info] Starting the backup...
2023-10-31 16:03:11 [Info] Backing up config files...
2023-10-31 16:03:11 [Info]   config files - OK
2023-10-31 16:03:11 [Info] Backing up global state files...
2023-10-31 16:03:11 [Info]   global state files – OK
2023-10-31 16:03:11 [Info] Backing up tables...
2023-10-31 16:03:11 [Info]   t (rt) [481B]...
2023-10-31 16:03:11 [Info]    OK
2023-10-31 16:03:11 [Info] Running sync
2023-10-31 16:03:11 [Info]  OK
2023-10-31 16:03:11 [Info] You can find backup here: /tmp/backup-20231031160311
2023-10-31 16:03:11 [Info] Elapsed time: 0.32s
2023-10-31 16:03:12 [Info] Done
2023-10-31 16:03:12 [Info]
Unfreezing all tables...
2023-10-31 16:03:12 [Info]   t...
2023-10-31 16:03:12 [Info]    OK

Upgrade Manticore to a dev version:

wget https://repo.manticoresearch.com/manticore-dev-repo.noarch.deb && \
sudo dpkg -i manticore-dev-repo.noarch.deb && \
sudo apt -y update && \
sudo apt -y install manticore manticore-extra manticore-common manticore-server manticore-server-core manticore-tools manticore-executor manticore-buddy manticore-backup manticore-columnar-lib manticore-server-core-dbgsym manticore-tools-dbgsym manticore-columnar-lib-dbgsym manticore-icudata-65l

searchd version:

root@b3e00313b77e:/# searchd -v
Manticore 6.2.13 3e7105823@231031 dev (columnar 2.2.5 b8be4eb@230928) (secondary 2.2.5 b8be4eb@230928)

Restore:

root@b3e00313b77e:/# systemctl stop manticore

root@b3e00313b77e:/# manticore-backup --backup-dir=/tmp/ --restore
Copyright (c) 2023, Manticore Software LTD (https://manticoresearch.com)

Manticore config file:
Backup dir: /tmp/

Available backups: 1
  backup-20231031160311 (Oct 31 2023 16:03:11)

root@b3e00313b77e:/# rm /etc/manticoresearch/manticore.conf /var/lib/manticore/manticore.json

root@b3e00313b77e:/# manticore-backup --backup-dir=/tmp/ --restore=backup-20231031160311 --force
Copyright (c) 2023, Manticore Software LTD (https://manticoresearch.com)

Manticore config file:
Backup dir: /tmp/
2023-10-31 16:07:35 [Info] Stored versions: {"manticore":"6.2.12 dc5144d35@230822 ","columnar":"2.2.4 5aec342@230822","secondary":"2.2.4 5aec342@230822"}
2023-10-31 16:07:35 [Info] Current versions: {"manticore":"6.2.13 3e7105823@231031 dev","columnar":"2.2.5 b8be4eb@230928","secondary":"2.2.5 b8be4eb@230928"}
2023-10-31 16:07:35 [Info] Warning: You are trying to restore a backup from a different version. Use --force to bypass this restriction.
2023-10-31 16:07:35 [Info] Starting to restore...

Manticore config
  endpoint =  http://127.0.0.1:9312
2023-10-31 16:07:35 [Info] Restoring config files...
2023-10-31 16:07:35 [Info]   config files - OK
2023-10-31 16:07:35 [Info] Restoring state files...
2023-10-31 16:07:35 [Info]   state files - OK
2023-10-31 16:07:35 [Info] Restoring data files...
2023-10-31 16:07:35 [Info]   tables' files - OK
2023-10-31 16:07:35 [Info] The backup '/tmp/backup-20231031160311' was successfully restored.
2023-10-31 16:07:35 [Info] Elapsed time: 0.03s
2023-10-31 16:07:35 [Info] Done
root@b3e00313b77e:/#
djklim87 commented 10 months ago

No, problem still available I tested at our linux_dockers

apt-get update && apt-get install -y wget gnupg unzip && \
wget https://repo.manticoresearch.com/manticore-dev-repo.noarch.deb && sudo dpkg -i manticore-dev-repo.noarch.deb && sudo apt -y update && sudo apt -y install manticore manticore-extra manticore-common manticore-server manticore-server-core manticore-tools manticore-executor manticore-buddy manticore-backup manticore-columnar-lib manticore-server-core-dbgsym manticore-tools-dbgsym manticore-columnar-lib-dbgsym manticore-icudata-65l && \
wget https://github.com/manticoresoftware/manticoresearch-backup/files/12859973/backup-20231009160538.zip && unzip backup-20231009160538.zip && \
manticore-backup --version && \
find /backup-20231009160538/config -type f -exec sh -c 'rm -f "${1#/backup-20231009160538/config}"' sh {} \; &&\
find /backup-20231009160538/state -type f -exec sh -c 'rm -f "${1#/backup-20231009160538/state}"' sh {} \; &&\
manticore-backup --force --backup-dir=/ --restore=backup-20231009160538

Result

Copyright (c) 2023, Manticore Software LTD (https://manticoresearch.com)

Manticore Backup version: 1.1.1
Copyright (c) 2023, Manticore Software LTD (https://manticoresearch.com)

Manticore config file: 
Backup dir: /
2023-11-01 12:03:37 [Info] Stored versions: {"manticore":"6.0.5 f77ce0e65@230524 dev","columnar":"0.0.0","secondary":"0.0.0"}
2023-11-01 12:03:37 [Info] Current versions: {"manticore":"6.2.13 3e7105823@231031 dev","columnar":"2.2.5 b8be4eb@230928","secondary":"2.2.5 b8be4eb@230928"}
2023-11-01 12:03:37 [Info] Warning: You are trying to restore a backup from a different version. Use --force to bypass this restriction.
2023-11-01 12:03:37 [Info] Starting to restore...

Manticore config
  endpoint =  http://127.0.0.1:9308
2023-11-01 12:03:37 [Info] Restoring config files...
2023-11-01 12:03:37 [Info]   config files - OK
2023-11-01 12:03:37 [Info] Restoring state files...
2023-11-01 12:03:37 [Error] Failed to create directory – "usr/share/manticore"
donhardman commented 10 months ago

The problem arises from an old backup that was created with an incompatible version. We plan to rectify this in future releases by also storing the backup version. However, for now, we recommend creating and restoring backups using the same Manticore-backup version.

djklim87 commented 10 months ago

Looks like all fine now