Add absolute path support for restore attr

[djklim87]

Now to restore the backup we need to use a nested path consisting from backup-dir and restore attribute:

manticore-backup --backup-dir=/tmp --restore=mybackup

We need to add absolute path support for restore attr, omitting backup-dir in that case:

manticore-backup --restore=/tmp/mybackup

[sanikolaev]

This issue blocks https://github.com/manticoresoftware/docker/issues/44

[sanikolaev]

@djklim87 Have you tried --backup-dir=/ --restore=backup for the case when the specific backup dir is mapped to /backup/ inside the container? Is there any issue with this?

[djklim87]

No it wasn't helpful

$ ls /docker-entrypoint-initdb.d
config  data  state  versions.json

$ manticore-backup --backup-dir=/ --restore=docker-entrypoint-initdb.d
Copyright (c) 2023, Manticore Software LTD (https://manticoresearch.com)

2023-10-18 22:27:34 [Error] Failed to find backup dir to store backup: /

[donhardman]

We've verified that the path is accessible and writable, which are necessary for proper functioning. In your instance, you've designated the backup-path as root. However, from the $ symbol in your shell log, it appears you're a non-root user. If you wish to access it as a non-root user, it would be advisable not to use /, but rather mount it somewhere where a non-root user could also access it. This solution could potentially resolve the issue. What's the purpose of accessing the backup from the root as a non-root user? Shouldn't we instead access it from a location that a non-root user can reach?

[sanikolaev]

As discussed on call with Dmitrii:

• @donhardman will change the logic, so we don't check if the dir is writable in case of --restore
• @djklim87, /docker-entrypoint-initdb.d seems to be too long dir name, let's shorten it.

[donhardman]

Please try the latest version from the main branch and if it works fine, close the issue

[sanikolaev]

@djklim87, /docker-entrypoint-initdb.d seems to be too long dir name, let's shorten it.

It's a common practice. This name is used in mysql, postgres and clickhouse. Let's use the same name too.

[djklim87]

I can conclude that backups now is broken:

root@0ba3c4818cbf:/var/lib/manticore# manticore-backup --backup-dir=/tmp
Copyright (c) 2023, Manticore Software LTD (https://manticoresearch.com)

Manticore config file: /etc/manticoresearch/manticore.conf
Tables to backup: all tables
Backup dir: /tmp

Manticore config
  endpoint =  http://$ip:9312
2023-10-25 19:17:57 [Error] Failed to send query to the manticoresearch daemon. Make sure it's listening for HTTP or HTTPS connections (with proper certificates).

root@0ba3c4818cbf:/var/lib/manticore# sh /etc/manticoresearch/manticore.conf | grep listen
    # https://manual.manticoresearch.com/Server_settings/Searchd#listen_backlog
    # listen_backlog = 5
    # https://manual.manticoresearch.com/Server_settings/Searchd#listen
    # listen_env = this directive allows to append listeners from environment variables
    listen = 9306:mysql41
    listen = /var/run/mysqld/mysqld.sock:mysql41
    listen = 172.17.0.2:9312
    listen = 9308:http
    listen = 172.17.0.2:9315-9325:replication
    # https://manual.manticoresearch.com/Server_settings/Searchd#listen_tfo
    # listen_tfo = 0

[donhardman]

I used official docker image and it works fine

root@5f7eba731fa9:/var/lib/manticore# manticore-backup --backup-dir=/tmp
Copyright (c) 2023, Manticore Software LTD (https://manticoresearch.com)

Manticore config file: /etc/manticoresearch/manticore.conf
Tables to backup: all tables
Backup dir: /tmp

Manticore config
  endpoint =  http://172.17.0.2:9312

Manticore versions:
  manticore: 6.2.13 fd92716d2@231026 dev
  columnar: 0.0.0
  secondary: 0.0.0

Manticore versions:
  manticore: 6.2.13 fd92716d2@231026 dev
  columnar: 0.0.0
  secondary: 0.0.0
2023-10-26 11:12:44 [Info] Starting the backup...
2023-10-26 11:12:44 [Error] You have no tables to backup.
root@5f7eba731fa9:/var/lib/manticore#

Please provide more info

[djklim87]

No it's still not working

2023-10-30T16:25:42.097509134Z + ls -la /docker-entrypoint-initdb.d
2023-10-30T16:25:42.100587736Z total 24
2023-10-30T16:25:42.100629582Z drwxr-xr-x 5 1000 1000 4096 Oct 30 13:51 .
2023-10-30T16:25:42.100637058Z drwxr-xr-x 1 root root 4096 Oct 30 16:25 ..
2023-10-30T16:25:42.100643952Z drwxr-xr-x 4 1000 1000 4096 Oct 30 13:51 config
2023-10-30T16:25:42.100650285Z drwxr-xr-x 3 1000 1000 4096 Oct 30 13:51 data
2023-10-30T16:25:42.100658211Z drwxr-xr-x 2 1000 1000 4096 Oct 30 13:51 state
2023-10-30T16:25:42.100664704Z -rw-r--r-- 1 1000 1000  113 Oct 30 13:51 versions.json

Version check

2023-10-30T16:25:42.101128918Z + manticore-backup --version
2023-10-30T16:25:42.160556834Z Copyright (c) 2023, Manticore Software LTD (https://manticoresearch.com)
2023-10-30T16:25:42.160636367Z 
2023-10-30T16:25:42.175068496Z Manticore Backup version: 1.0.13

Run restore

2023-10-30T16:25:42.181046062Z + manticore-backup --restore='/docker-entrypoint-initdb.d/'
2023-10-30T16:25:42.216630081Z Copyright (c) 2023, Manticore Software LTD (https://manticoresearch.com)
2023-10-30T16:25:42.216682849Z 
2023-10-30T16:25:42.227155457Z 2023-10-30 16:25:42 [Error] Failed to find backup dir to store backup: none

[donhardman]

As previously discussed, please try the following command:

manticore-backup --backup-dir=/ --restore=docker-entrypoint-initdb.d

We've implemented a fix to the existing logic, but we haven't completely altered the logic to eliminate the --backup-dir parameters in the request.