[nextcloud] calendar/contacts backup

[cyberb]

There is a tool for the backup which does not seem to work with Syncloud: https://codeberg.org/BernieO/calcardbackup

We can either provide an instruction or even include it into the app and expose a command.

[cyberb]

git clone https://codeberg.org/BernieO/calcardbackup.git
cd calcardbackup
sudo chown -R nextcloud:nextcloud .
mkdir backup
sudo chown -R nextcloud:nextcloud backup
sudo -u nextcloud ./calcardbackup "/var/snap/nextcloud/common/" -o backup

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
+
+  Wed May 27 13:22:39 UTC 2020 --> START calcardbackup ver. 1.5.0 (27.04.2020), AGPL-3.0
+  Checking dependencies and preparing...
+  no usersfile given:
+     - will fetch all available items from database
-- calcardbackup: ERROR --
-- ERROR: can't write files into '/root/calcardbackup/backup'.
-- I need to be able to have the according permissions for the directory (+wx). Check given path!
-- calcardbackup: Exiting.

[gb4oss]

Thanks cyberb! I see, what I can contribute.

[cyberb]

did you have the same output, can you paste yours?

[gb4oss]

I basically had the same output. git is missing on my installation and I scp-ed the repository there. For the permission problem above, it seems to be, because the path is '/root/calcardbackup/backup', where '/root/' has no permissions for the "nextcloud" user. I move everything to '/calcardbackup/backup' and get this:

root@raspberrypi3:/calcardbackup# sudo -u nextcloud ./calcardbackup "/var/snap/nextcloud/common/" -o backup

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
+
+  Thu May 28 13:51:55 UTC 2020 --> START calcardbackup ver. 1.5.0 (27.04.2020), AGPL-3.0
+  Checking dependencies and preparing...
+  no usersfile given:
+     - will fetch all available items from database
./calcardbackup: line 568: [[: );: syntax error: operand expected (error token is ");")
./calcardbackup: line 569: [[: );: syntax error: operand expected (error token is ");")
+  Using URL: );
+  can't read status.php (cURL code '6').
./calcardbackup: line 830: [[: );: syntax error: operand expected (error token is ");")
./calcardbackup: line 832: [[: );: syntax error: operand expected (error token is ");")
+  can't detect vendor.
+  Server ); detected.
-- calcardbackup: ERROR --
-- ERROR: Unsupported Database type: );
-- Only MySQL/MariaDB, SQLite3 and PostgreSQL are supported.
-- calcardbackup: Exiting.

[gb4oss]

Next I tried the "-p" option for nextcloud-snap:

root@raspberrypi3:/calcardbackup# sudo -u nextcloud ./calcardbackup "/var/snap/nextcloud/common/" -o backup -p

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
+
+  Thu May 28 13:56:09 UTC 2020 --> START calcardbackup ver. 1.5.0 (27.04.2020), AGPL-3.0
+  Checking dependencies and preparing...
+  no usersfile given:
+     - will fetch all available items from database
./calcardbackup: line 440: nextcloud.mysql-client: command not found
-- calcardbackup: ERROR --
-- ERROR: nextcloud-snaps cli-utility 'nextcloud.mysql-client' not found!
-- To work with nextcloud-snap this script needs to be invoked  with sudo (even when root).
-- If the script has been invoked with sudo: are you sure you are using nextcloud-snap?
-- calcardbackup: Exiting.

It seems not to know about the nextcloud.pgsql command, right?

[cyberb]

Ok there is a fix:

cd /tmp
git clone https://codeberg.org/BernieO/calcardbackup.git
cd calcardbackup
mkdir backup
sudo chown -R nextcloud:nextcloud .
sed 's/psql/snap run nextcloud\.psql/g' -i calcardbackup
sed 's/nextcloud.mysql-client/snap run nextcloud\.psql/g' -i calcardbackup
sudo -u nextcloud ./calcardbackup "/var/snap/nextcloud/current/nextcloud" -o backup

Gave me the result:

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
+
+  Thu May 28 17:20:43 UTC 2020 --> START calcardbackup ver. 1.5.0 (27.04.2020), AGPL-3.0
+  Checking dependencies and preparing...
+  no usersfile given:
+     - will fetch all available items from database
+  Using URL: http://localhost
+  no valid status.php found at http://localhost.
+  Nextcloud 18.0.1 detected.
+  Database of chosen Nextcloud installation is PostgreSQL.
+  Looking for calendars in your Nextcloud:
+  Saving calendar test-Personal.ics (from db)......success!
+  Looking for calendarsubscriptions in your Nextcloud:
+  No calendarsubscriptions found.
+  Looking for addressbooks in your Nextcloud:
+  Saving addressbook test-Contacts.vcf (from db)......success!
+  Compressing backup as *.tar.gz file. Be patient - this may take a while.
+  Backup successfully compressed!
+  Find your compressed backup here: /tmp/calcardbackup/backup/calcardbackup-2020-05-28.tar.gz
+  Thu May 28 17:20:45 UTC 2020 --> END calcardbackup
+
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

[cyberb]

Updated the wiki: https://github.com/syncloud/platform/wiki/Nextcloud#export-calendarscontacts

[cyberb]

@gb4oss did it work for you?

[gb4oss]

@cyberb Yes! It did! Thanks a lot!!! I had to import the files over the GUI and it worked perfectly fine! I had to install git thou, which seems to be missing on a plain installation. What's next?

[BernieO]

@gb4oss contacted me. Nice you found a workaround so my script also works with syncloud.

The command to access the database is snap run nextcloud.psql?

Is there any way to find out whether the installation is a syncloud installation? If yes, I could directly implement the command to the script.

Actually I think the snap-switch (-p) and the second sed command shouldn't be needed, as long as the database is PostgreSQL and my script can read Nextclouds config.php

[gb4oss]

@BernieO Just a thought: would the other way around also possible: manually using the snap-switch and the automatically detect the database-type?

[cyberb]

Actually I think the snap-switch (-p) and the second sed command shouldn't be needed, as long as the database is PostgreSQL and my script can read Nextclouds config.php

Config is here: /var/snap/nextcloud/current/nextcloud/config/config.php @BernieO Could you suggest a command?

[cyberb]

The command to access the database is snap run nextcloud.psql?

Yes, I think we should actually follow the official way of doing this and add on our side: export PATH=$PATH:/snap/bin so the command should become just: nextcloud.psql

[cyberb]

I had to install git thou, which seems to be missing on a plain installation.

@gb4oss which gui?

[BernieO]

would the other way around also possible: manually using the snap-switch and the automatically detect the database-type?

Yes, that is a better.

[BernieO]

Another question as I am not very experienced with snap: Do I have to pass parameters to nextcloud.psql (e.g. host, port, username, password)? Or are all necessary parameters to connect to PostgreSQL-Snap somehow inside the snap?

[gb4oss]

@gb4oss which gui?

So I needed to install git for the export. git was not necessary for the import. I just tried the NextCloud Webinterface for the import first and this worked fine and needed no git. Does this answer your question or am I getting the question wrong?

[cyberb]

@gb4oss which gui?

So I needed to install git for the export. git was not necessary for the import. I just tried the NextCloud Webinterface for the import first and this worked fine and needed no git. Does this answer your question or am I getting the question wrong?

Oh sorry I misread gut as gui :)

[cyberb]

Another question as I am not very experienced with snap: Do I have to pass parameters to nextcloud.psql (e.g. host, port, username, password)? Or are all necessary parameters to connect to PostgreSQL-Snap somehow inside the snap?

Port and host (Unix socket in our case) are preset: https://github.com/syncloud/nextcloud/blob/master/bin/psql.sh

[BernieO]

Thanks

Due to https://github.com/syncloud/nextcloud/blob/master/bin/psql.sh#L14-L18 I guess the unix user is authenticated via peer authentication to Postgres and thus doesn‘t need a username and password to gain access to the DB. Can you confirm this @cyberb ?

[cyberb]

yes, but I think you can still provide username and password, here is how we pass a username (we never pass a password, but I am not sure about Nextcloud probably it does): https://github.com/syncloud/nextcloud/blob/master/hooks/postgres.py#L48

[BernieO]

I added support for PostgreSQL in a snap installation with ff8322ec to the development branch.

@cyberb @gb4oss could you please check whether that solves the issue for you? Here is the direct link to the raw-file: calcardbackup development branch

[gb4oss]

@BernieO Tested with and without "-p". In both cases, it fails to detect the database type.

 sudo -u nextcloud ./calcardbackup "/var/snap/nextcloud/common/" -p -o backup

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
+
+  Sat May 30 07:10:22 UTC 2020 --> START calcardbackup ver. 1.5.0-1 (29.05.2020), AGPL-3.0
+  Checking dependencies and preparing...
+  no usersfile given:
+     - will fetch all available items from database
./calcardbackup: line 564: [[: );: syntax error: operand expected (error token is ");")
./calcardbackup: line 565: [[: );: syntax error: operand expected (error token is ");")
+  Using URL: );
+  can't read status.php (cURL code '6').
./calcardbackup: line 826: [[: );: syntax error: operand expected (error token is ");")
./calcardbackup: line 828: [[: );: syntax error: operand expected (error token is ");")
+  can't detect vendor.
+  Server ); detected.
-- calcardbackup: ERROR --
-- ERROR: Unsupported Database type: );
-- Only MySQL/MariaDB, SQLite3 and PostgreSQL are supported.
-- calcardbackup: Exiting.

An export PATH=$PATH:/snap/bin had apparently no effect.

[BernieO]

Please try again with this path:

sudo -u nextcloud ./calcardbackup /var/snap/nextcloud/current/nextcloud -p -o backup

[gb4oss]

Yeah, thanks for the hint! It is better now, but still something is missing. These are my preparations:

cd /tmp
git clone https://codeberg.org/BernieO/calcardbackup.git
cd calcardbackup
mv calcardbackup calcardbackup_bak 
wget https://codeberg.org/BernieO/calcardbackup/raw/branch/development/calcardbackup
mkdir backup
sudo chown -R nextcloud:nextcloud .
chmod +x calcardbackup
export PATH=$PATH:/snap/bin

I tested that nextcloud.psql is in the path of the current bash, but this seems not to be sufficient. I still get:

sudo -u nextcloud ./calcardbackup /var/snap/nextcloud/current/nextcloud -p -o backup

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
+
+  Sat May 30 12:08:19 UTC 2020 --> START calcardbackup ver. 1.5.0-1 (29.05.2020), AGPL-3.0
+  Checking dependencies and preparing...
+  no usersfile given:
+     - will fetch all available items from database
+  Using URL: http://localhost
+  no valid status.php found at http://localhost.
+  Nextcloud 16.0.5 detected.
./calcardbackup: line 991: nextcloud.psql: command not found
-- calcardbackup: ERROR --
-- ERROR: Nextcloud database is PostgreSQL. Therefore this script requires the PostreSQL command line client to be found in PATH.
-- calcardbackup: Exiting.

[BernieO]

Sorry, sudo strips environment variables. It needs to be:

sudo -u nextcloud PATH="${PATH}:/snap/bin" ./calcardbackup /var/snap/nextcloud/current/nextcloud -p -o backup

[cyberb]

This worked!

[cyberb]

@BernieO can you merge this change to the master branch so we can update the instruction and remove sed?

[BernieO]

@cyberb Sorry for the delay. I merged the change to master just now. You may proceed.

[cyberb]

@BernieO np, thanks for the fix, instruction is updated: https://github.com/syncloud/platform/wiki/Nextcloud#export-calendarscontacts

to this:

cd /tmp
git clone https://codeberg.org/BernieO/calcardbackup.git
cd calcardbackup
mkdir backup
sudo chown -R nextcloud:nextcloud .
sudo -u nextcloud PATH="${PATH}:/snap/bin" ./calcardbackup /var/snap/nextcloud/current/nextcloud -p -o backup

I see you have info for Synology users, maybe you could have a section for Syncloud users :)

[BernieO]

I see you have info for Synology users, maybe you could have a section for Syncloud users :)

Good idea - I will add that.
Cheers!

[BernieO]

As information:
a section for Syncloud users has been added to the READMEs of calcardbackup with this commit.

[cyberb]

@BernieO thank you very much!