Simplistic Qt GUI for borgbackup

vduseev commented 7 years ago

Idea of GUI client

Create a simple desktop application that executes borg behind the scenes and runs the very basics of backups.

Goal

Aimed at users with no understanding of backup tools, directory structures, ssh, backup algorithms, etc. Allow them to run their favorite backup tool on any platform.

Proposed tech stack

PyQt 5 (Qt framework)
QML for UI
borg executed as subprocess underneath

Upd (8/24/2017):

require borg >= 1.1.0rc1 (and later >= 1.1.0)
use --json(-lines)

Basic requirements

Backup files to external hard drive
Run on every major platform: ~~Windows~~, MacOS X, Linux, FreeBSD?

Available target platforms for Qt

Note: Qt is not targeting FreeBSD as of latest 5.9 version

Platform
Windows 7, 8.1, 10 (32/64-bit)
openSUSE 42.1 (64-bit)
Red Hat Enterprise Linux 6.6, 7.2 (64-bit)
Ubuntu 16.04 (64-bit)
(Linux 32/64-bit)
macOS 10.10, 10.11, 10.12
Embedded Linux
QNX 6.6.0, 7.0 (armv7le and x86)
INTEGRITY 11.4.x
Universal Windows Platform (UWP) (x86, x86_64, armv7)
Android (API Level: 16)

Development plan

I'd like to start on this project and create a separate repository for the Desktop GUI. I suppose "borgbackup" can then fork my repository.

Looking forward to any ideas and comments about requirements for such desktop client.

m3nu commented 5 years ago

Thanks for checking it out @enkore . I read up on src-based layouts and see the benefits. Will change it today.

m3nu commented 5 years ago

Some progress today:

move to src-based setup
add first tests (qt-style)
add scheduler
add rule to choose allowed WIFIs (the only macOS-specific feature currently)
save repo passwords to keychain

Too late for a proper release today, but hoping to get it in front of beta users in a few days.

Any other features that could be worth including?

mspeder commented 5 years ago

Tried to install vorta different ways. Unfortunately all failed.

OSX prebuilt binary gives

  File "vorta/application.py", line 20, in __init__
  File "vorta/scheduler.py", line 14, in __init__
  File "vorta/scheduler.py", line 19, in reload
AttributeError: can't set attribute
[93565] Failed to execute script __main__
Sentry is attempting to send 1 pending error messages

OSX from git clone gives

    load_entry_point('vorta', 'gui_scripts', 'vorta')()
  File "/usr/local/lib/python3.7/site-packages/pkg_resources/__init__.py", line 484, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/usr/local/lib/python3.7/site-packages/pkg_resources/__init__.py", line 2714, in load_entry_point
    return ep.load()
  File "/usr/local/lib/python3.7/site-packages/pkg_resources/__init__.py", line 2332, in load
    return self.resolve()
  File "/usr/local/lib/python3.7/site-packages/pkg_resources/__init__.py", line 2338, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
ModuleNotFoundError: No module named 'vorta'

Linux from pip install gives pretty much the same

    load_entry_point('vorta==0.2.3', 'gui_scripts', 'vorta')()
  File "/usr/lib/python3.7/site-packages/pkg_resources/__init__.py", line 484, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/usr/lib/python3.7/site-packages/pkg_resources/__init__.py", line 2725, in load_entry_point
    return ep.load()
  File "/usr/lib/python3.7/site-packages/pkg_resources/__init__.py", line 2343, in load
    return self.resolve()
  File "/usr/lib/python3.7/site-packages/pkg_resources/__init__.py", line 2349, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
ModuleNotFoundError: No module named 'vorta'

@m3nu any idea ?

m3nu commented 5 years ago

Hi Matthieu, thanks for testing.

It seems I caused some packaging issues after trying to put too much in setup.cfg. When I moved some of the options back to setup.py, it worked as expected.

Regarding the issue on macOS, you may have an old version. Unfortunately I didn't include a version number in my Sentry error reports before. I've changed that now. Since your errors first appeared 2 days ago, that may be the reason. Did you try the latest build from Github?

To summarize:

the Python packaging issue should be fixed. Latest version 0.2.6 is on pypi. Tested with tox and some conda-envs. Before I got the same error as you, when not using a develop-install.
the latest macOS version should also work. Your may have been 1-2 days old. If the error happens again, I will get full version details at least. Tested in a macOS guest account, but will test on a different machine or VM as well. I'm still in the process of setting this up.

mspeder commented 5 years ago

Hi Manuel

Thanks for your quick answer ! I downloaded latest OSX binary and I confirm the app now launches successfully.

OSX from pip package works too (got version 0.2.8).

Linux from pip package (got version 0.2.8) now fails with a message complaining about the absence of file /Library/Preferences/SystemConfiguration/com.apple.airport.preferences.plist :-)

Now that I have a version that works, I did some testing on OSX binary. Overall, Vorta looks very promising to me. I especially like to have it directly in the menubar (a different icon while backup is running would be a nice addition). Thanks for the very good work you did !

Here are some unexpected behaviors I spotted (maybe some of them are simply not-yet-implemented) :

when the backup is finished, the cancel button remains enabled
on snapshots view : delete and refresh button are always disabled (hence can't delete any snapshot manually)
on schedule view : prune checkbox has status "mixed values"/undefined each time I relaunch the app
is pruning implemented yet ? I never had any of my snapshots pruned (tested multiple shots with prune config = 1 hourly)
is there a way to clear the log ?

Thanks again !

m3nu commented 5 years ago

Good to have it working (sort of) now. I'll look into the stuff you noticed and everything that came in on Sentry. Another user was testing on macOS as well. In short,

It should be cross-platform and the Wifi-list is the only thing Mac-specific for now. I'll make it fail gracefully and then find a way to get the Wifi-list on Linux if it exists somewhere.
Pruning, deleting snapshots and repo-checking isn't implemented yet. I left a note about it in the README and will add it soon.

For the smaller things you reported, I'll have an update out later today.

I now realize that a desktop-app is much harder than a web-app because debugging and updating is more tricky. Still finding the best ways to tackle this.

m3nu commented 5 years ago

One more note since I saw some database-errors: I added database schema migrations a few versions ago. Settings DBs created before this change won't fully work when leaving out some versions.

You probably noticed this and deleted ~/Library/Application Support/Vorta (or the equivalent on Linux). This only affects those who installed an early version and then went to a very recent one. In all newer builds the database is versioned and will migrate the schema before starting.

m3nu commented 5 years ago

Collecting GUI-related bugs here: https://github.com/borgbase/vorta/issues

m3nu commented 5 years ago

Large update to support multiple profiles and local repos: https://github.com/borgbase/vorta/releases/tag/v0.4.0

Looking to focus on bug-fixes and getting Linux-support into better shape over the next days.

MTrage commented 5 years ago

After a long time of work I am now handing over a completely new version of BORG BackUP GUI.

Compared to the previous version the version 0.8 is extremely extended and automated to make it possible for a new version to do everything via the GUI.

At the first start you will be asked to create a new BackUP

new-archiv

Create a new snapshot

new-snapshot

A detailed overview (with videos) can be found in the Wiki section: https://github.com/MTrage/Borg-BackUP-GUI/wiki/Short-instruction

MTrage commented 5 years ago

In order to make it easier for other developers to use BORG BackUP GUI as a basis for their own ideas, the license model was changed to GPL-3.0.

MTrage commented 5 years ago

Conversion to the free selection of Qt5 themes.

qt5-style-min

A detailed overview (with videos) can be found in the Wiki section: https://github.com/MTrage/Borg-BackUP-GUI/wiki/Short-instruction

Nebucatnetzer commented 5 years ago

Just to let everyone know that I've started my diploma project today. However due to the fact that so many now have started to create a GUI I unsubscribed from this issue. It's just too tempting to look at the code you guys create :).

However I will post here from time to time to keep you updated how it is going. At minimum when I start the main repository with the code.

MTrage commented 5 years ago

@Nebucatnetzer, my intuition was to provide a basis on which to build or develop.

Nevertheless I am curious what you will develop in the future. I keep my fingers crossed for you and wish you a lot of fun in realizing your new goal.

Greetings Marc

Nebucatnetzer commented 5 years ago

@MTrage, I hope you understood your post correctly. I'm still going to develop the GUI for borg as part of the project. However because there are now so many similar projects making the rounds I would like to minimize the chance that I copy of you guys.

After I'm finished with the project I'm more than happy to work with you guys :)

MTrage commented 5 years ago

@Nebucatnetzer copy it, I won't tell anyone! °D Borg Backup offers so many features that you can still use or embed, many of them I didn't use either. Either way I am curious about your work.

henfri commented 5 years ago

@Nebucatnetzer there is nothing bad copying something. Even during a diploma thesis. There is no point reinventing the wheel.

Greetings, Hendrik

Nebucatnetzer commented 5 years ago

@henfri sure to a certain point you're right.

Anyway, I've created now the main repository where the project will live. You can find it here: https://github.com/borg-qt/borg-qt

Nebucatnetzer commented 5 years ago

Hey guys,

Another small update from me. As promissed I translated all the targets for the application. You can find them here: https://github.com/borg-qt/borg-qt/blob/master/docs/todos.org

In addition here are some screenshots of the UI. At the moment the settings part is working. Next I'm going to start working on the part which controls BorgBackup.

Two strangest things so far have been that configparser doesn't provide the functionality to read/write lists and the QtListWidget doesn't have a method which returns all it's items in a list. However for both of them I found workarounds.

Nebucatnetzer commented 5 years ago

Hey guys another little update from my side. I've uploaded now the first binary. For the moment I'm done with coding and have to focus on finalizing the thesis.

Currently it contains the following functionality: Creating, restoring, deleting and mounting archives. Graphical configuration for the provided options and for me most importantly background backups through systemd timers.

There is a lot of room for improvement and I'm looking forward to all constructive inputs so that I can improve the application further after the thesis is finished.

screenshot from 2019-02-25 10-58-46 borgqt_progress_v2 borgqt_settings_exclude_v2 borgqt_settings_schedule