ubports / ubports-installer

A simple tool to install Ubuntu Touch on UBports devices
https://github.com/ubports/ubports-installer/releases
GNU General Public License v3.0
546 stars 120 forks source link

Backups #141

Open Flohack74 opened 6 years ago

Flohack74 commented 6 years ago

While thinking what to do for a better backup (my magic-device-tool-backup is not the best option so far) I thought that maybe the installer could be extended into a multi-purpose-tool for install, backup and update.

We see that Android for example has much better update support for the firmware, since most vendors have kind of a notifier app to remind users of updates, and then they also package firmware updates with normal OS updates. If we fall behind with firmware, our users might face issues that could easily be solved, especially in the radio part. Its also the only way to flash a firmware update for users that are afraid to use fastboot or cannot do it due to their skills limitation.

The same goes for the backup. I would throw in my ideas about the backup reqirements, but unfortunately I am no good node dev, so I cant implement it ;)

NeoTheThird commented 6 years ago

Yep, these are features that I wanted to work on as well, but not before some of the refactoring in #58 is done. The code complexity has to decrease significantly before we should think about adding new features.

NeoTheThird commented 5 years ago

So, i thought a little about how to best do this. The simplest way is probably to tar the home folder on the device and pull it using adb. That file can be stored either in the installer cache dir, or saved in an arbitrary location on the pc. To restore, push the file to the device and untar it over the home folder. Here's the adb commands:

# Backup
adb shell rm /tmp/backup.tar.gz || echo ""
adb shell tar -cvpzf /tmp/backup.tar.gz --exclude=/userdata/user-data/phablet/.cache/upstart --exclude=/userdata/user-data/phablet/.cache/*/qmlcache --exclude=/userdata/user-data/phablet/.cache/*/qml_cache /userdata/user-data/phablet
adb pull /tmp/backup.tar.gz ./backup.tar.gz
# Restore
adb push ./backup.tar.gz /tmp/backup.tar.gz
adb shell tar -xvpzf /tmp/backup.tar.gz -C / --numeric-owner
NeoTheThird commented 5 years ago

And, for the record, firmware upgrades are already happening.

Flohack74 commented 5 years ago

I agree except that it could run low on diskspace if we are packing on the device, maybe we can redirct the gzip stream directly to the installer?

NeoTheThird commented 5 years ago

I agree except that it could run low on diskspace if we are packing on the device, maybe we can redirct the gzip stream directly to the installer?

I'd rather not do that. Those kinds of tasks over adb aren't very reliable and can create corrupted files. You wouldn't want your backup fail silently... If someone has very little free storage, they'll just have to exclude some more files. There's always some things you don't need to back up, like music for example.

NeoTheThird commented 4 years ago

Update: Someone started working on this on the forums.

Flohack74 commented 4 years ago

This is still missing the system-data equivalent, which might be not even readable to the phablet user. As a user I want my WiFi and other settings also backed up. the home directory does not contain this.

NeoTheThird commented 4 years ago

This is still missing the system-data equivalent, which might be not even readable to the phablet user. As a user I want my WiFi and other settings also backed up. the home directory does not contain this.

Are you sure, did you test it? Pretty sure that is in /userdata/user-data/phablet (not in /home/phablet, mind you!) as well. But we need to confirm that.

Flohack74 commented 4 years ago

Ok will take a look...

Alain94W commented 4 years ago

any results ?

Flohack74 commented 4 years ago
Alain94W commented 4 years ago

Hi Guys, so what do we do ? I nearly finished the backup and restore feature with remote file transfer (let's call the stream through adb to the computer like that).

If the device get disconnected during the backup, the backup will fail and display an error message to the screen. I have not yet tested the disconnection during restore but we can say that it will leave the device in an inconsistent state and the user will have to reinstall the OS, then try again the restore. I check the device free space before to start the remote restore and if the device doesn't have enough space, the restore will not be possible. I did it because for example, I have a Pro5 of 64Go and another one at 32Go of internal storage, It's safer to check if we have enough space before starting the restore possess.

If we go with the backup made on the internal flash then fetched via a adb pull, I have to change the code for both backup and restore functions and, also the graphical UI to allow the user to select what he would like to include in he's backup and restore (pictures, music, documents).

NeoTheThird commented 4 years ago

@Alain94W It's fine, you can go forward with it. We'll test it properly and put a note before it saying it's still experimental. If you push your latest changes, i'll review them and we can move forward :)

Alain94W commented 4 years ago

OK thanks ! Am I already banned from the telegram group ?

NeoTheThird commented 4 years ago

?

Alain94W commented 4 years ago

Question again, Is it possible to restore some system data and user data from a old OTA version to a new one ? I mean for example restoring a backup made on OTA-5 to OTA-7 ? is there a risk ? if yes I hace to check this before to restore

Flohack74 commented 4 years ago

I dont think there is a risk: It is mostly system settings that could be influenced by that, and so we must make sure that an OTA has migrations of settings where needed. But until now we did not have this function, so I cannot 100% be sure. I would slice this out still. Do it without check, do not try to solve all issues with the first iteration of the function.

NeoTheThird commented 4 years ago

I don't think it'll be a problem. As of today, you can migrate from canonical UT to Ubuntu Touch OTA-11 and keep your data without any issues. In any case, Florian is right, don't worry about it now.

Alain94W commented 4 years ago

Ok, I sarted again to work on it today, I'll push the sources as soon as possible.

Alain94W commented 4 years ago

I am stuck on one point, I don't know how to monitor with a progress bar when I restore the backup. I could not stat the folder as it is already existing with nearly the same used space as the backup space.

Any idea on how to monitor that ?