Weirdness when trying to restore wine and snap programs in an image, very strange behaviour when compressing (I found a way) a snapshot

[rp88-rp88]

Excuse me, "teejee tech" sent me here as you now maintain the project... I wanted to ask if you knew some further details about what goes on "inside" the Timeshift program you are running.

I've written a tutorial ( https://www.bleepingcomputer.com/forums/t/790526/compressing-a-timeshift-snapshot-to-fileeasily-portable-storage-of-snapshots/ ) about Timeshift usage before, but have recently discovered complications relating to WINE and SNAP programs when trying to perform the procedure I "discovered" (I'll explain below), they seem to occur at random sometimes, but particularly when one has tried compressing and then uncompressing (Again, I'll explain below) a timeshift image. I say recently discovered, but I have no particular reason to believe this is a recent software version specific matter, the images I was trying to restore from were old, it was just that recently they didn't work whereas in the past they have worked.

I really like the idea of using Timeshift to make "system image" type backups which can be used, as well as for restoring a system back to a good state after a disaster, as a way to clone one system (with all your programs installed, including WINE and SNAPS) to another.

Timeshift has a really strong attraction here above the likes of clonezilla because it is file/folder based not partition based. So it makes much smaller backups, and can work to restore to drives of different sizes than the original so long as all your new drive's partitions are each big enough to hold whatever files were on the appropriate equivalent partitions you cloned from.

The workflow of being able to take an "image" of a system with all your settings and programs on it, then load it up on any new PC (or the same PC after a catastrophic disaster) right after doing a fresh Linux Mint install on that new or disaster-recovered PC is really great in principle.

The other ideal situation would be to be able to make that "image" very portable by being able to hold it in one big "file" which unlike an ext4 timeshift folder could be transferred as a file over any medium... NTFS storage media... cloud backup uploads... and uncompressed back in to an ext4 timeshift folder when needed.

To this end I worked out this procedure:

https://www.bleepingcomputer.com/forums/t/790526/compressing-a-timeshift-snapshot-to-fileeasily-portable-storage-of-snapshots/

but I've found that when doing this I sometimes get a situation where the original timeshift "image" drive works to restore a system, but when trying the same thing from a copy of that drive (one made via the dd and 7z stuff I mention in the tutorial) the restoration mysteriously has WINE programs (Windows exe programs installed to run under the Wine compatibility layer, particularly with winetricks involved) and SNAP (programs running as snaps, including where you've held the version permanently so there's no possibility of updated versions of the snap suddenly turning incompatible) failing to work. Other than Wine and Snap programs not working after restoring, I couldn't see any other indication of the restored system not being a functionally adequate copy of the original system. Programs installed by the usual apt-get methods all worked fine, and kept their user suppleid settings. Programs installed by building from source code in the system's opt folder still worked, programs installed from deb files still worked, programs installed as appimages or simply run as linux executables (like firefox when downloaded directly from Mozilla, or Blender downloaded directly from the Blender site) from within sub directories of the home folder still worked. It was specifically Wine and Snap programs which failed to work in the post-restored system. One can purge away Wine and Snap in the new system, then apt-get install again and tinker around to get Wine and Snap apps working again, but it somewhat defeats the idea of having a Timeshift drive which you can just plug in to a freshly installed system and, without relying on an internet connection, use Timeshift to make that new system in to a clone of one you've already set up.

Something about the copied version of the timeshift drive is somehow different to the original, though I cannot work out what, checking sha256sum of the files on a timeshift drive and a copy of that drive gives the same results for all the stuff within the localhost subfolder of a snapshot folder.

I've also seen, much more rarely, a situation where when trying to restore to one of these snapshots timeshift will do so little it doesn't even decide to enter the "grey screen with scrolling text" phase, and instead marks itself as finished whilst still in the normal GUI.

I'd like to be able to correct my tutorial to fully account for what's going on here, but am mystified as to why Wine and Snap programs will often fail to restore correctly from the cloned timeshift drives whilst usually (but not always) restoring properly from the initial timeshift drives.

Can we discuss this further?

Is there a better means of resotration available in circumstances like this? Like directly using some sort of rsync command to restore from a timeshift snapshot in a fashion which would ignore all "exclude lists" and thereby be more likely to restore absolutely everything the way one would want to when rolling out a system image of one computer to other computers set up as identical copies of it.

Thank you

Please note, once we get talking about this, I can formally list out a procedure to give a specific example of when this occurs, like what to do on a fresh Linux Mint system to install a particular set of Wine and Snap programs, then attempt the timeshift restore on to another (or the same) system which has been wiped back to a freshly installed state and then see that the Wine and Snap things don't work after the restore. But I thought it best to give a more general overview description of the situation first, particularly pointing out the restoration procedure which I dsicovered but which turns out to sometimes introduce these weird troubles for Wine and Snap.

[tusch001]

Hi, I just read your tutorial. For me it seems that you are restoring from file using timeshift to a fresh linux install. I'm wondering if WINE is using something similiar to uuid to identify partitions. I don't know enough about how discs are identified but have you tried restoring the original uuid to the partitions (and changed fstab, crypttab and mdadm.conf) before starting timeshift? Maybe wine is looking for this.

Plus I remember in winecfg, where you assign drive letters to paths in the expanded settings, each mounted linux partition gets its own identification number. I don't know how it is created. Maybe you should make sure they are the same.

Edit: Have you ever encountered errors when restoring from zip file on the same system i.e. a system where the partitions were not formatted and kept their uuid?

[rp88-rp88]

I've looked inside winecfg, it definitely lists some "drives" but the lists don't look like uuids, the main c drive is the subfolder in the home folder, the other drives seem to be USB sticks or other removable media I've recently used (and not for anything wine related).

How does one set a uuid for a partition, the whole reason my procedure has that editing of fstab is because I didn't know there was any way by which, on a freshly installed Linux Mint system, one can change partition uuids to desired values. I'm not aware of any way to set them whilst doing the install from the live USb either, you can manually set up partitions from its GUI but I'm not aware of any setting to let you choose the uuids it will use.

"Have you ever encountered errors when restoring from zip file on the same system i.e. a system where the partitions were not formatted and kept their uuid?" That's a good thought, I'll go ahead and test it.

I'm aso pondering what might happen if, with the timeshift snapshot on the USB, I went in to it and modified the exclude.list file so it won't exclude any directories other than things that aren't like normal files at all (dev, proc, sys, media, mnt...). Maybe there's something wine related getting excluded there, but it would seem odd it gets excluded when resotring from a cloned copy (via the dd and zipping process) of a timeshift snapshot but not excluded when restoring from the original USB with the snapshot on it.

Another worth considering might be seeing whether tar zipping (because tar preserves hard links apparently) a snapshot might let it be cloned between USB sticks and still work afterwards. I developed my procedure, thanks for reading over it, because normal zipping (zip, 7z,...) didn't work to copy snapshots (due to those hard links inside the snapshots), maybe compressing with tar instead of dd might do something differently.

Thanks

[tusch001]

In general you can change the uuid of a ext4 partition by

sudo tune2fs <path to device> -U <UUID> -L <label for disk)

where 'path to device' is something like:

/dev/sda1
/dev/nvmen0p1
/dev/mapper/virtualdrive-lvm

First note all uuids of your old system. Your friends are (mainly the first, I've added that to .bashrc with an alias)

sudo lsblk -s -o name,label,path,mountpoints,uuid,serial,rota,fstype,size,fsuse%,fsused,fsroots
sudo fdisk -l
sudo lshw -C disk
sudo blkid -t TYPE=crypto_LUKS

If I have to revert a system, I install Linux from USB. But then it becomes a bit tricky: After the installation process - when asked - I "continue exploring". Now you need to be very diligent with the following changes. Make sure there are no typos. I can tell from experience :-) Otherwise your new system might not start properly. It's only scary for the first dozen times, later it becomes routine :-)

In general: if you use UUID for devices during startup in files /etc/fstab, /dev/crypttab (if encrypted) or /dev/etc/mdadm.conf (if part of a raid system) and you change any uuid, you have to update manually those files and tell the system (later more). Now is the time to be brave and change uuids of all disks. But from now on the new system will only boot with all the following changes!

sudo tune2fs <path to device> -U <UUID> -L <Volume-Label)

or - if not in use - format it with:

sudo mkfs.ext4 /dev/sdX1 -U <UUID> -L <Volume-Label>
sudo mkfs.ext4 /dev/nvmen0pX -U <UUID> -L <Volume-Label>
# open encrypted devices:
sudo mkfs.ext4 /dev/mapper/<Name-of-encrypted-device> -U <UUID> -L <Volume-Label>
# for logical volumes:
sudo mkfs.ext4 /dev/mapper/<VG-LVM> -U <UUID> -L <Volume-Label>

Now you have to chroot to your new system:

• The installer uses /target directory for this. So I just keep that: sudo mkdir -p /target and cd \
• Mount any USB / Sticks with any encryption keys you might need to access partitions
• open all encrypted devices and all logical volumes of any disk
• link current directories that are in use by the installer: sudo mount -o bind /proc /target/proc, sudo mount -o bind /dev /target/dev, sudo mount -o bind /mnt /target/mnt , and sudo mount -o bind /sys /target/sys
• (not sure if necessary, but to avoid error messages later on): sudo mkdir -p /target/dev/pts && sudo mount -o bind /dev/pts /target/dev/pts
• now you need to link boot partitions /boot and /boot/efi. That depends on your setup: sudo ln -s /dev/sda2 /target/boot && sudo ln -s /dev/sda1 /target/boot/efi or more complicated sudo ln -s /dev/md126 /target/dev/mapper/Boot (again if you use this you know what I'm talking about)
• you might want to mount all logical / encrypted volumes to their future mountpoints

Finally you can change to your newly installed fresh system:

sudo chroot /target

Now you are in your new environment as root.

In case you are using a raid system: In the latest version of linux mint finally they included mdadm (for raid systems) in the installer package. For older linux mint versions prior to 24 you might need to apt install mdadm.

So now you have to go through your new system and check everything:

Stop all your raid systems with mdadm --stop ... and assemble them under the new correct (old) name (mdadm --assemble ...). You might add those raid assemblies manually to your fstab / crypttab / mdadm.conf. If you have a raid system you know what I'm talking about.

If you use encryption: check uuids in /etc/crypttab and open them with their correct name for the mapper.

If you are using LVM: close and open all logical volumes again.

Again check fstab, crypttab and mdadm.conf for the correct uuids. Sorry for the repetition, I know what I'm talking about.

Afterwards (and before any reboot of the system) you need to update initramfs:

# -c = complete, no "update" -k all = all kernels, should only be 1
update-initramfs -c -k all

Make sure you resolve any issues that the process complains about otherwise your system might not boot properly (you can ignore your grafic card but all missing disks are important).

Repeat until all issues are solved.

Exit the new environment back to the installer-OS: exit

I hope I didn't forget anything, but most of it is copy & paste from my notes.

If you believe in god: pray

Reboot your system: If everything works well and you are not in "emergency mode" but have actually linux mint desktop, check uuids with lsblk and compare with your old system. Otherwise fix errors in terminal, update initramfs and reboot.

---

second part: my computer with wine and the virtual machine with windows is down because I had a shortcut in the riser cable to the sata card. And some very smelly magic smoke. Luckily it was only the cable. So at the moment I can only state things from memory:

Winecfg (where you assign drive letters) has an "extended or additional info" button. The windows identification number for assigned paths is only set for paths that are in fact on an extra disk partiton. So not every assigned drive letter gets a number. And I think wine creates info files for this volume in the "root folder" of this windows drive.

For me that is mainly my home directory, where in a subdirectory on an encrypted volume I keep my personal files. This subfolder I have linked to Windows "My files" with a new drive letter. Something like this is most likely the culprit if you partition your disk and wine has not the correct info file.

Everything else has to wait, I'm willing to test, but probably the soonest is next week.

Good luck.

Edit for typos :-)

[rp88-rp88]

Thanks for those tips, quite a lot to take in.

What I have established recently is the following:

1. with my compression and uncompression method the uuid used for the partition where the timeshift snapshot reside is identical on both the USB one originally writes it to, and on the cloned USB

2. after a first use of a timeshift USB for restoring extra files are created in the timeshift folder, an exclude-restore.list and an rsync-log-restore . I'm going to try deleting these two files from the cloned USB before restoring from it, as I can see that the order of things in the exclue-restore.list is different from those in the exclude.list, though all the same entires are in both. This difference doesn't explain why trouble happens only when resotring from the clone of the USb though.

3. There is an info.json file within the timeshift USB's folder structure which contains a uuid, this uuid matches that of the home partition of the computer on which the timeshift image was made. I may test the effect of changing the UUID in this file to match the UUID of the home partition one is resotring to on a frsh system, obviously I'd still have to do the whole fstab editing procedure still as epr my tutorial (because an fstab containing the original PC's partition uuids is part of the timeshift image), but maybe changing the uuid in the info.josn file may have some useful effect. The alternative is ofcourse renaming the UUIDs of the partition on the frshly instaleld system before doing the ret=store, now that you've explained it is indeed possible (thanks).

4. Regarding wine specifically, I compared the original timeshift USB to the cloned one, particukarly in terms of doing a recursive list of all the files on either drive containing the letters "wine" in their name. As one would expect for a cloning of a USb via dd, these are ofcourse identical, but by checking it I've ruled out anything really weird going on where some particular wine related file would magically refuse to copy.

5. On a test system where resotring from a cloned USB did not give working Wine programs, note that running winecfg in terminal gave the following error, although winecfg's GUI itself opened fine after several seconds of delay: wine: Read access denied for device L"\??\D:\", FS volume label and serial are not available. this error would imply something might well be up with wine volumes, but as far as I know this only means the new system couldn't see the UUID of an inserted USB stick (I can't think a drive letter as high as L could relate to anything actually belonging to the system as such) which the original system had, at least at one time, had plugged in to it. As I said, winecfg's list of Drives mentions some devices which have been attached only once and have no plausible reason to actually have ever had anything to do with Wine (like USB sticks for which one can be confident one has never saved, from within a Wine dependent windows program, files to them). Also, a similar error for winecfg appears on a working system were all installed Wine stuff is running just fine: MESA-INTEL: warning: Ivy Bridge Vulkan support is incomplete wine: Read access denied for device L"\??\E:\", FS volume label and serial are not available. wine: Read access denied for device L"\??\F:\", FS volume label and serial are not available. wine: Read access denied for device L"\??\G:\", FS volume label and serial are not available. wine: Read access denied for device L"\??\Z:\", FS volume label and serial are not available.

6. Having access to a working system with wine on it at this very instant I can report the following: There's a userdef.reg folder in the .wine directory, mostly it has double slahed folder paths, but there's a couple of entries which look like possible UUIDs, taking forms like: "{374DE290-123F-4565-9164-39C4925E467B}"="C:\users\username\Downloads" and "{374DE290-123F-4565-9164-39C4925E467B}"=str(2):"%USERPROFILE%\Downloads" system.reg also has some lines in it which mentiond a "HardwareId"=hex(7): and a "ClassGUID"= , maybe these are related to real hardware, or they might just by virtual names made up by wine to provide to any software running under wine which makes queries to the "windows" infrastructure" asking for this info in return.

................................

There's no hurry on testing this, it's a discovery I made a while back, it's just that it would be satisfying to find a procedure which actually works properly for everything on a Linux system. I've got some other tests I'm going to try too, and that can take several days to a few weeks depdning on how much time I have free.

P.S. the systems I've tested this with are neither encrypted nor raid using. Support for encrypted drives is well worth expanding the procedure to, but for simplicity I'd been doing this with unencrypted machines. Support for raid, well raid is much rarer than encryption, loads of Linux PCs are unencrypted because users (especially novice ones) know that if they turn on crypto then fixing things when they go wrong can become somewhere on a spectrum of much-hard to entirely-impossible-in-the-lifetime-of-the-universe, so use on unencrypted systems is absolutely expected. There's many good reasons to protect a system with encryption, so encrypted systems are also common too, particularly with more experienced linux users. Raid is, I might have misunderstood, only common for fairly high performance server systems which not many users have. Ofcourse, by their complexity those are just the same sort of systems which can take a lot of effort to set up, so there's a really good motivation to make it possible to clone them this way too. For the procedure to be fully general I'd expect it logically should work even with raid as well.

EDIT: my planned next steps are

1. Recently I did a restore to a test PC from a directly made timeshift USB stick, I did this just to confirm the image on that USB works properly. It did work properly, but I'd made a cloned image of the USB before using it, I will now wipe the system and try a restore from the cloned USB, just to encounter again the exact circumstances where the restore leaves one with broken Wine stuff afterwards.
2. With that done I'll copy down as many logs as I can about what sort of failures Wine is having, might take out the HDD of the test machine at that time and put it aside so I can query it later too.
3. I'll then (with different HDD in the test machine) try setting up a new fresh install and changing all its UUIDs to match those of the original system where the timeshift image was made. I'm not fully sure if the procedure needs to be as long as you describe it as? If I boot up the freshly installed system from a live USB instead I might be able to use tune2fs on the system's HDD while a live OS runs off a USB, then edit the fstab file manually to match the set UUIDs, at which point restarting might boot the internal HDD alright? (I'll try both ways to set UUIDs to desired values). Once UUIDs are set to desired values, I'll try resotring to the timeshift image on the cloned USB, and see if Wine still fails to run at this point.

[tusch001]

Without further tests on my computer, I can only give you an opinion. I still think it's not the uuid, more something that is stored in a "windows way" or something wine is looking for. Although I don't know much about windows systems.

Read access denied for device L"\??\D:\", FS volume label and serial are not available (mostly for all letters in the latin alphabet)

I remember seeing that too on my system for all volume labels I don't have assigned to in winecfg. I ignore them :-)

If you assign drive letters to a path, winecfg puts 2 files in that linux folder:

• .windows-label
• .windows-serial

Both files just contain one entry: the label is most likely the direct folder name or - if you have changed it - the label you have assigned to that drive in winecfg. The serial number is 0 if it is a Linux file system or e.g. the fat32 label (something like SD cards / USB are named in Hex as in ABCD-1234). Don't know for ntfs partitions.

When you create a new wine config by starting a program with wine program.exe (without a prefix) the first time wine creates the prefix-folder by default in /home/<username>/.wine. Using winetricks and prefixes the config files are by default in /home/<username>/.local/share/wineprefixes/<Prefixname>. In the subfolder dosdevices are the symbolic links to the linux folders. Only the links for drive letters in use are necessary. The missing drive letters are refered to in the error message.

So my question about your timeshift settings: Do you backup also the hidden files in your home directory because otherwise these wine configuration files are not in your timeshift file?

---

One other thing I remember reading somewhere (and made a note): You should keep drive Z: assigned to Linux root (/). Otherwise gecko doesn't work properly. Try to assign Z: in winecfg, maybe wine can't start gecko which is needed as a replacement for iexplore to display web content within a programme. It is not necessary for all programmes but maybe the reason wine is not working properly. No .windows-serial or .windows-label in the root folder necessary.

When my complete system is up again, I'll try to make a backup your way. Could be interesting. But will take some days. So long.

Edit concerning windows uuids:

UUIDs, taking forms like: "{374DE290-123F-4565-9164-39C4925E467B}"="C:\users\username\Downloads" and "{374DE290-123F-4565-9164-39C4925E467B}"=str(2):"%USERPROFILE%\Downloads" maybe these are related to real hardware

I checked on the computer I'm temporary using, where the download folder is on a separate linux partition (Not in wine but in a virtual machine). The windows uuid in userreg.def and the linux uuid of partitions are not the same. I didn't expect them to be, both are diffent operating systems and the uuid is created in software. Otherwise we wouldn't be able to change it.