Add VSS support to Windows version

[mreitinger]

For now this issue should be considered 'open to discussion'!

VSS Support should be added to support 'advanced' backup features on Windows (SQL Server, Exchange, ...).

This should

• be rather transparent to the user (--use-vss command line option or similar, see #983)
• have the ability to report VSS error messages
• have the option to continue even if VSS creating fails

Things to consider

• Should this be implemented by wrapping borg (scripting VSS is quite cumbersome)?
• Should some cross-platform support be implemented (VSS, LVM, QEMU Snapshots, others)?

Edit: I'm planning on opening a bounty for this once we agree on how to implement it.

[ThomasWaldmann]

I personally would rather object putting lots of stuff for that into borg. As you noted, every OS / filesystem / LVM / ... has its own way to snapshot. But having a wrapper sounds good.

[enkore]

As I've said in #983 it makes imho sense for Windows for multiple reasons. One is that VSS is better used from within an application and not via vssadmin which avoids a few issues (snapshot life cycle but also mounting problems and that it's just a PITA to do). Another is that on Windows VSS is simply the true one-and-only way to snapshot file systems with a well-defined API, while every Unix that has snap-shotting brings it's own tools and API -- and Linux has one or two for every FS. Finally VSS is pretty much the only way to avoid file locking problems on Windows and is required to be able to back up a live user profile let alone a live system.

These reasons makes them ("--vss" and ${support dozens of tools}) different features in my opinion. Otherwise I very much agree with @ThomasWaldmann that ${support dozens of tools} is not something that should go into Borg, since that'll be a never-ending story of untestable, obscure and very-low-bus-factor code, all three of which are undesirable for this project.

[mreitinger]

After discussing this with @ThomasWaldmann it would be preferable to implement snapshot features using external tools/scripts/... outside the main borg repository.

This allows us to use OS specific languages to implement whatever is needed (for example http://alphavss.alphaleonis.com/ or similar).

This should include examples on how to call the wrapper and subsequently borg itself.

[anarcat]

if this is implemented, could it be done in a modular enough way that other snapshot modules could be implemented as well?

i understand the hesitations in supporting $any_snapshot_tool_out_there, but like everything, this could be well tested and controled. i wrote code to get LVM snapshots in bup working, and it wasn't so bad.

it would be a bit bizarre to have snapshots accepted as a feature in Windows, and refused in Linux...

[enkore]

I wouldn't see it as bizarre, because the situations are so very different (1 standard vs. chaos and handle-based vs. path-based).

This kinda depends on #1038, if we do that then it should be little/no problem to have a very compact API that can be handed off to plugin-like things.

In either case I don't expect this issue to become practical (=near implementation) for probably some months.

[sinasalek]

The script wrapper solution probably has some limitations. Shadow volume should be mounted to become accessible via cmd. So the source path of the backup should be changed to another path or folder that should be translated back to the original source. #1038 might be related to what i mentioned but nonetheless more integration between the wrapper and the borg is probably required for hassle free usage

[ThomasWaldmann]

https://github.com/sblosser/pyshadowcopy

[zommuter]

Maybe interesting: https://github.com/kdar/ckbackup

[ThomasWaldmann]

https://wimlib.net/

They have some VSS code in C in their repo.

[deajan]

Since v1.2 roadmap has Windows as target, I'd like to "up" this feature request.

The minimalistic VSS api at https://github.com/sblosser/pyshadowcopy looks nice enough. The difficult part being that UNC paths provided by VSS (ie "\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy191\My Folder") have to be translated back in to plain FS paths (ie "C:\My Folder"). This could even be accomplished by the ShadowCopy class from pyshadowcopy itself, adding a "translate_unc_path" function that would map a standard path to a UNC path, in order for borg to store the original filename, when backing up a file from a VSS snapshot.

[ThomasWaldmann]

Help with the windows stuff is always welcome. Just keep projects in a reasonable order, i.e. first finish the simple, most basic support, then work on increasingly advanced features.

[deajan]

Can you give me a clue where to begin search in borg source ? (as long as it's python... not comfortable with C).

I'm thinking of a wrapper around the file backup function that would return the file contents of the VSS snapshotted file while holding the original filename.

[ThomasWaldmann]

I'm not involved in windows stuff and as I said, first the simple stuff needs finishing. Talk to @jrast.

[deajan]

Ok thanks.

[jrast]

@deajan VSS is currently not a priority. There is still other stuff which needs a lot of work. Last time I worked on brog was trying to get as much of the testsuite running as possible. If you want to help without coding: Test borg on windows and report the issues. If you like to code, you are more than welcome to support me on the windows side!

[deajan]

@jrast I am currently working on a onefile packager for Nuitka, which could probably help distributing borg Windows side, so that's my priority so far. As for VSS, I'd like someone to point me to the right borg source files to implement a VSS wrapper, so I can see whether this would be simple or pain in the ***. Can you guide me a bit ?

[jrast]

Sorry, I don't know VSS very well and I did'nt really get the idea you described some comments above:

The minimalistic VSS api at https://github.com/sblosser/pyshadowcopy looks nice enough. The difficult part being that UNC paths provided by VSS (ie "\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy191\My Folder") have to be translated back in to plain FS paths (ie "C:\My Folder"). This could even be accomplished by the ShadowCopy class from pyshadowcopy itself, adding a "translate_unc_path" function that would map a standard path to a UNC path, in order for borg to store the original filename, when backing up a file from a VSS snapshot.

Can you elaborate a little bit on this? Because I don't understand who is creating the shadow copy in this scenario and how borg is affected by this.