grke / burp

burp - backup and restore program
http://burp.grke.net
Other
482 stars 77 forks source link

Could not stat C:/Users/user/OneDrive due to the folder being a reparsepoint #832

Open deajan opened 4 years ago

deajan commented 4 years ago

Hello,

Reporting a problem that affects like almost all backup programs that use Windows VSS. Newer OneDrive clients allow to show online files in local filesystem without actually downloading them. When using that functionnality, it breaks backups because the actual onedrive folder becomes a reparsepoint instead of a standard folder, leading to "Could not stat" error message.

There are at least 3 folders that are known to maybe becomre reparsepoints:

C:\Users\user\OneDrive
C:\Users\user\OneDrive - CompanyName
C:\Users\user\CompanyName

The easy solution is to exclude [A-Z]:/Users/[^/]/OneDrive* from backup, but then there's the 3rd folder that isn't predictable.

Since this FS behavior may be more widespread than Microsoft OneDrive, Is there maybe a way to detect reparsepoints in burp in order to automatically exclude them (with a logging message) ?

Under windows, one can identify a reparsepoint with:

fsutil reparsepoint query c:\users\user\MyFolderNotBeingARealOne

Best regards.

Link to MS forum with that problem https://social.technet.microsoft.com/Forums/en-US/d612c19a-0f18-431a-9d75-9fb5dfd11aca/windows-10-1803-client-backups-failing-media-write-protected-errors

grke commented 4 years ago

Hello, I thought burp already supported junction points and reparse points. Search for FT_REPARSE in burp.

If you are getting 'Could not stat', it means FT_REPARSE is not getting set. The setting code is in src/client/find.c:

static void windows_reparse_point_fiddling(struct FF_PKT *ff_pkt)
{
        /*
        * We have set st_rdev to 1 if it is a reparse point, otherwise 0,
        *  if st_rdev is 2, it is a mount point.
        */
        /*
         * A reparse point (WIN32_REPARSE_POINT)
         *  is something special like one of the following:
         *  IO_REPARSE_TAG_DFS              0x8000000A
         *  IO_REPARSE_TAG_DFSR             0x80000012
         *  IO_REPARSE_TAG_HSM              0xC0000004
         *  IO_REPARSE_TAG_HSM2             0x80000006
         *  IO_REPARSE_TAG_SIS              0x80000007
         *  IO_REPARSE_TAG_SYMLINK          0xA000000C
         *
         * A junction point is a:
         *  IO_REPARSE_TAG_MOUNT_POINT      0xA0000003
         * which can be either a link to a Volume (WIN32_MOUNT_POINT)
         * or a link to a directory (WIN32_JUNCTION_POINT)
         *
         * Ignore WIN32_REPARSE_POINT and WIN32_JUNCTION_POINT
         */
        if (ff_pkt->statp.st_rdev == WIN32_REPARSE_POINT) {
                ff_pkt->type = FT_REPARSE;
        } else if (ff_pkt->statp.st_rdev == WIN32_JUNCTION_POINT) {
                ff_pkt->type = FT_JUNCTION;
        }
}
#endif

And st_rdev is set in src/win32/compat/compat.cpp:

           Win32 reparse point (junction point) is like a link
           though it can have many properties (directory link,
           soft link, hard link, HSM, ...
           A mount point is a reparse point where another volume
           is mounted, so it is like a Unix mount point (change of
           filesystem).  */
        if(*pdwFileAttributes & FILE_ATTRIBUTE_REPARSE_POINT)
                sb->st_rdev=WIN32_MOUNT_POINT;
        else
                sb->st_rdev=0;

        if((*pdwFileAttributes & FILE_ATTRIBUTE_REPARSE_POINT)
          && (*pdwReserved0 & IO_REPARSE_TAG_MOUNT_POINT))
        {
                sb->st_rdev=WIN32_MOUNT_POINT;
                /* Now to find out if the directory is a mount point or
                   a reparse point, we must do a song and a dance.
                   Explicitly open the file to read the reparse point, then
                   call DeviceIoControl to find out if it points to a Volume
                   or to a directory. */

etc...

So, why would these directories not trigger this code?

deajan commented 4 years ago

To be honest, I'm really bad at reading C. I've tried to check on the file attributes flag and came up with this google magic: https://docs.microsoft.com/en-us/windows/win32/w8cookbook/placeholder-files

From what I've read, those new file types were introduced in Win8 and are identified by a new flag, eg IO_REPARSE_TAG_FILE_PLACEHOLDER.

Since that flag is unknown to Mingw's winnt.h, it's definition is:

#define IO_REPARSE_TAG_FILE_PLACEHOLDER         (0x80000015L)       // winnt

So I think the code should look like this in src/win32/compat/compat.cpp (sorry, bad C probably):

#define IO_REPARSE_TAG_FILE_PLACEHOLDER         (0x80000015L)       // winnt

[...]

 Win32 reparse point (junction point) is like a link
           though it can have many properties (directory link,
           soft link, hard link, HSM, ...
           A mount point is a reparse point where another volume
           is mounted, so it is like a Unix mount point (change of
           filesystem).  */
        if(*pdwFileAttributes & FILE_ATTRIBUTE_REPARSE_POINT)
                sb->st_rdev=WIN32_MOUNT_POINT;
        else
                sb->st_rdev=0;

        if((*pdwFileAttributes & FILE_ATTRIBUTE_REPARSE_POINT)
          && ((*pdwReserved0 & IO_REPARSE_TAG_MOUNT_POINT)
           || (*pdwReserved0 & IO_REPARSE_TAG_FILE_PLACEHOLDER)
           )
        {
                sb->st_rdev=WIN32_MOUNT_POINT;
                /* Now to find out if the directory is a mount point or
                   a reparse point, we must do a song and a dance.
                   Explicitly open the file to read the reparse point, then
                   call DeviceIoControl to find out if it points to a Volume
                   or to a directory. */

Hope that's it :)

grke commented 4 years ago

Cool, I can try something like that. Maybe you can test an installer once I have built one?

deajan commented 4 years ago

Sure :) I'm currently working on getting burp-cross-tools to use librsync 2.1.0 to fix a memory leak so I have a full build environment ready. I can even build my own binaries if you commit a fix on a branch.

grke commented 4 years ago

Here: https://burp.grke.org/downloads/test/burp-win64-installer-2.3.15.exe I had a go at updating librsync a couple of days ago, but failed. I have updated pcre and openssl though.

deajan commented 4 years ago

Having fun with librsync here: https://github.com/librsync/librsync/issues/154 I managed to get latest git master compiled as static lib, but now I need to wait for a new release. Once this is done, I'll happily update the librsync patches on the depkgs in order to compile a static version with libtool.

As for the burp installer you gave me, the "cannot stat" error is random for me (depends if the user updated files I guess), so I need to setup a VM in order to make a quick test. I'll see to make that tomorrow, really want to find out what the problem is.

deajan commented 4 years ago

I've tried to reproduce the error with a machine I experienced the error previously. Couldn't get it "to show". I'll try my test VM tomorrow (not myself a onedrive user)...

grke commented 4 years ago

Sounds good so far.

deajan commented 4 years ago

Well f***. Didn't have that error in a while (and stopped checking too after a week or so), but since a couple of days, using Burp 2.3.16, I noticed the same problem again,eg:

WARNING: Could not stat C:/Users/<user>/OneDrive - <org> FRANCE: Unknown error

Is there any (easy) C code I can compile to dump the *pdwFileAttributes of the directory ?

deajan commented 4 years ago

Well, got some easier way to get the attributes of that directory using fsutil:

C:\Users\<user>>fsutil reparsepoint query "c:\Users\<user>\OneDrive - <org> FRANCE"
Valeur de la balise d’analyse : 0x9000701a
Valeur de balise : Microsoft
Valeur de balise : répertoire

Analyser la longueur des données : 0x00000064
Données d’analyse :
0000:  01 00 64 00 46 65 52 70  bf 73 95 b5 60 00 00 00  ..d.FeRp.s..`...
0010:  02 00 09 00 07 00 01 00  58 00 00 00 0a 00 04 00  ........X.......
0020:  5c 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  \...............
0030:  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  ................
0040:  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  ................
0050:  00 00 00 00 00 00 00 00  00 00 00 00 01 00 00 00  ................
0060:  76 00 00 00                                       v...

"Valeur de la balise d'analyse" translates to "Reparse Tag Value", so I guessed that's the attribute for that reparse point. Searching the value 0x9000701a one on google gave me an interresting winnt.h file that comes with current mingw64 in https://github.com/mirror/mingw-w64/blob/master/mingw-w64-tools/widl/include/winnt.h (line 2676):

There are listed all types of reparse point, mine being identified as "cloud 7", whatever this means.

#define IO_REPARSE_TAG_MOUNT_POINT      __MSABI_LONG(0xA0000003)
#define IO_REPARSE_TAG_HSM              __MSABI_LONG(0xC0000004)
#define IO_REPARSE_TAG_DRIVE_EXTENDER   __MSABI_LONG(0x80000005)
#define IO_REPARSE_TAG_HSM2             __MSABI_LONG(0x80000006)
#define IO_REPARSE_TAG_SIS              __MSABI_LONG(0x80000007)
#define IO_REPARSE_TAG_WIM              __MSABI_LONG(0x80000008)
#define IO_REPARSE_TAG_CSV              __MSABI_LONG(0x80000009)
#define IO_REPARSE_TAG_DFS              __MSABI_LONG(0x8000000A)
#define IO_REPARSE_TAG_FILTER_MANAGER   __MSABI_LONG(0x8000000B)
#define IO_REPARSE_TAG_SYMLINK          __MSABI_LONG(0xA000000C)
#define IO_REPARSE_TAG_IIS_CACHE        __MSABI_LONG(0xA0000010)
#define IO_REPARSE_TAG_DFSR             __MSABI_LONG(0x80000012)
#define IO_REPARSE_TAG_DEDUP            __MSABI_LONG(0x80000013)
#define IO_REPARSE_TAG_NFS              __MSABI_LONG(0x80000014)
#define IO_REPARSE_TAG_FILE_PLACEHOLDER __MSABI_LONG(0x80000015)
#define IO_REPARSE_TAG_WOF              __MSABI_LONG(0x80000017)
#define IO_REPARSE_TAG_WCI              __MSABI_LONG(0x80000018)
#define IO_REPARSE_TAG_WCI_1            __MSABI_LONG(0x90001018)
#define IO_REPARSE_TAG_GLOBAL_REPARSE   __MSABI_LONG(0xA0000019)
#define IO_REPARSE_TAG_CLOUD            __MSABI_LONG(0x9000001A)
#define IO_REPARSE_TAG_CLOUD_1          __MSABI_LONG(0x9000101A)
#define IO_REPARSE_TAG_CLOUD_2          __MSABI_LONG(0x9000201A)
#define IO_REPARSE_TAG_CLOUD_3          __MSABI_LONG(0x9000301A)
#define IO_REPARSE_TAG_CLOUD_4          __MSABI_LONG(0x9000401A)
#define IO_REPARSE_TAG_CLOUD_5          __MSABI_LONG(0x9000501A)
#define IO_REPARSE_TAG_CLOUD_6          __MSABI_LONG(0x9000601A)
#define IO_REPARSE_TAG_CLOUD_7          __MSABI_LONG(0x9000701A)
#define IO_REPARSE_TAG_CLOUD_8          __MSABI_LONG(0x9000801A)
#define IO_REPARSE_TAG_CLOUD_9          __MSABI_LONG(0x9000901A)
#define IO_REPARSE_TAG_CLOUD_A          __MSABI_LONG(0x9000A01A)
#define IO_REPARSE_TAG_CLOUD_B          __MSABI_LONG(0x9000B01A)
#define IO_REPARSE_TAG_CLOUD_C          __MSABI_LONG(0x9000C01A)
#define IO_REPARSE_TAG_CLOUD_D          __MSABI_LONG(0x9000D01A)
#define IO_REPARSE_TAG_CLOUD_E          __MSABI_LONG(0x9000E01A)
#define IO_REPARSE_TAG_CLOUD_F          __MSABI_LONG(0x9000F01A)
#define IO_REPARSE_TAG_CLOUD_MASK       __MSABI_LONG(0x0000F000)
#define IO_REPARSE_TAG_APPEXECLINK      __MSABI_LONG(0x8000001B)
#define IO_REPARSE_TAG_GVFS             __MSABI_LONG(0x9000001C)
#define IO_REPARSE_TAG_STORAGE_SYNC     __MSABI_LONG(0x8000001E)
#define IO_REPARSE_TAG_WCI_TOMBSTONE    __MSABI_LONG(0xA000001F)
#define IO_REPARSE_TAG_UNHANDLED        __MSABI_LONG(0x80000020)
#define IO_REPARSE_TAG_ONEDRIVE         __MSABI_LONG(0x80000021)
#define IO_REPARSE_TAG_GVFS_TOMBSTONE   __MSABI_LONG(0xA0000022)

My best guess would be that all the above are reparse points that should be excluded from backups.

grke commented 4 years ago

OK, thanks for the testing, I will have another go at doing this shortly. :)

grke commented 4 years ago

I have made a change in master, for the next release, to try to make this better. Maybe you can test the release, when it is out?

deajan commented 4 years ago

Sure ! I'll make a before upgrade backup followed by an after upgrade backup and will then compare file manifests in order to make sure everything stays as it should (except for the reparse points of course).

Having read your code, I don't understand at all what it does, although the reparse_or_mount_song_and_dance function is niecely named ;)

grke commented 4 years ago

My changes for this have been released for a month, and nobody has complained.

I will assume that all is OK and close, please re-open if you find a problem.

deajan commented 4 years ago

Sorry, being late for the party, partly because of holidays.

Version 2.3.18 still has the same problem for OneDrive:

2019-12-31 13:54:48 +0100: burp[20153] Backup started
2019-12-31 13:54:48 +0100: burp[20153] Client version: 2.3.18
2019-12-31 13:54:48 +0100: burp[20153] Protocol: 2
2019-12-31 13:54:48 +0100: burp[20153] Client is Windows
2019-12-31 13:54:48 +0100: burp[20153] WARNING: Client 'myclient.local' version '2.3.18' does not match server version '2.3.14'. An upgrade is recommended.
2019-12-31 13:54:48 +0100: burp[20153] Begin phase1 (file system scan)
2019-12-31 14:01:26 +0100: burp[20153] WARNING: Could not stat C:/Users/someuser/OneDrive - Company Name: Unknown error
2019-12-31 14:01:43 +0100: burp[20153] End phase1 (file system scan)
2019-12-31 14:01:43 +0100: burp[20153] forked champ chooser pid 22195
2019-12-31 14:01:46 +0100: burp[20153] Connected to champ chooser.
2019-12-31 14:01:46 +0100: burp[20153] Phase 2 begin (recv backup data)
2019-12-31 14:35:32 +0100: burp[20153] End backup
2019-12-31 14:35:32 +0100: burp[20153] Backup ending - disconnect from client.
2019-12-31 14:35:32 +0100: burp[20153] Begin phase3 (merge manifests)
2019-12-31 14:37:49 +0100: burp[20153] End phase3 (merge manifests)
2019-12-31 14:37:49 +0100: burp[20153] Begin phase4 (sparse generation)
2019-12-31 14:38:34 +0100: burp[20153] locked: sparse index
2019-12-31 14:38:39 +0100: burp[20153] End phase4 (sparse generation)

Can I provide anything, like executing a small c code to get attributes of that folder somehow ? Best regards.

deajan commented 4 years ago

From what I read, FILE_ATTRIBUTE_REPARSE_POINT is 0x400, so none of the list I suggested above has that bit set and applies to be a reparse point.

Could you by any chance include that list / use a newer winnt.h and check files / directories against it ?

grke commented 4 years ago

Hello, Sorry for taking forever. I have a windows installer that adds this to the top of the reparse_or_mount_song_and_dance function:

printf("reparse_or_mount_song_and_dance: '%s' '%lu' '%lx'\n",
        file,
        reparse_tag,
        reparse_tag);

Maybe this could produce something useful?

https://burp.grke.org/downloads/test/burp-win64-installer-2.3.23.exe

deajan commented 4 years ago

No probs ;) I've updated, but didn't manage to reproduce the problem yet (it's actually kind of random). Guess you didn't change anything in the function yet ? I'll retest this week. Thanks.

grke commented 4 years ago

No, I literally only added a printf. I don't even know if it will give anything useful.

mikaku commented 2 years ago

Hello,

I've just found this issue and I think it is related to what I'm experiencing. I'm having problems with fresh installations of Windows 10 and Windows 11 that include the C:/Users/<user>/OneDrive directory:

0000037 2022-02-10 16:09:02 +0100

Last 1000 lines of /mnt/burp/win10-sergi/current/log.gz:

2022-02-10 16:09:02 +0100: burp[850881] Backup started
2022-02-10 16:09:02 +0100: burp[850881] Client version: 2.5.4
2022-02-10 16:09:02 +0100: burp[850881] Protocol: 1
2022-02-10 16:09:02 +0100: burp[850881] Using librsync hash blake2
2022-02-10 16:09:02 +0100: burp[850881] Client is Windows
2022-02-10 16:09:02 +0100: burp[850881] WARNING: Client 'win10-sergi' version '2.5.4' does not match server version '2.4.0'. An upgrade is recommended.
2022-02-10 16:09:02 +0100: burp[850881] Begin phase1 (file system scan)
2022-02-10 16:10:03 +0100: burp[850881] WARNING: Will not descend: file system change not allowed C:/Users/sergi/OneDrive: Unknown error
2022-02-10 16:10:03 +0100: burp[850881] End phase1 (file system scan)
2022-02-10 16:10:03 +0100: burp[850881] Begin phase2 (receive file data)
2022-02-10 16:11:30 +0100: burp[850881] End phase2 (receive file data)
2022-02-10 16:11:30 +0100: burp[850881] Backup ending - disconnect from client.
2022-02-10 16:11:30 +0100: burp[850881] Begin phase3 (merge manifests)
2022-02-10 16:11:31 +0100: burp[850881] End phase3 (merge manifests)
2022-02-10 16:11:31 +0100: burp[850881] Begin phase4 (shuffle files)
2022-02-10 16:11:31 +0100: burp[850881] Previous backup is not a hardlinked_archive
2022-02-10 16:11:31 +0100: burp[850881]  will generate reverse deltas
2022-02-10 16:11:31 +0100: burp[850881] Duplicating current backup.
2022-02-10 16:11:33 +0100: burp[850881] New backup is not a hardlinked_archive
2022-02-10 16:11:33 +0100: burp[850881] Doing the atomic data jiggle...
2022-02-10 16:11:44 +0100: burp[850881] End phase4 (shuffle files)
--------------------------------------------------------------------------------
Start time: 2022-02-10 16:09:01
[...]

Same happens if the user has also a Google Drive, in that case I get the following message:

2022-02-09 11:53:57 +0100: burp[767177] Begin phase1 (file system scan)
2022-02-09 11:54:23 +0100: burp[767177] WARNING: Generate VSS snapshot of drive "g:\" failed.
2022-02-09 11:54:23 +0100: burp[767177] got error: Problem with VSS

2022-02-09 11:54:23 +0100: burp[767177] error in phase 1
2022-02-09 11:54:23 +0100: burp[767177] Backup failed

Such errors prevent a complete backup. Do you have an idea of how we can solve this issue? Thanks.

grke commented 2 years ago

Hello,

I don't know much about OneDrive and Google Drive, but I think they are cloud storage services so that your files are backed up "to the cloud". Do you want to back them up to burp, or do you want burp to skip them? I'm not sure that it makes sense to back them up again, since these things are already backed up "to the cloud".

If you want burp to skip them, you can just add an exclude line for them.

The "Problem with VSS" error is because burp thinks it is just another local drive that it can do a VSS snapshot on. There is a 'vss_drives' option that you can use to tell burp which drives are VSS snapshottable that you can try.

For the OneDrive reparse point issue, burp ought to detect it and backup that filesystem entry as a reparse point (or whatever it is supposed to be), so that the backup includes that one filesystem entry. I will see if I can borrow somebody's Windows laptop to debug this myself.

mikaku commented 2 years ago

I don't know much about OneDrive and Google Drive, but I think they are cloud storage services so that your files are backed up "to the cloud". Do you want to back them up to burp, or do you want burp to skip them?

Same here, I don't know much about these cloud drives, but it seems that some people uses Google Drive as a main storage for small group of people that want to share documents easily. That's why I'd like to include the contents of Google Drive in the backup. Do you think this is possible?

In the case of OneDrive I saw that it comes by default in fresh installs of Windows 10 and Windows 11, and some people might store something there inadvertently. So I'd also want to include it in the backup.

The "Problem with VSS" error is because burp thinks it is just another local drive that it can do a VSS snapshot on. There is a 'vss_drives' option that you can use to tell burp which drives are VSS snapshottable that you can try.

Hmm, I don't know how exactly vss_drives=[list of drive letters] option works:

Anyway, for me the must important issue is OneDrive as it comes by default in all new Windows.

For the OneDrive reparse point issue, burp ought to detect it and backup that filesystem entry as a reparse point (or whatever it is supposed to be), so that the backup includes that one filesystem entry. I will see if I can borrow somebody's Windows laptop to debug this myself.

If you cannot get a Windows of someone, try downloading a Windows 10/11 ISO and install a 30days demo on a VM (set it up with "only" 3GB of memory and 20GB of disk). This should suffice for your testings.

Looking forward to your results. :smiley:

grke commented 2 years ago

Hello,

As far as I understand, OneDrive and Google Drive allow Windows users to store and retrieve remote files, like network file systems.

In general, backup software (like burp), operates on local drives, because there are multiple issues with trying to back up files over network file systems.

For example, what happens when the remote service is inaccessible? Does burp just skip that drive and backup all the local files? That means that when the server comes back in time for the next backup, all the remote files will look like new files and will have to be backed up again. Maybe instead, you can just fail the whole backup, but that means the local files won't be backed up at all.

So this implies that you ought to split the backup into two, so that you backup the remote files separately to the local files: You are better off with running the backup software on the actual host to which the files are local.

If you don't have access to the actual host, because it is some proprietory cloud system, then you probably need to sync the files to a local directory first, then backup that copy to your backup server.

Each cloud service will have it's own different set of apis too. I'm not going to attempt to try to keep up with them all and add native support for each OS and cloud service to burp. Maybe something like "rclone" is interesting to you? It claims to support "over 40 cloud products": https://rclone.org/

In conclusion: The best thing burp can do in the case of 'OneDrive' is to backup the special file system node within the C: partition that represents the 'OneDrive' link, rather than trying to download all of the files out of that link (this is what issue #832 was originally about, and what we have been trying to address).

In the case of Google Drive, which appears as a separate drive letter, as far as I can tell, it is a similar issue to the backing up of a "Mapped Network Drive" (which also doesn't support VSS).

  • The list of drive letters must include a separator? if so, a space? a comma? ...

I think it is a list of drive letters, for example: "CDE" Did you try it? What happened?

  • It is possible to do a backup of a drive without doing a VSS snapshot?, if no, what is the point of this option?

The point is to make it possible to attempt to backup a drive for which it is impossible to make a VSS snapshot. For example, some partition using a file system format that doesn't support VSS. Or some removable storage, like a CD or a USB stick. Did you try it? What happened?

  • If I set it with a 0 this means that Burp won't backup anything even on C:?

It means that it won't try to make VSS snapshots of any drives before it tries to back them up. Did you try it? What happened?

mikaku commented 2 years ago

For example, some partition using a file system format that doesn't support VSS. Or some removable storage, like a CD or a USB stick. Did you try it? What happened?

I've included the option vss_drives=[G] and the result was:

2022-02-16 09:49:02 +0100: burp[1258771] Backup started
2022-02-16 09:49:02 +0100: burp[1258771] Client version: 2.5.4
2022-02-16 09:49:02 +0100: burp[1258771] Protocol: 1
2022-02-16 09:49:02 +0100: burp[1258771] Using librsync hash blake2
2022-02-16 09:49:02 +0100: burp[1258771] Client is Windows
2022-02-16 09:49:02 +0100: burp[1258771] WARNING: Client 'win10-sergi' version '2.5.4' does not match server version '2.4.0'. An upgrade is recommended.
2022-02-16 09:49:02 +0100: burp[1258771] Begin phase1 (file system scan)
2022-02-16 09:49:07 +0100: burp[1258771] WARNING: Generate VSS snapshot of drive "g:\" failed.
2022-02-16 09:49:07 +0100: burp[1258771] got error: Problem with VSS

2022-02-16 09:49:07 +0100: burp[1258771] error in phase 1
2022-02-16 09:49:07 +0100: burp[1258771] Backup failed

It means that it won't try to make VSS snapshots of any drives before it tries to back them up. Did you try it? What happened?

Then, I've included the option vss_drives=[0] and the result was:

2022-02-16 10:09:01 +0100: burp[1259778] Backup started
2022-02-16 10:09:01 +0100: burp[1259778] Client version: 2.5.4
2022-02-16 10:09:01 +0100: burp[1259778] Protocol: 1
2022-02-16 10:09:01 +0100: burp[1259778] Using librsync hash blake2
2022-02-16 10:09:01 +0100: burp[1259778] Client is Windows
2022-02-16 10:09:01 +0100: burp[1259778] WARNING: Client 'win10-sergi' version '2.5.4' does not match server version '2.4.0'. An upgrade is recommended.
2022-02-16 10:09:01 +0100: burp[1259778] Begin phase1 (file system scan)
2022-02-16 10:09:16 +0100: burp[1259778] WARNING: Could not stat G:/: Unknown error
2022-02-16 10:09:16 +0100: burp[1259778] End phase1 (file system scan)

So, there seems no way to backup the Google Drive (G:).

grke commented 2 years ago

OK, so as I explained above, trying to backup files hosted remotely isn't going to work well. IMHO, the best thing you can do is either: a) rely on the cloud provider's backup capabilities to keep backups within the cloud, or b) use some tool that downloads them to the local computer and then back up those local files.

mikaku commented 2 years ago

b) use some tool that downloads them to the local computer and then back up those local files.

Yes, I could use the option backup_script_pre=[path] to use rsync or any other tool to download the contents of G: before the backup. But I need to make sure that there is enough disk space in the local drive.

Anyway, thanks for your assistance. I'll keep an eye here to see your results on the OneDrive issue.

mikaku commented 2 years ago

Yes, I could use the option backup_script_pre=[path] to use rsync or any other tool to download the contents of G: before the backup. But I need to make sure that there is enough disk space in the local drive.

This leads me to the question: it is possible to add an option on the fly when the client connects to the server without having to modify the burp.conf in the client?

I mean, I'd like to define on the server side which clients will execute a script before the backup, but I don't want to have to configure manually the client because I not always have a remote access to it (because it's behind firewall, is a road warrior, etc.).

grke commented 2 years ago

This leads me to the question: it is possible to add an option on the fly when the client connects to the server without having to modify the burp.conf in the client?

I mean, I'd like to define on the server side which clients will execute a script before the backup, but I don't want to have to configure manually the client because I not always have a remote access to it (because it's behind firewall, is a road warrior, etc.).

I'm not sure what configuration you want to change on the fly. If it were me, I'd add the script to all clients, and make the script generic.

mikaku commented 2 years ago

I'm not sure what configuration you want to change on the fly.

As I said in my previous posts, executing the rsync tool before the backup could be able to include the Google Drive (G:) into the backup since its contents now would be in C:

Following this reasoning, I'd want to be able to include the option backup_script_pre to the client when it connects to the server, but only to an specific client. This would save me from the situation where I'm no longer have access to the client remotely, so I'm unable to change its burp.conf and include there the new backup_script_pre option.

If it were me, I'd add the script to all clients, and make the script generic.

If I configure the same backup_script_pre option to all clients, they will download the contents of Google Drive to their C: and will be included in the backup multiple times, which seems pretty wasteful to me.

grke commented 2 years ago

Following this reasoning, I'd want to be able to include the option backup_script_pre to the client when it connects to the server, but only to an specific client. This would save me from the situation where I'm no longer have access to the client remotely, so I'm unable to change its burp.conf and include there the new backup_script_pre option.

OK, but if you don't have access to the client, how are you going to add a new script to the client?

mikaku commented 2 years ago

OK, but if you don't have access to the client, how are you going to add a new script to the client?

This is not really the case as all clients eventually will include the same script, but will be from the server side when I'd like to decide which client will run the script.

That's why I asked you if it's possible to inject a new option in the burp.conf of the client when it connects to the server every 20 minutes.