x8699876 / FastCopy

The fastest mass copy and sync, backup tool for Windows, Mac and Linux/Unix
Apache License 2.0
130 stars 17 forks source link

Sync operation terminates before destination files are deleted #12

Open MrGreg1 opened 6 months ago

MrGreg1 commented 6 months ago

I am using Veeam Backup and Replication software. My backup job runs a post job batch script. The script runs FastCopy command line Sync to mirror backups to another local storage location. When VBR retention runs, it deletes older backups. This means that FC will delete backups from the destination that no longer exist in the source. The full backups are quite large resulting in very large file deletions.

What happens is that FC and the script terminate before the all destination files are deleted. The files are eventually deleted but can take 7-10 minutes after the script has terminated. This gives a false sense of completion.

I am guessing you are using an API call to do the deletions. I suspect that your code does not wait until the file is actually deleted but just checks the request to delete was successful.

Is there a way that your code can wait for all files to be deleted before FC terminates?

x8699876 commented 6 months ago

This means that FC will delete backups from the destination that no longer exist in the source But FC does not delete destination files. what app or program does the deletion in your scenario? FC does not do mirroring, i.e. does not delete the target file it the file does not exist in the source. It only copies files from source to target.

MrGreg1 commented 6 months ago

Thanks for the speedy reply. The source is the VBR repository containing backups. It is a large Raid Array. The destination is where the production backups are copied to via FC Sync. It is also a large Raid Array.

Normally, FC will copy the new nightly backups from the source to the destination.

However, when VBR retention runs, VBR deletes backups from the repository (Source) that are older than 2 weeks. In this case FC will copy new backups from the source to the destination and delete backups from the destination that no longer exist in the source.

So FC is doing the deletions. Here is what the FC documentation says about Sync...

Sync (Size/date) - Copy if size or date is different, or it does not exist. In addition, delete all destination files/directories that do not exist in Source.

x8699876 commented 6 months ago

"> Sync (Size/date) - Copy if size or date is different, or it does not exist. In addition, delete all destination files/directories that do not exist in Source. "

please show me where you get this.

MrGreg1 commented 6 months ago

This is directly from FC documentation. Operation mode Sync (Size/date).

https://fastcopy.jp/help/fastcopy_eng.htm#mode

x8699876 commented 6 months ago

it is a different project. That is a windows application which I don't think is open source.

MrGreg1 commented 6 months ago

My apology. I now see that this is a different program with the same name.