Not an product issue, but an interesting discovery involving a security scan... (Good)

Arcyne commented 7 years ago

So I was testing your software with the fix that you provided (see other thread) and after I started running the backup it failed with an error notification....

2017-07-15 11:39:30.978 INFO UPLOAD_PROGRESS Skipped chunk 1016 size 10111695, 71.98MB/s 00:01:27 56.2% 2017-07-15 11:39:35.151 ERROR UPLOAD_CHUNK Failed to upload the chunk d68a50fa04dfe17157f1c7831a44d89f001919bafb82b23d28f9b3a053086293: rename j:\gobackup2/chunks/d6/8a/50fa04dfe17157f1c7831a44d89f001919bafb82b23d28f9b3a053086293.qfbqbjol.tmp j:\gobackup2/chunks/d6/8a/50fa04dfe17157f1c7831a44d89f001919bafb82b23d28f9b3a053086293: Operation did not complete successfully because the file contains a virus or potentially unwanted software.

I took a look at Windows.Defender (it popped up) and it showed that it blocked the rename of the backup chunk due to it containing the TrojanDownloader:JS/Nemucod malware.

At this point I said OMG (--editorial, think of the voice of one of the characters in the Movie Trolls) and promply ran a virus scan on the drive that I was backing up. Nothing... (I also did it in the VM's themselves, etc... Nothing....)

Ok, I then figured I would partition the backup to back up each subdirectory seperately, to save time I'll skip the details but finally I found that the offending backup directory was under the $RECYCLE.BIN. I then ran the backup, identified the offending directory and identified the issue as being localized to a security VM that I used to test out some downloads for compression programs. (e.g. a throwaway sandbox VM that I used to download/test some software.)

To finalize: What did I learn?

Don't do a normal delete of a security sandbox VM (e.g. don't let it get to the recycle bin.)
Security software doesn't always find issues in underlying VM's. (normally I checkpoint, revert before leaving them stable but apparently in this case I just deleted the whole VM.)
That backing up the chunks individually allowed the host machine to identify malware that was previously hidden in the VHDX file.

Finally, I have one suggestion, it might be a good idea to include the original file(s) that contained the failing chunks in the -debug version of the log output. That would have made it easier to track down the offending directory/file. (If it is already there, feel free to point me to it.)

Sincerely, R

Arcyne commented 7 years ago

Note, humor, writing this finding up got my account flagged by github.

gilbertchen commented 7 years ago

Interesting. I wonder why Windows.Defender didn't detect the error when the chunk was being written to the .tmp file.

If you look for the last "Packing ..." log message before the failing chunk then that log message includes the name of the file that the chunk belongs to.

gilbertchen / duplicacy

Not an product issue, but an interesting discovery involving a security scan... (Good) #95