Some files with local contents are 0B on Dropbox

[inklesspen]

Describe the bug I use Dropbox to sync my Scrivener projects. Each project file is a Mac OS "Bundle" (a directory which appears in the GUI as a single file). Inside this directory are a number of directories and files in various formats, such as XML, RTF, and PDF.

Recently, I've noticed that some of the files within a Scrivener project are not being synced. It's unpredictable which files will be affected, but for some of the files, their size on my Mac will be > 0B, but their size on Dropbox will be 0B. It is easiest to reproduce this by removing the project file (which, again, is actually a directory containing hundreds of small files) from my sync folder, waiting for Maestral to sync, and then moving the project file back into the sync folder. Each time I do this, some files will not get synced up to Dropbox properly, but it will be different ones each time. Occasionally this has caused me to lose some data, when I open the project on another machine and Dropbox thinks the 0B file has been touched, and then Maestral dutifully syncs that 0B file back down.

If, after Maestral has finished syncing the project with its hundreds of small files, I open an individual RTF file which did not properly sync, add a character or two, and save it, that file will then sync properly and Dropbox will have the full contents. But this requires me to constantly monitor my Dropbox for silent sync issues.

I saw the comment in #910 about running with log level DEBUG, so I tried that, and I got a great deal of logging, but one thing I noticed repeatedly in the log:

client DEBUG: Retrying call <function DropboxClient._upload_helper at 0x105773420> on <class 'maestral.exceptions.DataCorruptionError'>: 1/10

I have attached the full log, along with a sample Scrivener project.

To Reproduce

sync test.scriv.zip is a compressed copy of the Scrivener "Tutorial" project. Unzip it before using it to reproduce the bug.

1. Drag the Scrivener project file ("sync test.scriv") into the sync folder.
2. Check filesizes on dropbox.com (or the sync folder on another machine) and note that some of the files are 0B when they should be larger.

Expected behaviour

1. Maestral should sync all files, or at least should report a sync error when unable to do so.

System:

• Maestral version: 1.7.3
• OS: MacOS 13.4

[samschott]

Thanks for the detailed report, including the logs, and apologies for the slow response time. I was able to reproduce this in an isolated test case and should have a fix now which will be available in the next release.

The issue was indeed in the retry logic on data corruption issues during upload.

[JNGRichardson]

Apologies for commenting on a closed issue, but I've been having this exact same issue for the last year or so, up to the present day (ver. 1.9.4, running macOS Sonoma 14.5). I've set logging to debug so I can catch the issue in more detail when it next occurs, but sadly don't have any more detail to offer beyond what OP described.

EDIT: Managed to catch it occurring! It happened some time after 20:00hr in the logs maestral.log

[samschott]

Hi @JNGRichardson, thanks for the report and the logs. Just to confirm, in your case its the file /Writing/Space Novel.scriv/QuickLook/Preview.html that ended up with 0 bytes on Dropbox?

[JNGRichardson]

Hi @samschott cheers for following this up!

I'm afraid you'll understand what you're looking at much better than me, but the issue as it presents itself in Scrivener is probably linked to the various content.rtf files (e.g. Files/Data/0A7EDD9F-9DE0-4CC9-9AC1-EE0E3769B6A8/content.rtf). Sometimes, after an apparently successful sync, these will appear to be blank in the Scrivener editor.

I've followed up with Literature and Latte (Scrivener devs) and tested syncing through the Dropbox macOS app, and the issue doesn't present itself, for what that's worth. A few choice quotes from L&L, if this helps:

Maestral should be seeing nothing more unusual than a modification date uptick, same as ⌘S in TextEdit would do. It just gets a lot more of them at once from Scrivener, since you might have a dozen files pending auto-save at once.

On mobile, we're in fact using the same exact API that Maestral uses. It would be accurate to say that Scrivener has its own home brew Dropbox client built in, and it probably does a lot of the same things Maestral does---checking modification dates, uploading and downloading as needed, etc. It's just confined in its scope to that one sync folder. But that in and of itself should be irrelevant to Maestral, as the result of that operation will be updates on the server, no different than any other tool (even official) would do. Perhaps the only interesting aspect there is that we've never had a zero-byte problem with the API, so that would most likely indicate an implementation problem in Maestral rather than a problem with the Dropbox third-party API as a whole.

Happy to help if there's anything else you'd like me to try!