File Names with Leading Spaces are Improperly Stripped

[unamundan]

Expected behaviour

I should be able to begin filenames with a space or spaces. Up through Nextcloud Desktop Client 3.3.6, filename LEADING spaces were fully compliant and acceptable by both Mac OS X / macOS as well as Nextcloud Server and Client.

Actual behaviour

After updating to Nextcloud Desktop Client 3.4.1 (NDC3.4.1) from 3.3.6, hundreds (possibly thousands?) of files that began with spaces were tagged as illegal; NDC3.4.1 refused to sync them. Worse: NDC3.4.1 — without requesting operator permission or confirmation — RENAMED all files with LEADING spaces by REMOVING LEADING spaces from all filenames that had them.

Steps to reproduce

1. For decades, name files using leading spaces for personal sorting reasons which are, without question, accepted by Mac OS X as well as Nextcloud.
2. Update to Nextcloud Desktop Client 3.4.1 from 3.3.6.
3. Watch in horror as all hell breaks loose.

Client configuration

Client version: Nextcloud Desktop Client for Mac 3.4.1

Operating system: Mac OS X 10.13.6

OS language: English

Installation path of client: /Applications/Nextcloud.app

Nextcloud version:

Nextcloud 22.2.0

Storage backend (external storage): N/A

Logs

I’d rather not publicly post lengthy logs containing detailed information about directory contents. If there’s a project lead who needs them, I’ll provide them privately ... or any other additional troubleshooting info.

——————————

My ...

• MacPro5,1 tower with
• OS X 10.13.6 runs an
• Ubuntu 20.04.3 LTS VM with
• Nextcloud Server 22.2 while
• Nextcloud Desktop Client for Mac 3.3.6 provides sync management.

The setup was and had been working smoothly since November 2020; so much so that after updating to client 3.3.6, for months I hadn’t bothered to check for recent Desktop Client updates. When I finally did, version 3.4.1 was available. So I downloaded and installed it.

All hell broke loose.

Innumerable files became un-sync-able. Upon closer inspection, all “problem” files shared a common trait: their file names contained LEADING spaces. Scrambling for an answer, I frantically searched Github and found this thread:

Trim TRAILING spaces before uploading files #3778

If the TITLE of that thread is accurate, it’s not related to my issue. But, if that revised feature committed Oct 14, 2020, is the issue ... its title should be:

Trim LEADING spaces before uploading files

I reverted to Desktop Client 3.3.6 as quickly as possible, hoping to reverse the damage. It didn’t help. Not only were leading spaces stripped from file names on the server, but those now-leading-space-less files were synced DOWN. Thankfully, the original leading-space files LOCALLY were untouched. But, LOCALLY, there were TWO copies of each “problem” file: one with leading spaces, and a duplicate without leading spaces. The SERVER contained only a single version of each “problem” file that had been stripped of leading spaces.

Since I still had the properly-named LOCAL files (marked with red Xs which Nextcloud refused to sync), I thought leaving things alone would give me time to troubleshoot until I could reverse the fallout. I was wrong: now, the majority of my safety / fallback leading-space files have been removed from LOCAL storage as well as SERVER-SIDE. Oddly, there are still some leading-space files.

—————

I understand that all platforms have differing system-specific rules regarding legal and illegal characters in file names. Mac OS X / macOS allows leading characters. For YEARS I’ve been using that perfectly-acceptable file-naming convention to “float” files to the top of directory listings. And for YEARS, that also hasn’t been a problem for Nextcloud Client ... as it shouldn’t ... until upgrading to client 3.4.1.

If it was just a few files, I’d shrug, sigh, and rename them. But there are hundreds, perhaps thousands, of files with LEADING spaces. And frequently, these files are LINKED to page layouts. So, even if I did remove all leading spaces to comply with this new Nextcloud Client quirk, I’d also be creating a nightmare of epic proportions to relink them all to page layouts.

Any guidance is greatly appreciated.

[github-actions[bot]]

This bug report did not receive an update in the last 4 weeks. Please take a look again and update the issue with new details, otherwise the issue will be automatically closed in 2 weeks. Thank you!

[unamundan]

I’m leaving a comment to prevent auto thread closure. I’m truly surprised that I seem to be the only one — of what must be many more silent Nextcloud Mac users — who’s encountered this bug.

I love Nextcloud but this is the kind of data-defilement error that would make anyone afraid to use it. Users — even those who use the free version like me — rely on Nextcloud to provide their files safe harbor from intrusive data mining by the likes of Google and Dropbox ... but never expect the gatekeeper to defile its own castle.

I’m hoping some generous code hacker can solve this egregious regression. Please. And ... THANK YOU.

[Tegenwind]

We have the same issue - files or folders with leading or trailing spaces aren't syncing properly.

Also, when you have folders with trailing spaces, synchronisation often craps out completely and refuses to sync 'beyond' the problem folder, potentially causing conflicts if the end user doesn't notice for a while and continues to work in folders that are now not synced any longer.

I've noticed when creating files or folders in the web interface filenames are automatically corrected if there's a leading or trailing space. Existing files or folders are left untouched. When an existing folder with trailing space gets synced to the client it will throw an error after synchronisation is complete.

Fixing files and folders on one machine usually has them pop up duplicated on the other clients that have access to those folders as well, making fixing it a sort of really repetitive and sadistic version of "whack-a-mole", if you'd have to go through it manually.

I've since fixed all files and folders with trailing or leading spaces server-side, so now we basically 'just' ended up with a lot of local-only identical copies on the clients, for which we're now using a script per client to move these duplicate files and folders out of the local sync folder.

When we're done with this mess we hope to have some time to dive deeper into this issue and what's causing it, and if it's worthwile to roll back to an earlier version for now.

[Tegenwind]

Be careful with downgrading the client! We did some tests and noticed if we changed " folder" to "folder" on the server, that change wouldn't sync to a 3.4.2 client (so it would be stuck with just a red-crossed " folder"). After downgrading the client both " folder" and "folder" would be gone and "folder" would be found in the trash.

We used 3.2.4 as 3.3.* had been giving us nothing but problems as well (mainly issues with authentication that kept popping up at each start of the client).

[Tegenwind]

Update 3.4.3: The client now correctly renames folders that have trailing or leading spaces. However, after correcting the filename the file or folder is not uploaded at all, but the client shows a green orb next to it like it was. When the client is restarted the folder gets deleted if it's empty. If there are files in it, it errors out stating the file cannot be found. Only after moving the folder out of the sync folder and back in does it get uploaded. Corrected files do get uploaded after restarting the client.

Almost there, I guess? How stuff like this can get through testing is beyond me..

[vvvllll]

For me there's a regression in 3.4.3, trailing spaces in folders lead to error 400 on reading, on OSX at least. It totally stops the syncing task (it tries several times in a row the same request and then stops).

It wasn't as bad in 3.4.2. But reading this bug, i see previous versions had similar/related issues.

[Tegenwind]

Update 3.4.3: The client now correctly renames folders that have trailing or leading spaces. However, after correcting the filename the file or folder is not uploaded at all, but the client shows a green orb next to it like it was. When the client is restarted the folder gets deleted if it's empty. If there are files in it, it errors out stating the file cannot be found. Only after moving the folder out of the sync folder and back in does it get uploaded. Corrected files do get uploaded after restarting the client.

Almost there, I guess?

[Tegenwind]

I've made a separate bugreport regarding the deletion of automatically renamed files: https://github.com/nextcloud/desktop/issues/4387

[floriandusch]

I just updated to 3.4.4 on macOS and the client started renaming all the files with leading spaces. It'll be thousands on our instance. Is there a solution yet?

[github-actions[bot]]

This bug report did not receive an update in the last 4 weeks. Please take a look again and update the issue with new details, otherwise the issue will be automatically closed in 2 weeks. Thank you!

[github-actions[bot]]

This bug report is getting automatically closed due to no answer since the issue has been staled. Thank you!

[ugroempi]

This is still an unsolved issue for me, in version 3.5.4 on Windows 10. Files with leading spaces are not synchronized, and what's worse, I constantly get popup messages that complain about this issue.

[RokeJulianLockhart]

@ugroempi, for me, too, as and and demonstrate. I utilize Linux, so this should not be problematic for me.

"http://help.nextcloud.com/t/130865" is relevant.

[davidc]

After upgrading to the latest version (windows-10.0.19044) from a version that was probably a year old, I was informed of a file with leading spaces. Instead of renaming it, since I did not want it, I removed it from the filesystem. I also removed it from the Nextcloud web UI.

However every time it syncs, I am still told that the file contains a leading space. If I try to deal with this 'conflict' by renaming it in the desktop client, it fails: "Could not rename local file. Source file does not exist.". Even if I exclude the parent directory, I get this conflict on every sync, there does not seem to be a way to proceed, not even a restart!

[arthurgustin]

Hello, I used owncloud, then moved on to nextcloud to sync my ableton projects. All the frozen files can not be synced because they start with a space. It used to work well. I don't understand why there is this limitation ... Example: " (Gelé) [2022-02-12 190752].wav.asd"

[davidc]

Nobody understands why Nextcloud thinks they can decide to modify things while they sync them, such as altering file names or failing to preserve mtime on directories. That's par for the course though and not something we're going to change here.

What's absolutely unacceptable however is to introduce bugs in the process of implementing this and not fix them, resulting in sync errors on every single sync for the rest of time.

Probably they're too busy introducing features nobody asked for like a groupware shared dream journal or compulsory enforced upgrades to buggy versions you don't want.

[unamundan]

In about a week, it’ll have been a year since my initial report.

I’m holding back, sticking with NC 22, still using NC client 3.3.6 on several Macs running everything from OS X 10.11 to macOS 12.6.1.

Was a workable solution ever found? Investigated? Someone ... anyone: you can tell us. We can handle it. We just want to know whether we need to change the way we’ve done things for 20 years, or hold out hope that a fix is coming.

Should we read between the lines and reason that at this point, devs are probably just rolling their eyes at the MacIdiots, whispering: “Just shut up and let them figure out for themselves that we’re never going to implement a feature that works on only a 6% market share platform?” (It’s 18% if you count iOS.)

Is the only viable solution truly ...

RENAME EVERYTHING

... ?

MacFolk won’t be ecstatic. But at least we can move on.

Thanks NC devs. Truly. I love NC. I just want it to work for everyone. And I’d love to hear from one kind coder who knows what, if anything, is happening with this issue.

Apparently there are sync issues on Windows machines with similar symptoms and consequences.

[Tegenwind]

I agree there should be a setting in the admin console somewhere like “Enforce Windows compatibility” which you can just turn on or off if you please. Easy to make (I’d think) and everybody would be happy. I would argue though that even though you can start or end a filename with a space, it doesn’t mean that you should, and would generally be considered bad practice.

If you’re tired of waiting I’d suggest using a tool like NameMangler to scan your data dir and fix the filenames for you.

[JBM888]

Hi All, I have just updated my NexCloud installation to version 25 stable. That worked well and then I started working on getting one of my windows clients updated as well and got stuck immediately with an annoying sync error claiming that a few files were not following the naming convention i.e. had leading spaces in the filename. That was initially also true and several files were also such that they could be bluntly deleted being just unnoticed leftover temp files from a previous obsolete software installation. Those files then disappeared, for a while, but there were also some other files that needless to say are NOT subject to deletion. After checking these filenames they indeed contained leading spaces which was then corrected on the server for a couple of them but the error persisted for those files! We then manually changed them locally as well to no avail! The sync error persists but there are NO such files in any place of this storage environment! The files we did delete were also there again, according to the client! Trying to force a re-sync gives the same error and so did logging out and reattaching. I then decided to see what happened if I moved the two directories in question out of the directory tree that was being synced by the client. I then waited for the sync to catch up and the files were gone. I then moved the directories back into the synced part and things are now looking good again syncing as normal! I'm running Windows 10.0.19044 and the client I have installed that gave this problem is version 3.6.6. I wanted to post this since I noticed that most of the posts in the thread are mentioning Apple/Mac environments but from my experience, it is NOT just a thing concerning Macs. I will try to remove the files that I have difficulties with out of NextCloud and try to get the syncing working and then put them back. This is NOT a solution for those with numerous files in different locations but I have 2 locations

I won't try to explain why this happened nor would I say this is a solution for those with numerous files in many different locations but perhaps a workaround for some like me with a limited amount of files showing up having this problem.

//Regards

[organgtool]

I've been having issues syncing files in Windows if they begin with a space and I came upon this thread. Is it true that newer versions of the client automatically rename files without prompting the user for consent? Does it just rename them in Windows or on the server too? If it renames them everywhere, that is absolutely outrageous and completely unacceptable!!! Files and folders do not live in a vacuum - they are often referred to within project files, such as audio and video projects. Renaming those files may break those projects to the point that they may no longer even open! I understand that Windows is a pain in the ass (when is it not?) when it comes to its requirements for naming files, but files and folders should never be renamed without the user's explicit consent. I'm currently in the process of looking for a different cloud storage system as I can no longer trust NextCloud to preserve my data in a manner that won't break my files.

[Sergeij2000]

I got the same problem with an existing installation (Server on Ubuntu 22.04.2 LTS, Windows Client 3.7.4) with one file with a leading space in the filename, which had been synchronized without issues with the earlier client. Renaming the file through the dialog offered by NC doesn't fix the problem, renaming the file on the server and the client doesn't help either. Even excluding the complete directory containing the file from synchronization doesn't fix the error message. My educated guess it's a major problem in the sync data base, where the faulty sync entry persists even if the conflict is solved manually. I'm missing a way to drop one of these kind of error messages manually from the sync log. As i don't know if NC is syncing my files correctly after the error occurs (the message popping up in Windows message center says, that NC was not able to sync) i'm kind of worried that i may lose data.

[Tegenwind]

@organgtool, Anything newer than 3.4 does not do this anymore. 3.7 throws a warning and requires the user to review the file and change the filename, or they won't be uploaded.

@Sergeij2000, have you tried running a files:scan --all (or --path "path/to/folder") on your server? That should fix it if there's something amiss in your database, after which the error should either solve itself or actually be solvable :)

[Sergeij2000]

I did a files:scan --all on the server and it didn't fix the problem. In the end i was able to solve the issue by deleting the hidden '‘.sync_xxxxxxxxxxxx*.db' files in the client's Nextcloud root directory. The client started a new sync, which only included the files which were not synchronized properly because of error and after that every thing was fine. It seems to me that the client's database was broken. @Tegenwind: Thanks for your support...

[MaxXor]

This also affects paths/files with leading ## which I just found out myself after some troubleshooting.:)

[frkca]

Is there a path forward with this one? While I found this thread only by looking up few folders going into erred state but can understand there are people using spaces in filenames for absolutely legitimate reasons. Dangerous is that due to the aforementioned behavior people may even loose data which should lead to bumping severity of this bug - no matter how it's going to be fixed...

[EduCollin]

Hi! Just a layperson here. Moved recently to Nextcloud as my best friend hosts a server, with the idea to upload my ableton projects from an old iMac. 100GB worth of thousands of files. There is no way i can start to delve into file names. I really hope this is fixed.

[rogozaur]

FYI,

Almost on topic...

We use Nextcloud and Exchange 2016. External emails had their subject lines appended with [External...]. We observed that when users saved emails by dragging them from Outlook into Nextcloud folders, the .eml files were saved with a leading space.

After removing the brackets from the email subjects, the leading space issue disappeared for newly saved files.

However, the latest versions of Nextcloud have syncing issues with these files, requiring us to rename them manually or via a script. Additionally, complications arise when files or folders are shared among multiple users, not just the ones opening support tickets.

I am not aware of a fix yet, but changing the way email subjects are appended has prevented the issue from worsening in our case.

[Plop]

I have the same issue. Is there a way to fix it ? Thanks

[sebadamus]

Seem to still be happening, also with "|" or ">" and trailing spaces (anyway dont know how those damn users put that simbology o how, because windows at least wont let you enter them in a filename...

[despens]

Nextcloud shouldn't mess with file names. Files do not stand on their own, they're referenced from other places: web pages link to each other, images are referenced in layout documents, audio and video files might be referenced in a video editing project. Users might for whatever reason create files with leading spaces or characters in their names that some operating systems don't like. In that case, the operating system or software tool that cannot deal with these types of characters in file names can raise an error. Nextcloud is in general the wrong place to solve this problem. If there are teams working across several incompatible systems and tools, they need to coordinate, or use some nextcloud extension or app for particular folders perhaps. Imposing this problem on users that otherwise wouldn't be affected doesn't seem thought through.

[jancborchardt]

Imposing this problem on users that otherwise wouldn't be affected doesn't seem thought through.

@despens since currently we are not solving anything, but only showing errors (which easily get ignored), in fact the current situation is deferring the problem to users.

We have this happening plenty on our internal instance, and also privately I witnessed someone having a near data-loss cause files and folders with spaces were not synced up. No, the spaces were not intentional. Turns out a bunch of people just put a space at the end of filenames or headings (it is visible in some docs or emails).

Warnings are ignored as usual, and we can’t fault people for that. Nextcloud just has to work. And if we have to pick between a link breaking vs outright data loss, then we need to pick the lesser issue.