Version 2.5: Client does not sync/The server file discovery reply is missing data

klada commented 5 years ago

I have installed version 2.5.0 of the Desktop Client (Windows, completely fresh install) and the client does not sync at all. I am getting the following sync error right after going through the configuration assistant:

A HTTP transmission error happened. The server file discovery reply is missing data.

The requests on the server look fine, I am getting many PROPFIND requests there, all answered with HTTP 207.

:warning: Version 2.3.3 of the client is working fine on the very same machine with the very same user. → regression

Expected behaviour

The sync process should start after configuration.

Actual behaviour

The sync fails immediately after configuration (fresh config, fresh installation, empty sync dir).

Steps to reproduce

Install and configure client (clean/fresh install)
Sync fails immediately

Client configuration

Client version: 2.5.0

Operating system: Windows 10 (x64)

OS language: English/German

Server configuration

Operating system: CentOS 7.5

Web server: Apache 2.4.6-80

Database: Postgres

PHP version:7.2.12

Nextcloud version: 14.0.3

Storage backend (external storage): -

Logs

Client logfile: Output of nextcloud --logwindow or nextcloud --logfile log.txt (On Windows using cmd.exe, you might need to first cd into the Nextcloud directory) (See also https://docs.nextcloud.com/desktop/2.3/troubleshooting.html#log-files)

https://gist.github.com/klada/fd88210850c4f4ec783c9e837a622acb

Web server error log:

(there are no errors)

192.168.12.16 - otto [13/Nov/2018:11:06:26 +0100] "PROPFIND /remote.php/dav/files/otto/ HTTP/1.1" 207 399 "-" "Mozilla/5.0 (Windows) mirall/2.5.0v2.5.0 (build 20181112) (Nextcloud)" 1
192.168.12.16 - otto [13/Nov/2018:11:06:38 +0100] "PROPFIND /remote.php/dav/files/otto/ HTTP/1.1" 207 400 "-" "Mozilla/5.0 (Windows) mirall/2.5.0v2.5.0 (build 20181112) (Nextcloud)" 1
192.168.12.16 - otto [13/Nov/2018:11:06:56 +0100] "PROPFIND /remote.php/dav/files/otto/ HTTP/1.1" 207 399 "-" "Mozilla/5.0 (Windows) mirall/2.5.0v2.5.0 (build 20181112) (Nextcloud)" 1
192.168.12.16 - otto [13/Nov/2018:11:07:10 +0100] "PROPFIND /remote.php/dav/files/otto/ HTTP/1.1" 207 400 "-" "Mozilla/5.0 (Windows) mirall/2.5.0v2.5.0 (build 20181112) (Nextcloud)" 1
192.168.12.16 - otto [13/Nov/2018:11:07:26 +0100] "PROPFIND /remote.php/dav/files/otto/ HTTP/1.1" 207 399 "-" "Mozilla/5.0 (Windows) mirall/2.5.0v2.5.0 (build 20181112) (Nextcloud)" 1
192.168.12.16 - otto [13/Nov/2018:11:07:42 +0100] "PROPFIND /remote.php/dav/files/otto/ HTTP/1.1" 207 400 "-" "Mozilla/5.0 (Windows) mirall/2.5.0v2.5.0 (build 20181112) (Nextcloud)" 1
192.168.12.16 - otto [13/Nov/2018:11:07:56 +0100] "PROPFIND /remote.php/dav/files/otto/ HTTP/1.1" 207 399 "-" "Mozilla/5.0 (Windows) mirall/2.5.0v2.5.0 (build 20181112) (Nextcloud)" 1
192.168.12.16 - otto [13/Nov/2018:11:08:14 +0100] "PROPFIND /remote.php/dav/files/otto/ HTTP/1.1" 207 400 "-" "Mozilla/5.0 (Windows) mirall/2.5.0v2.5.0 (build 20181112) (Nextcloud)" 1
192.168.12.16 - otto [13/Nov/2018:11:08:26 +0100] "PROPFIND /remote.php/dav/files/otto/ HTTP/1.1" 207 399 "-" "Mozilla/5.0 (Windows) mirall/2.5.0v2.5.0 (build 20181112) (Nextcloud)" 1
192.168.12.16 - otto [13/Nov/2018:11:08:46 +0100] "PROPFIND /remote.php/dav/files/otto/ HTTP/1.1" 207 400 "-" "Mozilla/5.0 (Windows) mirall/2.5.0v2.5.0 (build 20181112) (Nextcloud)" 1
192.168.12.16 - otto [13/Nov/2018:11:08:56 +0100] "PROPFIND /remote.php/dav/files/otto/ HTTP/1.1" 207 399 "-" "Mozilla/5.0 (Windows) mirall/2.5.0v2.5.0 (build 20181112) (Nextcloud)" 1
192.168.12.16 - otto [13/Nov/2018:11:08:58 +0100] "GET /ocs/v2.php/apps/notifications/api/v2/notifications?format=json HTTP/1.1" 304 - "-" "Mozilla/5.0 (Windows) mirall/2.5.0v2.5.0 (build 20181112) (Nextcloud)" 0
192.168.12.16 - otto [13/Nov/2018:11:09:18 +0100] "PROPFIND /remote.php/dav/files/otto/ HTTP/1.1" 207 400 "-" "Mozilla/5.0 (Windows) mirall/2.5.0v2.5.0 (build 20181112) (Nextcloud)" 1
192.168.12.16 - otto [13/Nov/2018:11:09:26 +0100] "PROPFIND /remote.php/dav/files/otto/ HTTP/1.1" 207 399 "-" "Mozilla/5.0 (Windows) mirall/2.5.0v2.5.0 (build 20181112) (Nextcloud)" 1
192.168.12.16 - otto [13/Nov/2018:11:09:50 +0100] "PROPFIND /remote.php/dav/files/otto/ HTTP/1.1" 207 400 "-" "Mozilla/5.0 (Windows) mirall/2.5.0v2.5.0 (build 20181112) (Nextcloud)" 1
192.168.12.16 - otto [13/Nov/2018:11:09:56 +0100] "PROPFIND /remote.php/dav/files/otto/ HTTP/1.1" 207 399 "-" "Mozilla/5.0 (Windows) mirall/2.5.0v2.5.0 (build 20181112) (Nextcloud)" 1

Server logfile: nextcloud log (data/nextcloud.log):

Not applicable.

klada commented 5 years ago

This seems to be the relevant part of the client log:

[OCC::LsColJob::finished    LSCOL of QUrl("https://mycloud.local/remote.php/dav/files/otto/") FINISHED WITH STATUS "OK"
[OCC::DiscoverySingleDirectoryJob::directoryListingIteratedSlot     Missing properties: "test" 2 0 1542103459 "DNVS" "" "-0000001ocrvl2k5jw23"
[csync_ftw  opendir failed for  - errno 10011
[OCC::SyncEngine::handleSyncError   ERROR during  csync_update :  "Es hat sich ein HTTP-Übertragungsfehler ereignet. Bei der Server-Dateierkennungsantwort fehlen Daten."

What is interesting is the Missing properties part. Apparently an ETAG is missing for the file or directory "test". The check which is failing has been introduced by 687b6f5655, which explains why the client version 2.3.3 is working fine.

The strange part is that the user does not have a directory or file with the name "test" in his user folder. I have already tried running occ files:scan for the user, but this did not change anything.

BTW: In Client version 2.3.3.1 I am getting this warning:

11-13 13:49:20:680 8260 OCC::DiscoverySingleDirectoryJob::directoryListingIteratedSlot: WARNING: etag of test is   This must not happen.

In version 2.3.3 this was a warning and in version 2.5.0 this is now an error which stops the entire sync.

merlinschumacher commented 5 years ago

Same issue for me on Arch Linux

rullzer commented 5 years ago

This seems more like a server issue. But even then it is weird.

There is of course a good reason that the client refuses to sync if it gets back things it can't make sense of from the server. Because it can't make sense of what is happening.

I assume it only happens for both of you on some accounts and not on all?

klada commented 5 years ago

Yes, it's probably a server issue. It still triggers a regression in the client:

Version 2.3.3 is working fine
Version 2.5.0 is not working at all → regression

And you are right, I can only reproduce the issue with one user account. Other user accounts are working fine with version 2.5.0. The strange part is that I have tried a occ files:scan, but it does not solve the issue.

@merlinschumacher Do you have a line like this in your client log?

[OCC::DiscoverySingleDirectoryJob::directoryListingIteratedSlot     Missing properties: "test" 2 0 1542103459 "DNVS" "" "-0000001ocrvl2k5jw23"

If yes, what does the "Missing properties" message say? Does it mention a filename you recognize?

nliautaud commented 5 years ago

Same here, caused by a bug in Google Drive external storage extension. Removing or updating the extension eliminated the client error.

OCC::DiscoverySingleDirectoryJob::directoryListingIteratedSlot  Missing properties: "GoogleDrive"

Note that the 2.3.3 client version was working even with this server extension issue.

merlinschumacher commented 5 years ago

The Gdrive-Plugin is indeed the issue. After installing a newer version from the app-store sync works fine.

tofuSCHNITZEL commented 5 years ago

I have a similar issue. But I don't have the Gdrive Plugin installed and all other Plugins are up to date.

[OCC::DiscoverySingleDirectoryJob::directoryListingIteratedSlot Missing properties: "Fotos Tauschrausch" 2 0 1542792323 "DNVS" "" "-0000001oc90bef3ea20" [csync_ftw opendir failed for - errno 10011

the above mentioned folder is not visible in the webclient - but it appears in the "select files/folders to sync dialog" and even if it is not checked the above error appears.

in the oc_mounts and oc_shares table an entry exists for this folder. I removed the entry and now the client syncs again.

messimuc commented 5 years ago

I had a similar issue. As I am just a user I cannot post logs. Some other user shared a folder with me, at some point this other user got deleted but her shared folder stayed in my files view. Client 2.3.3 had no issues, Client 2.5 stopped working: "A HTTP transmission error happened". Deleting the shared folder solved the issue for me.

r2evans commented 5 years ago

Same symptom, for me it was an external share that could not connect (the remote end is shut down for a bit). I don't know that it's always the problem (nor if SFTP is the only culprit), but when I disabled the external share the error went away and normal syncing resumed.

klada commented 5 years ago

In my case it's also a stale share causing the issue. Pretty much like @tofuSCHNITZEL described it:

In my case is a share named "test" (see log in post 2)
I see this share in the WebUI under "Shares", but not in "All Files".
I see this folder in the sync client properties
When I exclude this folder from syncing the sync client is working

seanbaker74 commented 5 years ago

I have the same issue. I had set up a CIFS/SMB external storage that was "broken" due to a configuration issue on my part. Until I fixed the external storage, my NextCloud client sync was failing with the "server file discovery reply is missing data" error message. Seems pretty easy to reproduce.

mlanner commented 5 years ago

It seems that the "external storages" server component that was changed in the 14.x series caused this bug, as I reported here:

https://github.com/nextcloud/server/issues/11264

r2evans commented 5 years ago

@mlanner, we might be talking about something different. My server is still on 13 (not stated earlier, but it now appears to be more of a server-bug than a client-bug).

nathanael-h commented 5 years ago

I had the same problem on ArchLinux, I tried to delete my old non working external storages (SFTP I did not use anymore) and it worked. Server is 14 on Debian

mioiox commented 5 years ago

Server NC13, client NC 2.51. I am using a bunch of SMB folders (the SMB server is Windows Server 2019, but is shouldn't matter). All folders are accessible except one (that is physically missing on the server at the moment). However, when I start syncing any of the fine folders, I get the same error. I managed to clear it by removing the bad folder from the NC configuration and the sync started without error.

However, I would expect to be able to sync the folders that are OK, as issues with one folder should not impact the sync capability of any other folder.

ghost commented 5 years ago

Same here I guess.

https://github.com/nextcloud/desktop/issues/908

bjoernv commented 5 years ago

Now I see this bug too:

nextcloud 14.04 (with PHP 7.2 on Ubuntu 14.04) nextcloud-client 2.5.1git (openSUSE Tumbleweed)

I do not have external storage except another Nextcloud 15.0.0 server which is connected with this Nextcloud server.

bjoernv commented 5 years ago

I was able to analyze my "The server file discovery reply is missing data." error message which stopped client synchronization because the error is thrown in the discovery phase before the synchronization phase.

I had two problematic files (one file, one directory). Unfortunately I have only partial logs. I used the --logfile nc.log --logdebug options.

I changed the file and directory names for privacy reasons. The server send this message for the file: Please note the trailing slash for the file "20150727_myexcelfile KVP20150820.pptx/". This is a normal Powerpoint file and should not have a trailing slash.

  <d:response>
    <d:href>/nextcloud/remote.php/dav/files/myusername/20150727_myexcelfile KVP20150820.pptx/</d:href>
    <d:propstat>
      <d:prop>
        <d:resourcetype>
          <d:collection/>
        </d:resourcetype>
        <oc:size>0</oc:size>
        <oc:permissions>SGDNV</oc:permissions>
        <oc:fileid>-1</oc:fileid>
      </d:prop>
      <d:status>HTTP/1.1 200 OK</d:status>
    </d:propstat>
  </d:response>

The NC client complained:

[OCC::DiscoverySingleDirectoryJob::directoryListingIteratedSlot     Missing properties: "20150727_myexcelfile KVP20150820.pptx" 2 0 1547298781 "DNVS" "" "-0000001oc2khb5drj3s"
[OCC::DiscoverySingleDirectoryJob::directoryListingIteratedSlot     Missing properties: "200 MyDirectory" 2 0 1547298781 "DNVS" "" "-0000001oc2khb5drj3s"
[csync_ftw  opendir failed for  - errno 10011
[OCC::SyncEngine::handleSyncError   ERROR during  csync_update :  "Es hat sich ein HTTP-Übertragungsfehler ereignet. Bei der Server-Dateierkennungsantwort fehlen Daten."
[OCC::ActivityWidget::addError  Item  "/var/home/myusername/nextcloud"  retrieved resulted in  "Es hat sich ein HTTP-Übertragungsfehler ereignet. Bei der Server-Dateierkennungsantwort fehlen Daten."

The directory "200 MyDirectory" exists in the filesystem, but only in files_versions and files_trashbin. The file 20150727_myexcelfile KVP20150820.pptx exists as a normal file.

I checked the file and directory in the NC database and removed the entries in oc_filecache. After that I started the command "./occ files:scan --all".

Now NC client 2.5.1 synchronizes again. I have to wait for the next problem to analyze the problem further. I also did not checked, if NC client 2.3.3 would also have stopped on the problematic files. I would guess, that NC client 2.3.3 was more fault tolerant here so that this problem comes up with NC 2.5.1. But also with NC client 2.3.3 I had problems with files, which were in oc_filescache but not in the filesystem.

harcesz commented 5 years ago

same problem, tested o 2-3 stations, good thing, we havent upgraded all our clients yet...

a simple workaround for ubuntu/linux is to use the 3.3 appimage; https://download.nextcloud.com/desktop/releases/Linux/Nextcloud-2.3.3-x86_64.AppImage

rrrnld commented 5 years ago

My problem was a folder shared with me by a person that no longer existed. I removed the folder and sync is working again.

aaronSkar commented 5 years ago

Had a similar issue with a file that showed up with as examplefile.ext/ rather than examplefile.ext it also was mounted in incorrect spots. I deleted all entries for the file in oc_mounts and oc_filecache and sync works again.

Note this issue came about after a migration from a snap deployment to a docker deployment in my case.

gerroon commented 5 years ago

I had the same issue, which was caused by a missing path to a share. However it would have been nice if it throws the error and syncs the initial folders

thoro commented 5 years ago

Agree, the error message could at least contain the path to the folder which is the issue, for non-IT users absolutely impossible to figure out ...

pachevalier commented 5 years ago

Same error on Ubuntu Mate. What is the trick ?

thoro commented 5 years ago

Start the client with --logwindow then check what folder is there, and then try to figure out where the folder comes from.

For me it was a unshared / shared folder, only visible in the "Shared" section of the website.

a-schild commented 5 years ago

I see the same problem here, with the GoogleDrive external storage no longer supported in NC 15.x I'm not able to remove the invalid (unknown) external storage type

averi commented 5 years ago

Same problem here with no external storage plugin installed, what we observed when using LDAP as backend is that whenever a specific user was removed from the LDAP group authorized to access the instance it broke the desktop client as it was trying to sync a directory that was bound to an inactive user, the fix here was either to remove the share manually from oc_share and oc_mounts OR extend the LDAP groups that could access the instance including the one that had former members.

tofuSCHNITZEL commented 5 years ago

Yes this is exactly my situation! I'm spit balling here but, maybe it has something to do with an issue I noticed with the function getGroupsByMember (and subsequently walkNestedGroups) in https://github.com/nextcloud/server/blob/master/apps/user_ldap/lib/Group_LDAP.php

As far as I could test and understand it the function does not work with LDAP implementations that don't use an entity's DN for the membership attribute (I believe like openLDAP where it is the uid)

unfortunately I didn't have the resources yet to devise a fix, also I'm lacking an LDAP where the DN is used as the membership to test it further

denics commented 5 years ago

If this can help, I confirm that the same thing happened to me installing sharepoint backend.

marcelklehr commented 5 years ago

@blizzz Can you take a look at this?

nonbinary commented 5 years ago

I'm having a similar problem, but with a federated share. The server with the problem has moved behind a firewall, but has a share from a public server. The user on the hidden server cannot remove the federated share, and cannot sync with the shared folder there. Disabling file sharing does solve the problem temporarily, but the server needs that functionality...

blizzz commented 5 years ago

It sounds like there are two possible causes:

unresolvable shares. And this can have a different issue if you are on 16 (fix → https://github.com/nextcloud/server/pull/15399). For the LDAP part, if there are server logs ideally with exceptions, that would be lovely.
external storages, with different backends. also here, please look out for server logs.

PetrVladimirov commented 5 years ago

It sounds like there are two possible causes:

1. unresolvable shares. And this can have a different issue if you are on 16 (fix → [nextcloud/server#15399](https://github.com/nextcloud/server/pull/15399)). For the LDAP part, if there are server logs ideally with exceptions, that would be lovely.

2. external storages, with different backends. also here, please look out for server logs.

1 for me, just SMB with no LDAP (nextcloud 15).

blizzz commented 5 years ago

@IvanKazak could you enforce this error to happen and provide the resulting nextcloud.log file?

PetrVladimirov commented 5 years ago

@IvanKazak could you enforce this error to happen and provide the resulting nextcloud.log file?

@blizzz here are the steps to reproduce and nextcloud log:

Connected a SMB share from scratch (Host=pure IP; user/pass auth) via web-console; made it avaivalble for user test
Logged in via desktop client with user test + ensured that the share was available (folder syncing was unchecked)
Stopped SMB share
Openned nc desktop client (was running OK in the backgroun with green light icon) and tried to expand the share - application crashed & was closed by windows
Started nc desktop client - got the error "A HTTP transimission error happened. The server file discovery reply is missing data." 6.nextcloud.log is attached nextcloud-1.log

blizzz commented 5 years ago

@IvanKazak There is an error related to SMB: stream does not support seeking at

Please update to latest NC 15 as there were some SMB related fixes since 15.0.5

vpecinka commented 5 years ago

Same for me - NC 15.0.8, LDAP, when somebody leaves company and left shares, those unresolvable shares causes desktop sync client stop as mentioned above. Removing the share solves problem (but is unfriendly).

InfamousUser commented 5 years ago

Same error here...

Edit: Disappeared the next day.

InfamousUser commented 5 years ago

I wonder if this has anything to do with unreachable external storage, obviously it shouldn't since you can't expect everything to be available at every given point.

sblaisot commented 5 years ago

I can confirm the behaviour with desktop client 2.5.2git (build 20190319) and nextcloud server 16.0.1 with LDAP authentication. Removing a user from LDAP server leads to broken sync if the removed user had shares. Removing the shares manually from the database fix the sync (i.e. delete from oc_share where uid_owner='<internal id of removed user>).

Pisoko commented 5 years ago

At our school we run into this issue more often. Is there a way to find out which deactivated account coursing this problem? Currently the only way to fix this is to downgrade from 2.5 to 2.3. :(

sblaisot commented 5 years ago

@pisoko your (server) log will tell you which user is not available anymore. Upgrading nextcloud server to 16.0.3 resolved the issue for me.

kyhuynhduc commented 5 years ago

same with NC 16.01 client : 2.5.3

FreeMinded commented 5 years ago

Upgrading nextcloud server to 16.0.3 resolved the issue for me.

Not for us! Problem still exists with NC 16.0.3 with LDAP users, Group Folders and Sync Client 2.5.2 on OSX