Moving from one shared folder to a different shared folder fails (database gets out of sync)

[fgrsnau]

Steps to reproduce

1. create 3 different users (a, b, c)
2. let user "a" share a directory "da" with "c"
3. let user "b" share a directory "db" with "c"
4. login as "c" and move a subdirectory from "da" to "db"

Expected behaviour

The directory should visible in db.

Actual behaviour

The move fails with an generic error. The underlying files are moved to directory "db". The database query fails and the database is not updated. All files are "visible" in "da" but are inaccessible. An occ files:scan --all fixes this.

Server configuration

Operating system: Debian Stable

Web server: ngnix

Database: Postgres 9.4

PHP version: PHP 5.6.30-0+deb8u1

Nextcloud version: 11.0.2 (stable)

Updated from an older Nextcloud/ownCloud or fresh install: Any (tested 3 different servers, one of them was freshly installed)

Where did you install Nextcloud from: Website

List of activated apps: Only stock NextCloud.

Are you using encryption: no

Are you using an external user-backend, if yes which one: LDAP, but reproducible without any external user-backend

NextCloud Logs

{"reqId":"NE81Fi3k0jojqFyF0HT3","remoteAddr":"xxx","app":"webdav","message":"Exception: {\"Message\":\"An exception occurred while executing 'SELECT \\\"path\\\", \\\"fileid\\\" FROM \\\"oc_filecache\\\" WHERE \\\"storage\\\" = ? AND \\\"path\\\" LIKE ?' with params [false, \\\"Photos\\\\\\\/%\\\"]:\\n\\nSQLSTATE[22P02]: Invalid text representation: 7 ERROR:  invalid input syntax for integer: \\\"\\\"\",\"Exception\":\"Doctrine\\\\DBAL\\\\Exception\\\\DriverException\",\"Code\":0,\"Trace\":\"#0 \\\/var\\\/www\\\/3rdparty\\\/doctrine\\\/dbal\\\/lib\\\/Doctrine\\\/DBAL\\\/DBALException.php(128): Doctrine\\\\DBAL\\\\Driver\\\\AbstractPostgreSQLDriver->convertException('An exception oc...', Object(Doctrine\\\\DBAL\\\\Driver\\\\PDOException))\\n#1 \\\/var\\\/www\\\/3rdparty\\\/doctrine\\\/dbal\\\/lib\\\/Doctrine\\\/DBAL\\\/Connection.php(836): Doctrine\\\\DBAL\\\\DBALException::driverExceptionDuringQuery(Object(Doctrine\\\\DBAL\\\\Driver\\\\PDOPgSql\\\\Driver), Object(Doctrine\\\\DBAL\\\\Driver\\\\PDOException), 'SELECT \\\"path\\\", ...', Array)\\n#2 \\\/var\\\/www\\\/lib\\\/private\\\/DB\\\/Connection.php(194): Doctrine\\\\DBAL\\\\Connection->executeQuery('SELECT \\\"path\\\", ...', Array, Array, NULL)\\n#3 \\\/var\\\/www\\\/lib\\\/private\\\/Files\\\/Cache\\\/Cache.php(526): OC\\\\DB\\\\Connection->executeQuery('SELECT `path`, ...', Array)\\n#4 \\\/var\\\/www\\\/lib\\\/private\\\/Files\\\/Cache\\\/Wrapper\\\/CacheJail.php(313): OC\\\\Files\\\\Cache\\\\Cache->moveFromCache(Object(OCA\\\\Files_Sharing\\\\Cache), 'Photos', 'files\\\/Test2\\\/Pho...')\\n#5 \\\/var\\\/www\\\/apps\\\/files_sharing\\\/lib\\\/Cache.php(106): OC\\\\Files\\\\Cache\\\\Wrapper\\\\CacheJail->moveFromCache(Object(OCA\\\\Files_Sharing\\\\Cache), 'Photos', 'Photos')\\n#6 \\\/var\\\/www\\\/lib\\\/private\\\/Files\\\/Cache\\\/Updater.php(198): OCA\\\\Files_Sharing\\\\Cache->moveFromCache(Object(OCA\\\\Files_Sharing\\\\Cache), 'Photos', 'Photos')\\n#7 \\\/var\\\/www\\\/lib\\\/private\\\/Files\\\/View.php(330): OC\\\\Files\\\\Cache\\\\Updater->renameFromStorage(Object(OCA\\\\Files_Trashbin\\\\Storage), 'Photos', 'Photos')\\n#8 \\\/var\\\/www\\\/lib\\\/private\\\/Files\\\/View.php(804): OC\\\\Files\\\\View->renameUpdate(Object(OCA\\\\Files_Trashbin\\\\Storage), Object(OCA\\\\Files_Trashbin\\\\Storage), 'Photos', 'Photos')\\n#9 \\\/var\\\/www\\\/apps\\\/dav\\\/lib\\\/Connector\\\/Sabre\\\/ObjectTree.php(268): OC\\\\Files\\\\View->rename('Test\\\/Photos', 'Test2\\\/Photos')\\n#10 \\\/var\\\/www\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/CorePlugin.php(642): OCA\\\\DAV\\\\Connector\\\\Sabre\\\\ObjectTree->move('Test\\\/Photos', 'Test2\\\/Photos')\\n#11 [internal function]: Sabre\\\\DAV\\\\CorePlugin->httpMove(Object(Sabre\\\\HTTP\\\\Request), Object(Sabre\\\\HTTP\\\\Response))\\n#12 \\\/var\\\/www\\\/3rdparty\\\/sabre\\\/event\\\/lib\\\/EventEmitterTrait.php(105): call_user_func_array(Array, Array)\\n#13 \\\/var\\\/www\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/Server.php(479): Sabre\\\\Event\\\\EventEmitter->emit('method:MOVE', Array)\\n#14 \\\/var\\\/www\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/Server.php(254): Sabre\\\\DAV\\\\Server->invokeMethod(Object(Sabre\\\\HTTP\\\\Request), Object(Sabre\\\\HTTP\\\\Response))\\n#15 \\\/var\\\/www\\\/apps\\\/dav\\\/appinfo\\\/v1\\\/webdav.php(60): Sabre\\\\DAV\\\\Server->exec()\\n#16 \\\/var\\\/www\\\/remote.php(165): require_once('\\\/var\\\/www\\\/apps\\\/d...')\\n#17 {main}\",\"File\":\"\\\/var\\\/www\\\/3rdparty\\\/doctrine\\\/dbal\\\/lib\\\/Doctrine\\\/DBAL\\\/Driver\\\/AbstractPostgreSQLDriver.php\",\"Line\":91,\"User\":\"stefan\"}","level":4,"time":"2017-03-30T21:09:29+00:00","method":"MOVE","url":"\/remote.php\/webdav\/Test\/Photos","user":"fgrsnau","version":"11.0.2.7"}

#0 /var/www/cloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/DBALException.php(128): Doctrine\DBAL\Driver\AbstractPostgreSQLDriver->convertException('An exception oc...', Object(Doctrine\DBAL\Driver\PDOException))
#1 /var/www/cloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Connection.php(836): Doctrine\DBAL\DBALException::driverExceptionDuringQuery(Object(Doctrine\DBAL\Driver\PDOPgSql\Driver), Object(Doctrine\DBAL\Driver\PDOException), 'SELECT "path", ...', Array)
#2 /var/www/cloud/lib/private/DB/Connection.php(194): Doctrine\DBAL\Connection->executeQuery('SELECT "path", ...', Array, Array, NULL)
#3 /var/www/cloud/lib/private/Files/Cache/Cache.php(526): OC\DB\Connection->executeQuery('SELECT `path`, ...', Array)
#4 /var/www/cloud/lib/private/Files/Cache/Wrapper/CacheJail.php(313): OC\Files\Cache\Cache->moveFromCache(Object(OCA\Files_Sharing\Cache), 'manuals/UPS_Giz...', 'files/Server/UP...')
#5 /var/www/cloud/apps/files_sharing/lib/Cache.php(106): OC\Files\Cache\Wrapper\CacheJail->moveFromCache(Object(OCA\Files_Sharing\Cache), 'manuals/UPS_Giz...', 'UPS_Gizeh')
#6 /var/www/cloud/lib/private/Files/Cache/Updater.php(198): OCA\Files_Sharing\Cache->moveFromCache(Object(OCA\Files_Sharing\Cache), 'manuals/UPS_Giz...', 'UPS_Gizeh')
#7 /var/www/cloud/lib/private/Files/View.php(330): OC\Files\Cache\Updater->renameFromStorage(Object(OCA\Files_Trashbin\Storage), 'manuals/UPS_Giz...', 'UPS_Gizeh')
#8 /var/www/cloud/lib/private/Files/View.php(804): OC\Files\View->renameUpdate(Object(OCA\Files_Trashbin\Storage), Object(OCA\Files_Trashbin\Storage), 'manuals/UPS_Giz...', 'UPS_Gizeh')
#9 /var/www/cloud/apps/dav/lib/Connector/Sabre/ObjectTree.php(268): OC\Files\View->rename('AG DSN/manuals/...', 'Server/UPS_Gize...')
#10 /var/www/cloud/3rdparty/sabre/dav/lib/DAV/CorePlugin.php(642): OCA\DAV\Connector\Sabre\ObjectTree->move('AG DSN/manuals/...', 'Server/UPS_Gize...')
#11 [internal function]: Sabre\DAV\CorePlugin->httpMove(Object(Sabre\HTTP\Request), Object(Sabre\HTTP\Response))
#12 /var/www/cloud/3rdparty/sabre/event/lib/EventEmitterTrait.php(105): call_user_func_array(Array, Array)
#13 /var/www/cloud/3rdparty/sabre/dav/lib/DAV/Server.php(479): Sabre\Event\EventEmitter->emit('method:MOVE', Array)
#14 /var/www/cloud/3rdparty/sabre/dav/lib/DAV/Server.php(254): Sabre\DAV\Server->invokeMethod(Object(Sabre\HTTP\Request), Object(Sabre\HTTP\Response))
#15 /var/www/cloud/apps/dav/appinfo/v1/webdav.php(60): Sabre\DAV\Server->exec()
#16 /var/www/cloud/remote.php(165): require_once('/var/www/cloud/...')
#17 {main}

I was digging a little bit deeper today. The failure is visible in https://github.com/nextcloud/server/blob/4821c00ea81797fcb5a99c31105ad42be598f113/lib/private/Files/Cache/Cache.php#L515 as sourceStorageId is false. This is, of course, not a valid integer in PostgresSQL and we get this nice error: ERROR: invalid input syntax for integer: \"\"

I tracked this value back to https://github.com/nextcloud/server/blob/4821c00ea81797fcb5a99c31105ad42be598f113/apps/files_sharing/lib/Cache.php#L81.

Somehow the $sourceCache is not populated correctly (the numericId is not set). Currently I do not now how to debug this further.

[blizzz]

@schiessle @rullzer

[rullzer]

Let me summon @icewind1991

[MorrisJobke]

I had the same issue :/

[schiessle]

In the past we blocked moving one mount point into another one because this create a lot of problems for re-shares, allow to create share-circles, etc. I think that's still true, right? So what we need to fix here is the check for moving one mount point into another one and block it.

Or did I miss something, @rullzer ?

[rullzer]

@schiessle it is not moving the mount point it is moving a subfolder of a mount point.

[schiessle]

ah, I see... maybe a permission issue? user 'c' needs to have delete permissions of "da" and create permissions on "db".... I read nothing about the share permissions in this thread that's why I'm asking.

[fgrsnau]

I did not move the mount point itself (“shared folder“ symbol in the root directory) but a subdirectory from one shared folder to another one (as @rullzer said). Interestingly, moving a single file from one share to a different share seems to work.

Permissions are not the problem. Full access was granted for all shares.

I was also able to reproduce this using only two users. User "b" shares two directories "d1" and "d2". User "a" is not able to successfully move a subdirectory of "d1" to a location anywhere in "d2".

[rullzer]

You move it via webdav I assume? Or in the webinterface?

[rullzer]

Mmmm I'm getting a permission denied (not ideal but at least no data is lost)

Steps I took

1. create user0
2. user0 creates a
3. user0 creates b
4. user0 creates b\c
5. create user1
6. user0 shares a with user1
7. user0 shares b with user1
8. user1 logs in see a and b
9. via webdav user1 tries to move b\c to a\c
10. Permission denied

[fgrsnau]

No, I move the files with the web ui (which seems to issue a MOVE webdav call).

I perform exactly the same steps as @rullzer described. I don’t get a Permission denied failure (but the very generic error message "Could not move $DIRECTORY") and the files are moved physically (user1 has full r/w permissions on both shares a and b).

Additionally, I think that this operation should not lead to any error and has to be implemented correctly as moving folders between shares is a very common workflow. A workaround is to let user0 move b/c to d/c and afterwards d/c to b/c where d/ is a "normal" folder owned by user0.

[rullzer]

@fgrsnau I think I was actually hitting https://github.com/nextcloud/server/issues/4327 I'll retest later.

[MorrisJobke]

Should be fixed with #4329

[rullzer]

I'm not so sure @MorrisJobke. Sice hte bug that is fixes by #4329 was not really present in 11 yet.

[icewind1991]

Some quick testing suggests that this on is fixed now

[fgrsnau]

I tried with current master branch and moving directories between shares is working! (Note that 983210de2e2798a70d was not working.)

Unfortunately, applying #4329 to the current stable release does not help. It is still not working.

[rullzer]

@fgrsnau mmm good to know it is fixed. But not so good that we don't know why. Are you familiar with git? And if so could you do a bisect to find out when we fixed it?