nextcloud / server

☁️ Nextcloud server, a safe home for all your data
https://nextcloud.com
GNU Affero General Public License v3.0
26.71k stars 4k forks source link

Error reporting for too long filepaths (>255 chars) #9907

Open jzaers opened 6 years ago

jzaers commented 6 years ago

Steps to reproduce

  1. Use filenames with a resulting path longer than 255 characters
  2. This seems to result in various strange side effects like move not executed correctly and the following error message:
{"reqId":"xZAVDUDUMFn0yANyU6rA","level":4,"time":"2018-06-14T06:52:08+00:00","remoteAddr":"10.0.50.4","user":"87c48f84-4b31-4c8e-aac8-1c554f1b500b","app":"webdav","method":"MOVE","url":"\/owncloud\/remote.php\/dav\/files\/LONG_FILENAME_REPLACED_WITH_247_CHARACTERS","message":"Exception: {\"Exception\":\"Doctrine\\\\DBAL\\\\Exception\\\\DriverException\",\"Message\":\"An exception occurred while executing 'UPDATE \\\"oc_properties\\\" SET \\\"propertypath\\\" = ? WHERE \\\"userid\\\" = ? AND \\\"propertypath\\\" = ?' with params [\\\"files\\\\\\\LONG_FILENAME_REPLACED_WITH_247_CHARACTERS", \\\"LONG_FILENAME_REPLACED_WITH_247_CHARACTERS"]:\\n\\nSQLSTATE[22001]: String data, right truncated: **7 ERROR:  value too long for type character varying(255)**\",\"Code\":0,\"Trace\":\"#0 \\\/opt\\\/nextcloud\\\/3rdparty\\\/doctrine\\\/dbal\\\/lib\\\/Doctrine\\\/DBAL\\\/DBALException.php(128): Doctrine\\\\DBAL\\\\Driver\\\\AbstractPostgreSQLDriver->convertException('An exception oc...', Object(Doctrine\\\\DBAL\\\\Driver\\\\PDOException))\\n#1 \\\/opt\\\/nextcloud\\\/3rdparty\\\/doctrine\\\/dbal\\\/lib\\\/Doctrine\\\/DBAL\\\/Statement.php(178): Doctrine\\\\DBAL\\\\DBALException::driverExceptionDuringQuery(Object(Doctrine\\\\DBAL\\\\Driver\\\\PDOPgSql\\\\Driver), Object(Doctrine\\\\DBAL\\\\Driver\\\\PDOException), 'UPDATE \\\"oc_prop...', Array)\\n#2 \\\/opt\\\/nextcloud\\\/apps\\\/dav\\\/lib\\\/DAV\\\/CustomPropertiesBackend.php(179): Doctrine\\\\DBAL\\\\Statement->execute(Array)\\n#3 \\\/opt\\\/nextcloud\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/PropertyStorage\\\/Plugin.php(147): OCA\\\\DAV\\\\DAV\\\\CustomPropertiesBackend->move('files\\\/87c48f84-...', 'files\\\/87c48f84-...')\\n#4 [internal function]: Sabre\\\\DAV\\\\PropertyStorage\\\\Plugin->afterMove('files\\\/87c48f84-...', 'files\\\/87c48f84-...')\\n#5 \\\/opt\\\/nextcloud\\\/3rdparty\\\/sabre\\\/event\\\/lib\\\/EventEmitterTrait.php(105): call_user_func_array(Array, Array)\\n#6 \\\/opt\\\/nextcloud\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/CorePlugin.php(648): Sabre\\\\Event\\\\EventEmitter->emit('afterMove', Array)\\n#7 [internal function]: Sabre\\\\DAV\\\\CorePlugin->httpMove(Object(Sabre\\\\HTTP\\\\Request), Object(Sabre\\\\HTTP\\\\Response))\\n#8 \\\/opt\\\/nextcloud\\\/3rdparty\\\/sabre\\\/event\\\/lib\\\/EventEmitterTrait.php(105): call_user_func_array(Array, Array)\\n#9 \\\/opt\\\/nextcloud\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/Server.php(479): Sabre\\\\Event\\\\EventEmitter->emit('method:MOVE', Array)\\n#10 \\\/opt\\\/nextcloud\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/Server.php(254): Sabre\\\\DAV\\\\Server->invokeMethod(Object(Sabre\\\\HTTP\\\\Request), Object(Sabre\\\\HTTP\\\\Response))\\n#11 \\\/opt\\\/nextcloud\\\/apps\\\/dav\\\/lib\\\/Server.php(258): Sabre\\\\DAV\\\\Server->exec()\\n#12 \\\/opt\\\/nextcloud\\\/apps\\\/dav\\\/appinfo\\\/v2\\\/remote.php(33): OCA\\\\DAV\\\\Server->exec()\\n#13 \\\/opt\\\/nextcloud\\\/remote.php(164): require_once('\\\/opt\\\/nextcloud\\\/...')\\n#14 {main}\",\"File\":\"\\\/opt\\\/nextcloud\\\/3rdparty\\\/doctrine\\\/dbal\\\/lib\\\/Doctrine\\\/DBAL\\\/Driver\\\/AbstractPostgreSQLDriver.php\",\"Line\":91}","userAgent":"Mozilla\/5.0 (Macintosh) mirall\/2.4.1 (build 9367)","version":"12.0.6.1"}

Expected behaviour

I would like to

Actual behaviour

Files with too long paths are synced to the server and encounter problems on later move operations. It looks like the move is executed like a copy and the original file stays in the source directory. But this is not happening always. I assume that it depends on the resulting file path in the destination directory.

Server configuration

Operating system: ubuntu 14.04 LTS

Web server: Apache 2.4

Database: postgresql 9.4

PHP version: php 5.6

Nextcloud version: (see Nextcloud admin page) see above, 12.0.6.1, (I updated the server to 13 afterwards, a few server configs below are from the new server, I try to reproduce this under 13.)

Updated from an older Nextcloud/ownCloud or fresh install: Migration from owncloud 9 or 10.

Where did you install Nextcloud from:

Signing status:

Signing status No errors have been found.

List of activated apps: This is from the server after upgrade to v 13.

Enabled:

Nextcloud configuration:

Config report { "system": { "instanceid": "***REMOVED SENSITIVE VALUE***", "passwordsalt": "***REMOVED SENSITIVE VALUE***", "secret": "***REMOVED SENSITIVE VALUE***", "trusted_domains": [ "***REMOVED SENSITIVE VALUE***", "***REMOVED SENSITIVE VALUE***" ], "datadirectory": "***REMOVED SENSITIVE VALUE***", "overwrite.cli.url": "***REMOVED SENSITIVE VALUE***", "version": "13.0.4.0", "dbtype": "pgsql", "dbhost": "***REMOVED SENSITIVE VALUE***", "dbname": "***REMOVED SENSITIVE VALUE***", "dbuser": "***REMOVED SENSITIVE VALUE***", "dbpassword": "***REMOVED SENSITIVE VALUE***", "dbtableprefix": "oc_", "logtimezone": "UTC", "installed": true, "ldapIgnoreNamingRules": false, "loglevel": 3, "log_type": "owncloud", "logfile": "\/opt\/oc-data\/nextcloud.log", "maintenance": false, "singleuser": false, "memcache.local": "\\OC\\Memcache\\Redis", "filelocking.enabled": "true", "memcache.distributed": "\\OC\\Memcache\\Redis", "memcache.locking": "\\OC\\Memcache\\Redis", "redis": { "host": "***REMOVED SENSITIVE VALUE***", "port": 6379, "timeout": 0, "dbindex": 0 }, "trashbin_retention_obligation": "auto, 60", "versions_retention_obligation": "auto", "theme": "", "updater.release.channel": "production", "ldapProviderFactory": "\\OCA\\User_LDAP\\LDAPProviderFactory", "updater.secret": "***REMOVED SENSITIVE VALUE***" } }

Are you using external storage, if yes which one: local/smb/sftp/... NO

Are you using encryption: yes/no NO

Are you using an external user-backend, if yes which one: LDAP/ActiveDirectory/Webdav/...

YES, LDAP Apple OD, config not pasted, as not relevant.

Client configuration

Browser:

Operating system: MACOS, nextcloud client 2.3.3

Logs

Web server error log

not relevant

Nextcloud log (data/nextcloud.log)

see above

Browser log

not relevant

rafacouto commented 5 years ago

I confirm this issue with docker images:

Just applied this path manually and waiting for results:

alter table properties alter column propertypath type character varying(1024);
J0WI commented 4 years ago

I'm able to reproduce during rename:

An exception occurred while executing 'UPDATE "oc_properties" SET "propertypath" = ? WHERE "userid" = ? AND "propertypath" = ?' with params ["***very long test path***", "***test user***", "***long test path***"]:
SQLSTATE[22001]: String data, right truncated: 7 ERROR:  value too long for type character 

https://github.com/nextcloud/server/blob/ee5e76780174331f3447d6bea4f4c55f1e0cfef6/core/Migrations/Version13000Date20170718121200.php#L298-L302

kesselb commented 4 years ago

https://github.com/nextcloud/server/blob/ee5e76780174331f3447d6bea4f4c55f1e0cfef6/core/Migrations/Version13000Date20170718121200.php#L146-L149

We should probably align this with path. Would you mind to create a pull request? ;)

J0WI commented 4 years ago

What is the right place for this change? I assume this is for 18 only?

kesselb commented 4 years ago

Yes. I probably forgot that this kind of change requires a bit more work. Just changing the value over there will not work :see_no_evil:

  1. php occ migrations:generate core 18000 use occ to generate a migration

  2. Delete preSchemaChange and postSchemaChange

        /** @var ISchemaWrapper $schema */
        $schema = $schemaClosure();

        if ($schema->hasTable('properties')) {
            $table = $schema->getTable('properties');

            $table->changeColumn('propertypath', [
                'notnull' => true,
                'length' => 4000,
                'default' => '',
            ]);
        }

        return $schema;
  1. Probably something like above for changeSchema but have not tested it.

  2. php occ migrations:execute core 18000Date201911xzy run the migration

ChristophWurst commented 4 years ago

Discussed this with @rullzer briefly.

J0WI commented 4 years ago

For 19 we can change the original migration so new installations get a more appropriate column width by default

Since 19 has already been released I'll tag it for 20.

szaimen commented 1 year ago

Hi, please update to 24.0.8 or better 25.0.2 and report back if it fixes the issue. Thank you!

jzaers commented 1 year ago

Hi,

sorry, I’m no longer using nextcloud, so I can’t check it.

Cheers, Jo.

— Grüntenweg 8a 86859 Igling Tel. +49 172 83 800 82

HINWEIS: Dies ist eine vertrauliche Nachricht und nur für den Adressaten bestimmt. Es ist nicht erlaubt, diese Nachricht zu kopieren oder Dritten zugänglich zu machen. Sollten Sie diese Nachricht irrtümlich erhalten haben, bitte ich um Ihre Mitteilung per E-Mail oder unter der oben angegebenen Telefonnummer.

Am 09.01.2023 um 12:40 schrieb Simon L. @.***>:

Hi, please update to 24.0.8 or better 25.0.2 and report back if it fixes the issue. Thank you!

— Reply to this email directly, view it on GitHub https://github.com/nextcloud/server/issues/9907#issuecomment-1375500552, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJGS33I6BKR3SGUV5F2Q4P3WRP2MRANCNFSM4FFT2J5A. You are receiving this because you authored the thread.

J0WI commented 1 year ago

This is still an issue.

susnux commented 8 months ago

This is still an issue.

@J0WI are you sure? This should have been fixed with https://github.com/nextcloud/server/pull/19242