search

nextcloud / server

☁️ Nextcloud server, a safe home for all your data
https://nextcloud.com
GNU Affero General Public License v3.0
27.31k stars 4.06k forks source link

Command-line clients fail to download files after upgrade to Nextcloud 13 #8367

Closed BastiTee closed 6 years ago

BastiTee commented 6 years ago

Steps to reproduce

Prerequisite: I have a self-hosted nextcloud instance running, that is accessible via files.mydomain.com and anotherdomain.com. I was using the command-line calls as described below for many weeks without problems. After upgrading from NC 11 to NC 13 via NC 12, the command line clients don't download any files anymore. The GUI clients still work, including an Ubuntu-based Owncloud client as well as my Android client.

Steps:

nextcloud nextcloud2

#!/bin/bash

function run_test() {

    domain=$1
    client=$2

    rm -rf /tmp/oc-test
    mkdir -p /tmp/oc-test

    $client \
        -u username \
        -p 'password' \
        /tmp/oc-test \
        https://$domain/remote.php/webdav/Test/

    [ -f /tmp/oc-test/hello-world.txt ] && result="success" || result="failure"

    echo "test for domain '$domain' with client '$client': $result"
}

run_test "files.mydomain.com" "owncloudcmd" 
run_test "files.mydomain.com" "nextcloudcmd" 
run_test "anotherdomain.com" "owncloudcmd" 
run_test "anotherdomain.com" "nextcloudcmd"

Expected behaviour

All four tests should not fail, i.e., the client should download the hello-world.txt file.

Actual behaviour

Files are not downloaded. See log files:

02-15 10:02:23:518 [ info sync.accessmanager ]: 2 "" "https://files.mydomain.com/ocs/v1.php/cloud/capabilities?format=json" has X-Request-ID "2c674aea-ec8d-4613-b7eb-9828805cd81d"
02-15 10:02:23:519 [ info sync.networkjob ]:    OCC::JsonApiJob created for "https://files.mydomain.com" + "ocs/v1.php/cloud/capabilities" ""
02-15 10:02:24:543 [ info sync.networkjob.jsonapi ]:    JsonApiJob of QUrl("https://files.mydomain.com/ocs/v1.php/cloud/capabilities?format=json") FINISHED WITH STATUS QNetworkReply::NetworkError(NoError) ""
02-15 10:02:24:544 [ info sync.clientproxy ]:   Set proxy configuration to use system configuration
02-15 10:02:24:544 [ info sync.engine ]:    There are 13137354752 bytes available at "/tmp/oc-test/"
02-15 10:02:24:544 [ info sync.engine ]:    New sync (no sync journal exists)
02-15 10:02:24:544 [ info sync.engine ]:    "Using Qt 5.6.2 SSL library OpenSSL 1.0.2g  1 Mar 2016 on Ubuntu 17.10"
02-15 10:02:24:545 [ info sync.database ]:  sqlite3 version "3.19.3"
02-15 10:02:24:545 [ info sync.database ]:  sqlite3 journal_mode= "wal"
02-15 10:02:24:560 [ info sync.database ]:  possibleUpgradeFromMirall_1_5 detected!
02-15 10:02:24:567 [ info sync.database ]:  Forcing remote re-discovery by deleting folder Etags
02-15 10:02:24:568 [ info sync.engine ]:    NOT Using Selective Sync
02-15 10:02:24:568 [ info sync.engine ]:    #### Discovery start ####################################################
02-15 10:02:24:568 [ info sync.engine ]:    Server "" 
02-15 10:02:24:569 [ info sync.accessmanager ]: 6 "PROPFIND" "https://files.mydomain.com/remote.php/dav/files/username/Test" has X-Request-ID "2a507f38-6e41-44af-a691-ec0db7b4876b"
02-15 10:02:24:569 [ info sync.networkjob ]:    OCC::LsColJob created for "https://files.mydomain.com" + "/Test" "OCC::DiscoverySingleDirectoryJob"
02-15 10:02:25:360 [ warning sync.networkjob ]: QNetworkReply::NetworkError(ContentNotFoundError) "Server replied \"404 Not Found\" to \"PROPFIND https://files.mydomain.com/remote.php/dav/files/username/Test\"" QVariant(int, 404)
02-15 10:02:25:360 [ info sync.networkjob.lscol ]:  LSCOL of QUrl("https://files.mydomain.com/remote.php/dav/files/username/Test") FINISHED WITH STATUS QNetworkReply::NetworkError(ContentNotFoundError) "Server replied \"404 Not Found\" to \"PROPFIND https://files.mydomain.com/remote.php/dav/files/username/Test\""
02-15 10:02:25:360 [ warning sync.discovery ]:  LSCOL job error "Error transferring https://files.mydomain.com/remote.php/dav/files/username/Test - server replied: Not Found" 404 QNetworkReply::NetworkError(ContentNotFoundError)
02-15 10:02:25:360 [ warning sync.engine ]: ERROR during  csync_update :  "CSync failed to access "
02-15 10:02:25:361 [ info sync.database ]:  Closing DB "/tmp/oc-test/._sync_3e02db10375e.db"
02-15 10:02:25:376 [ info sync.engine ]:    CSync run took  807 ms
02-15 10:02:25:376 [ info sync.database ]:  Closing DB "/tmp/oc-test/._sync_3e02db10375e.db"

/tmp/oc-test
/tmp/oc-test/._sync_3e02db10375e.db

test for domain 'files.mydomain.com' with client 'owncloudcmd': failure

[9] csync_exclude_load  Adding entry: *~
[9] csync_exclude_load  Adding entry: ~$*
[9] csync_exclude_load  Adding entry: .~lock.*
[9] csync_exclude_load  Adding entry: ~*.tmp
[9] csync_exclude_load  Adding entry: ]*.~*
*9] csync_exclude_load  Adding entry: ]Icon
[9] csync_exclude_load  Adding entry: ].DS_Store
[9] csync_exclude_load  Adding entry: ].ds_store
[9] csync_exclude_load  Adding entry: ._*
[9] csync_exclude_load  Adding entry: ]Thumbs.db
[9] csync_exclude_load  Adding entry: desktop.ini
[9] csync_exclude_load  Adding entry: System Volume Information
[9] csync_exclude_load  Adding entry: .*.sw?
[9] csync_exclude_load  Adding entry: .*.*sw?
[9] csync_exclude_load  Adding entry: ].TemporaryItems
[9] csync_exclude_load  Adding entry: ].Trashes
[9] csync_exclude_load  Adding entry: ].DocumentRevisions-V100
[9] csync_exclude_load  Adding entry: ].Trash-*
[9] csync_exclude_load  Adding entry: .fseventd
[9] csync_exclude_load  Adding entry: .apdisk
[9] csync_exclude_load  Adding entry: .htaccess
[9] csync_exclude_load  Adding entry: .directory
[9] csync_exclude_load  Adding entry: *.part
[9] csync_exclude_load  Adding entry: *.filepart
[9] csync_exclude_load  Adding entry: *.crdownload
[9] csync_exclude_load  Adding entry: *.kate-swp
[9] csync_exclude_load  Adding entry: *.gnucash.tmp-*
[9] csync_exclude_load  Adding entry: .synkron.*
[9] csync_exclude_load  Adding entry: .sync.ffs_db
[9] csync_exclude_load  Adding entry: .symform
[9] csync_exclude_load  Adding entry: .symform-store
[9] csync_exclude_load  Adding entry: .fuse_hidden*
[9] csync_exclude_load  Adding entry: *.unison
[9] csync_exclude_load  Adding entry: .nfs*
[9] csync_exclude_load  Adding entry: My Saved Places.
[6] csync_statedb_load  sqlite3 version "3.19.3"
[6] csync_statedb_load  Success
[7] csync_memstat_check  Memory: 571992K total size, 22448K resident, 20260K shared
[9] csync_walker  file: /tmp/oc-test/._sync_3e02db10375e.db-shm [inode=1707082 size=32768]
[9] _csync_detect_update  ._sync_3e02db10375e.db-shm excluded  (1)
[9] csync_walker  file: /tmp/oc-test/._sync_3e02db10375e.db-wal [inode=1707076 size=127752]
[9] _csync_detect_update  ._sync_3e02db10375e.db-wal excluded  (1)
[9] csync_walker  file: /tmp/oc-test/._sync_3e02db10375e.db [inode=1707059 size=4096]
[9] _csync_detect_update  ._sync_3e02db10375e.db excluded  (1)
[9] csync_ftw   <= Closing walk for /tmp/oc-test with read_from_db 0
[8] csync_update  Update detection for local replica took 0,00 seconds walking 0 files.
[7] csync_memstat_check  Memory: 572020K total size, 22448K resident, 20260K shared
[6] csync_statedb_close  sqlite3_close=0

/tmp/oc-test
/tmp/oc-test/._sync_3e02db10375e.db

test for domain 'files.mydomain.com' with client 'nextcloudcmd': failure

02-15 10:02:26:712 [ info sync.accessmanager ]: 2 "" "https://anotherdomain.com/ocs/v1.php/cloud/capabilities?format=json" has X-Request-ID "a38ca35f-f7a5-49bd-8a3c-24e63aa498e7"
02-15 10:02:26:713 [ info sync.networkjob ]:    OCC::JsonApiJob created for "https://anotherdomain.com" + "ocs/v1.php/cloud/capabilities" ""
02-15 10:02:27:407 [ info sync.networkjob.jsonapi ]:    JsonApiJob of QUrl("https://anotherdomain.com/ocs/v1.php/cloud/capabilities?format=json") FINISHED WITH STATUS QNetworkReply::NetworkError(NoError) ""
02-15 10:02:27:408 [ info sync.clientproxy ]:   Set proxy configuration to use system configuration
02-15 10:02:27:408 [ info sync.engine ]:    There are 13137354752 bytes available at "/tmp/oc-test/"
02-15 10:02:27:408 [ info sync.engine ]:    New sync (no sync journal exists)
02-15 10:02:27:408 [ info sync.engine ]:    "Using Qt 5.6.2 SSL library OpenSSL 1.0.2g  1 Mar 2016 on Ubuntu 17.10"
02-15 10:02:27:409 [ info sync.database ]:  sqlite3 version "3.19.3"
02-15 10:02:27:409 [ info sync.database ]:  sqlite3 journal_mode= "wal"
02-15 10:02:27:423 [ info sync.database ]:  possibleUpgradeFromMirall_1_5 detected!
02-15 10:02:27:429 [ info sync.database ]:  Forcing remote re-discovery by deleting folder Etags
02-15 10:02:27:429 [ info sync.engine ]:    NOT Using Selective Sync
02-15 10:02:27:429 [ info sync.engine ]:    #### Discovery start ####################################################
02-15 10:02:27:429 [ info sync.engine ]:    Server "" 
02-15 10:02:27:430 [ info sync.accessmanager ]: 6 "PROPFIND" "https://anotherdomain.com/remote.php/dav/files/username/Test" has X-Request-ID "8eada83e-958d-421a-99f7-f5d5a43cfe82"
02-15 10:02:27:431 [ info sync.networkjob ]:    OCC::LsColJob created for "https://anotherdomain.com" + "/Test" "OCC::DiscoverySingleDirectoryJob"
02-15 10:02:28:025 [ warning sync.networkjob ]: QNetworkReply::NetworkError(ContentNotFoundError) "Server replied \"404 Not Found\" to \"PROPFIND https://anotherdomain.com/remote.php/dav/files/username/Test\"" QVariant(int, 404)
02-15 10:02:28:025 [ info sync.networkjob.lscol ]:  LSCOL of QUrl("https://anotherdomain.com/remote.php/dav/files/username/Test") FINISHED WITH STATUS QNetworkReply::NetworkError(ContentNotFoundError) "Server replied \"404 Not Found\" to \"PROPFIND https://anotherdomain.com/remote.php/dav/files/username/Test\""
02-15 10:02:28:025 [ warning sync.discovery ]:  LSCOL job error "Error transferring https://anotherdomain.com/remote.php/dav/files/username/Test - server replied: Not Found" 404 QNetworkReply::NetworkError(ContentNotFoundError)
02-15 10:02:28:025 [ warning sync.engine ]: ERROR during  csync_update :  "CSync failed to access "
02-15 10:02:28:025 [ info sync.database ]:  Closing DB "/tmp/oc-test/._sync_c74d5f875239.db"
02-15 10:02:28:038 [ info sync.engine ]:    CSync run took  609 ms
02-15 10:02:28:039 [ info sync.database ]:  Closing DB "/tmp/oc-test/._sync_c74d5f875239.db"

/tmp/oc-test
/tmp/oc-test/._sync_c74d5f875239.db

test for domain 'anotherdomain.com' with client 'owncloudcmd': failure

[9] csync_exclude_load  Adding entry: *~
[9] csync_exclude_load  Adding entry: ~$*
[9] csync_exclude_load  Adding entry: .~lock.*
[9] csync_exclude_load  Adding entry: ~*.tmp
[9] csync_exclude_load  Adding entry: ]*.~*
*9] csync_exclude_load  Adding entry: ]Icon
[9] csync_exclude_load  Adding entry: ].DS_Store
[9] csync_exclude_load  Adding entry: ].ds_store
[9] csync_exclude_load  Adding entry: ._*
[9] csync_exclude_load  Adding entry: ]Thumbs.db
[9] csync_exclude_load  Adding entry: desktop.ini
[9] csync_exclude_load  Adding entry: System Volume Information
[9] csync_exclude_load  Adding entry: .*.sw?
[9] csync_exclude_load  Adding entry: .*.*sw?
[9] csync_exclude_load  Adding entry: ].TemporaryItems
[9] csync_exclude_load  Adding entry: ].Trashes
[9] csync_exclude_load  Adding entry: ].DocumentRevisions-V100
[9] csync_exclude_load  Adding entry: ].Trash-*
[9] csync_exclude_load  Adding entry: .fseventd
[9] csync_exclude_load  Adding entry: .apdisk
[9] csync_exclude_load  Adding entry: .htaccess
[9] csync_exclude_load  Adding entry: .directory
[9] csync_exclude_load  Adding entry: *.part
[9] csync_exclude_load  Adding entry: *.filepart
[9] csync_exclude_load  Adding entry: *.crdownload
[9] csync_exclude_load  Adding entry: *.kate-swp
[9] csync_exclude_load  Adding entry: *.gnucash.tmp-*
[9] csync_exclude_load  Adding entry: .synkron.*
[9] csync_exclude_load  Adding entry: .sync.ffs_db
[9] csync_exclude_load  Adding entry: .symform
[9] csync_exclude_load  Adding entry: .symform-store
[9] csync_exclude_load  Adding entry: .fuse_hidden*
[9] csync_exclude_load  Adding entry: *.unison
[9] csync_exclude_load  Adding entry: .nfs*
[9] csync_exclude_load  Adding entry: My Saved Places.
[6] csync_statedb_load  sqlite3 version "3.19.3"
[6] csync_statedb_load  Success
[7] csync_memstat_check  Memory: 571992K total size, 25144K resident, 20920K shared
[9] csync_walker  file: /tmp/oc-test/._sync_c74d5f875239.db-wal [inode=1707076 size=127752]
[9] _csync_detect_update  ._sync_c74d5f875239.db-wal excluded  (1)
[9] csync_walker  file: /tmp/oc-test/._sync_c74d5f875239.db-shm [inode=1707082 size=32768]
[9] _csync_detect_update  ._sync_c74d5f875239.db-shm excluded  (1)
[9] csync_walker  file: /tmp/oc-test/._sync_c74d5f875239.db [inode=1707059 size=4096]
[9] _csync_detect_update  ._sync_c74d5f875239.db excluded  (1)
[9] csync_ftw   <= Closing walk for /tmp/oc-test with read_from_db 0
[8] csync_update  Update detection for local replica took 0,00 seconds walking 0 files.
[7] csync_memstat_check  Memory: 572020K total size, 25144K resident, 20920K shared
[6] csync_statedb_close  sqlite3_close=0

/tmp/oc-test
/tmp/oc-test/._sync_c74d5f875239.db

test for domain 'anotherdomain.com' with client 'nextcloudcmd': failure

Server configuration

Operating system: Ubuntu 17.10

Web server: apache2 2.4.18

Database: postgresql 9.5

PHP version: php 7.0

Nextcloud version: 13.0.0.14

Updated from an older Nextcloud/ownCloud or fresh install: Updated from 11 via 12

Where did you install Nextcloud from: https://nextcloud.com/install/ manually installed

Signing status:

Signing status 
No errors have been found.

List of activated apps:

App list 
Enabled:
  - activity: 2.6.1
  - bruteforcesettings: 1.0.3
  - comments: 1.3.0
  - dav: 1.4.6
  - federatedfilesharing: 1.3.1
  - federation: 1.3.0
  - files: 1.8.0
  - files_pdfviewer: 1.2.0
  - files_sharing: 1.5.0
  - files_texteditor: 2.5.1
  - files_trashbin: 1.3.0
  - files_versions: 1.6.0
  - files_videoplayer: 1.2.0
  - firstrunwizard: 2.2.1
  - gallery: 18.0.0
  - logreader: 2.0.0
  - lookup_server_connector: 1.1.0
  - nextcloud_announcements: 1.2.0
  - notifications: 2.1.2
  - oauth2: 1.1.0
  - password_policy: 1.3.0
  - provisioning_api: 1.3.0
  - serverinfo: 1.3.0
  - sharebymail: 1.3.0
  - survey_client: 1.1.0
  - systemtags: 1.3.0
  - theming: 1.4.1
  - twofactor_backupcodes: 1.2.3
  - updatenotification: 1.3.0
  - workflowengine: 1.3.0
Disabled:
  - admin_audit
  - encryption
  - files_external
  - user_external
  - user_ldap

Nextcloud configuration:

Config report 
{
    "system": {
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "files.mydomain.com",
            "anotherdomain.com"
        ],
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "overwrite.cli.url": "http:\/\/anotherdomain.com",
        "dbtype": "pgsql",
        "version": "13.0.0.14",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbport": "",
        "dbtableprefix": "oc_",
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "logtimezone": "UTC",
        "installed": true,
        "memcache.local": "\\OC\\Memcache\\APCu",
        "updater.secret": "***REMOVED SENSITIVE VALUE***",
        "theme": "",
        "loglevel": 2,
        "maintenance": false
    }
}

Are you using external storage, if yes which one: local

Are you using encryption: no

Are you using an external user-backend, if yes which one: no

BastiTee commented 6 years ago

Stuff I tried after initially filing issue without any change to the problem described:

BastiTee commented 6 years ago

I found the reason. Apparently the username-part used in the WebDAV path of the request is now case-sensitive. On the file system my account's subfolder was /files/Username/ even though I login with 'username' all-lowercase all the time. The casing was not relevant for the client-connection in NX11.