Connection closed - Operation cancelled

[RandieM]

I got the following error on my ownCloud desktop client, while trying to synchronise several files from one computer to another:

Connection closed Operation cancelled

My desktop client version is 1.6.4 running on Fedora 20, but updating to the latest 1.7.0 did not make any difference.

The ownCloud server version I am using is 7.0.2 and while the desktop client is trying to access the files, the server logs HTTP GET 302.

I moved the "problematic" files to a newly created folder (inside the owncloud folder) and deleted them from their initial location and finally the desktop client was able to synchronise them in their new location.

[karlitschek]

Do you see something else in the owncloud.log?

[RandieM]

I can't see anything in the owncloud.log Trying to synchronise the same files from a third computer, causes the same error.

[PVince81]

@RandieM did you set "loglevel" to 0 in config.php when checking the logs ?

Can you post the "oc_filecache" entries from the DB for the broken files ?

[PVince81]

Also, you should use the issue template: https://raw.github.com/owncloud/core/master/issue_template.md

We're missing important information like whether you're using encryption, external storage, etc.

[RandieM]

Thanks both for your answers.

You can find below the completed issue template. Let me know if you need more information.

The oc_filecache did not contain any entries related to the "problematic" files. Basically, the files are OK; they just don't get synchronised for some reason and it is possible that they finally do get synchronised at a later stage (several hours to a few days).

Steps to reproduce

1. Copy a file (so far I've had issues with .tif, .png, .svg) into the owncloud sync folder (typically user/ownCloud).
2. It happens randomly.

Expected behaviour

File should be uploaded to the owncloud server and synchronised among devices running the sync client

Actual behaviour

File does not get uploaded and the error message "connection closed - operation cancelled" appears in the sync client

Server configuration

Operating system: Debian 7

Web server: nginx 1.6.2

Database: PostgreSQL 9.1.14

PHP version: 5.4.34

ownCloud version: (see ownCloud admin page) 7.0.2

Updated from an older ownCloud or fresh install: updated from older

List of activated apps: default apps, encryption app

The content of config/config.php:

<?php
$CONFIG = array (
  'instanceid' => '<DELETED>',
  'passwordsalt' => '<DELETED>',
  'datadirectory' => '/mnt/data',
  'dbtype' => 'pgsql',
  'version' => '7.0.2.1',
  'dbname' => '<DELETED>',
  'dbhost' => 'localhost',
  'dbtableprefix' => 'oc_',
  'dbuser' => '<DELETED>',
  'dbpassword' => '<DELETED>',
  'installed' => true,
  'forcessl' => true,
  'theme' => '<DELETED>',
  'enable_avatars' => false,
  'mail_domain' => '<DELETED>',
  'mail_smtpmode' => 'smtp',
  'log_authfailip' => true,
  'loglevel' => '0',
  'maintenance' => false,
  'customclient_desktop' => '<DELETED>',
  'trusted_domains' =>
  array (
    0 => '<DELETED>',
  ),
  'share_folder' => '/Shared',
  'overwritewebroot' => '/',
  'overwriteprotocol' => 'https',
);

Are you using external storage, if yes which one: no

Are you using encryption: yes

Client configuration

Browser: n/a

Operating system: Windows 7, Fedora 20

Logs

Web server error log

no error appears

ownCloud log (data/owncloud.log)

{"reqId":"546a6bf4c3591","app":"webdav","message":"Sabre\\DAV\\Exception\\NotAuthenticated: No basic authentication headers were found","level":0,"time":"2014-11-17T21:43:16+00:00","method":"PROPFIND","url":"\/remote.php\/webdav"} 

Browser log

n/a

[RandieM]

any news may be?

[RandieM]

Just an update: Our ownCloud version has been upgraded to 7.0.3, but the problem is still there.

[hostingnuggets]

I also encounter this issue with ownCloud 7.0.4. Is anyone working on this bug?

[ghost]

I am receiving this error on ownCloud 8.0 - freshly installed today.

Edit: Updated the desktop client to 1.8.0 and am no longer receiving this issue.

[mscodemonkey]

I was having this problem with ownCloud 7.04 and 1.8.0 client. Reverted to 1.7.1.1655 client and the problem is no longer there.

[gbrlwck]

i'm experiencing the same problem after upgrading from oc7 to oc8. neither 1.7.1.1655 nor 1.8 client do work... i guess it's a server issue, cause it seems not working from my phone either. the logs remain empty/not informative at all. webDAV seems to be working though.

[frankuit]

i can confirm this, we have the same problem. (client 1.8.0, Debian 64 Server 8.0.3-1) it happens more often when a user has a large number of small files in his repository.

[reneretz]

Same problem here. Client: 1.8.1 (build 2335) Server: 8.0.3 (stable)

[PVince81]

So do you guys all have the same config as @RandieM here https://github.com/owncloud/core/issues/12193#issuecomment-63469787 ? NGinx, Postgres ?

More details could help cross-reference environments to find out how to reproduce the issue.

Did you try increasing the PHP process timeout in the config ?

If the server just suddenly closes the connection without logging anything, it sounds like a timeout issue. Does the error happen after several minutes or directly after a few seconds ?

[sleepysmurf]

can confirm same problem - 1.8.1 sync client on owncloud 8.0 "Operation Cancelled Connection Closed"

[ghost]

I think I have the same problem with Owncloud 8.03 and client 1.8.0 (build 38ef52)

Here is the log from the client (owncloud --logwindow), at the moment of the error.

I think the error comes from * Discarded as is hidden! "/media/stephane/CLE16MP3/2010/.Calvin Harris - Thinking About You.mp3.~53f39eab" *

I checked on my disk and this file is not hidden and doesn't begin with a dot. The filepath indicated in the log is therefore wrong (?)

    05-13 22:08:31:624 0x153a6f0 No more remote ETag check jobs to schedule. 
    05-13 22:08:31:712 0x153a6f0 void OCC::PropagateDownloadFileQNAM::slotGetFinished()  QUrl( "https://cloud.XXXXX.eu/remote.php/webdav/Musique STD/2010/C2C - Happy.mp3" )  FINISHED WITH STATUS 0 "" 
    05-13 22:08:31:813 0x153a6f0 OCC::SyncJournalFileRecord::SyncJournalFileRecord(const OCC::SyncFileItem&, const QString&) "/media/stephane/CLE16MP3/2010/C2C - Happy.mp3" Retrieved inode  5334 (previous item inode:  0 ) 
    05-13 22:08:31:814 0x153a6f0 "INSERT OR REPLACE INTO metadata (phash, pathlen, path, inode, uid, gid, mode, modtime, type, md5, fileid, remotePerm, filesize) VALUES (?1 , ?2, ?3 , ?4 , ?5 , ?6 , ?7,  ?8 , ?9 , ?10, ?11, ?12, ?13);" 502101708580962391 20 "2010/C2C - Happy.mp3" 5334 0 "1412439481" "0" "7f7f4c6aeeab013e0de645862cd5225c" "00053921oc16982b0ad5" "SRDNVW" 4176668 
    05-13 22:08:31:814 0x153a6f0 "DELETE FROM downloadinfo WHERE path=?1" "2010/C2C - Happy.mp3" 
    05-13 22:08:31:815 0x153a6f0 void OCC::SyncJournalDb::commitInternal(const QString&, bool) Transaction commit  "download file start2" and starting new transaction 
    05-13 22:08:31:816 0x153a6f0 void OCC::SyncEngine::slotJobCompleted(const OCC::SyncFileItem&) "2010/C2C - Happy.mp3" INSTRUCTION_NEW 4 "" 
    05-13 22:08:31:823 0x153a6f0 * Discarded as is hidden! "/media/stephane/CLE16MP3/2010/.C2C - Happy.mp3.~4a7c4e2" 
    05-13 22:08:31:827 0x153a6f0 detected changes in paths: QSet("/media/stephane/CLE16MP3/2010/C2C - Happy.mp3") 
    05-13 22:08:31:828 0x153a6f0 void OCC::Folder::watcherSlot(QString) Sync running, IGNORE event for  "/media/stephane/CLE16MP3/2010/C2C - Happy.mp3" 
    05-13 22:08:31:829 0x153a6f0 virtual void OCC::PropagateDownloadFileQNAM::start() "2010/Calvin Harris - Thinking About You.mp3" 2 
    05-13 22:08:31:830 0x153a6f0 "INSERT OR REPLACE INTO downloadinfo (path, tmpfile, etag, errorcount) VALUES ( ?1 , ?2, ?3, ?4 )" "2010/Calvin Harris - Thinking About You.mp3" "2010/.Calvin Harris - Thinking About You.mp3.~53f39eab" "6aa0d5374b5a95345b60fc589b57f300" 0 
    05-13 22:08:31:830 0x153a6f0 void OCC::SyncJournalDb::commitInternal(const QString&, bool) Transaction commit  "download file start" and starting new transaction 
    05-13 22:08:31:831 0x153a6f0 virtual void OCC::GETFileJob::start() OCC::BandwidthManager(0x2a101f8) false false 
    05-13 22:08:31:831 0x153a6f0 void OCC::BandwidthManager::registerDownloadJob(OCC::GETFileJob*) OCC::GETFileJob(0x2b5b110) 
    05-13 22:08:31:832 0x153a6f0 !!! OCC::GETFileJob created for  QUrl( "https://cloud.XXXXXX.eu" )  querying "Musique STD/2010/Calvin Harris - Thinking About You.mp3" 
    05-13 22:08:31:836 0x153a6f0 * Discarded as is hidden! "/media/stephane/CLE16MP3/2010/.Calvin Harris - Thinking About You.mp3.~53f39eab" 
    05-13 22:08:32:815 0x153a6f0 void OCC::AbstractNetworkJob::slotFinished() 2 "Connexion fermée" 
    05-13 22:08:32:815 0x153a6f0 void OCC::PropagateDownloadFileQNAM::slotGetFinished()  QUrl( "https://cloud.XXXXXX.eu/remote.php/webdav/Musique STD/2010/Calvin Harris - Feel So Close.mp3" )  FINISHED WITH STATUS 2 "Connexion fermée" 
    05-13 22:08:32:815 0x153a6f0 void OCC::SyncEngine::slotJobCompleted(const OCC::SyncFileItem&) "2010/Calvin Harris - Feel So Close.mp3" INSTRUCTION_NEW 1 "Connexion fermée" 
    05-13 22:08:32:819 0x153a6f0 void OCC::AbstractNetworkJob::slotFinished() 5 "Opération annulée" 
    05-13 22:08:32:819 0x153a6f0 void OCC::PropagateDownloadFileQNAM::slotGetFinished()  QUrl( "https://cloud.XXXXXXXX.eu/remote.php/webdav/Musique STD/2010/Calvin Harris - Summer.mp3" )  FINISHED WITH STATUS 5 "Opération annulée" 
    05-13 22:08:32:819 0x153a6f0 void OCC::SyncEngine::slotJobCompleted(const OCC::SyncFileItem&) "2010/Calvin Harris - Summer.mp3" INSTRUCTION_NEW 1 "Opération annulée" 
    05-13 22:08:32:822 0x153a6f0 void OCC::AbstractNetworkJob::slotFinished() 5 "Opération annulée" 
    05-13 22:08:32:822 0x153a6f0 void OCC::PropagateDownloadFileQNAM::slotGetFinished()  QUrl( "https://cloud.XXXXXXX.eu/remote.php/webdav/Musique STD/2010/Calvin Harris - Thinking About You.mp3" )  FINISHED WITH STATUS 5 "Opération annulée" 
    05-13 22:08:32:833 0x153a6f0 "DELETE FROM downloadinfo WHERE path=?1" "2010/Calvin Harris - Thinking About You.mp3" 
    05-13 22:08:32:834 0x153a6f0 void OCC::SyncEngine::slotJobCompleted(const OCC::SyncFileItem&) "2010/Calvin Harris - Thinking About You.mp3" INSTRUCTION_NEW 1 "Opération annulée" 
    05-13 22:08:32:840 0x153a6f0 * Discarded as is hidden! "/media/stephane/CLE16MP3/2010/.Calvin Harris - Thinking About You.mp3.~53f39eab" 
    05-13 22:08:32:845 0x153a6f0 void OCC::SyncJournalDb::walCheckpoint() took 0 msec 
    05-13 22:08:32:846 0x153a6f0 void OCC::SyncJournalDb::commitInternal(const QString&, bool) Transaction commit  "All Finished."  
    05-13 22:08:32:847 0x153a6f0 CSync run took  120377 
    05-13 22:08:32:848 0x153a6f0 virtual OCC::BandwidthManager::~BandwidthManager() 

[pdesanex]

I'm having the same issue running encrypted OC 8.03 on allinkl shared hosting / mySQL and OC client 1.81 ... the problem arises on all client platforms (linux, mac, win). With a fresh user account, synchronisation works flawlessly for the first couple of files, then the problem kicks in rather arbitrarily. Eventually, all files will be synchronized. The issue seems to arise more frequently, if the user has many files - c.f. https://github.com/owncloud/core/issues/12193#issuecomment-99784216 ...

[Idealtrajectory]

I am having the same issue as the above. It happens after a few seconds. It will discover for awhile, then begin the download. It will say connection closed, etc after downloading between 500kb and 4mb. To try to relieve some of the pressure, I unchecked everything that it syncs with, and am slowly introducing the files back into the system. This is very time consuming, though. I believe I have successfully synced 60mb in 3 hours. Someone mentioned a process timeout in the config. What is the config file called on Linux? Thanks again for helping. I've been tearing my hair out over this!

[fnerdwq]

Something which helped me with this issue - at least making the intervals between the cancelled operation much larger - is setting KeepAlive On in the apache config.

To be clear, it not fixing the issue.

[rwestlund]

Same issue affects me. Clients on Mac, Android, and FreeBSD all experience the same issue. Furthermore, I can't download files from the browser.

My server runs 8.0.3. It's the FreeNAS plugin. Uses Apache and SQLite, behind an Nginx reverse proxy. Problem appeared several days after upgrading to 8.0.3.

I found the problem:

randy@jorrvaskr ~/Downloads % fetch https://cumulus.textplain.net/index.php/s/uNYcaXv0EFkvqtW/download
download                                        0% of 2552 kB 6956 kBps 00m00s
fetch: download appears to be truncated: 15493/2613596 bytes

randy@jorrvaskr ~/Downloads % curl -I https://cumulus.textplain.net/index.php/s/uNYcaXv0EFkvqtW/download
HTTP/1.1 200 OK
Server: nginx/1.8.0
Date: Fri, 22 May 2015 21:04:33 GMT
Content-Type: image/jpeg
Content-Length: 2613596
Connection: keep-alive
X-Powered-By: PHP/5.4.32
Set-Cookie: oc6sh30jtvdm=4172f04245aff20db3a5e8ebb14c906f; path=/; HttpOnly
Expires: 0
Cache-Control: must-revalidate, post-check=0, pre-check=0
Pragma: public
Strict-Transport-Security: max-age=0
X-XSS-Protection: 1; mode=block
X-Content-Type-Options: nosniff
X-Frame-Options: Sameorigin
Content-Security-Policy: default-src 'self'; script-src 'self' 'unsafe-eval'; style-src 'self' 'unsafe-inline'; frame-src *; img-src *; font-src 'self' data:; media-src *; connect-src *
X-Robots-Tag: none
Content-Disposition: attachment; filename*=UTF-8''20150507_212737.jpg; filename="20150507_212737.jpg"
Content-Transfer-Encoding: binary

The Content-Length header is much larger than it should be. Feel free to hit that URL yourselves, it's public.

[rwestlund]

I was wrong, Content-Length is correct. My problem was unrelated to owncloud, although I had identical symptoms. I'm leaving my post intact in case others are having my problem.

My nginx reverse proxy was having issues. It seems I upgraded nginx but forgot to restart the service.

In /var/log/nginx-error.log:

2015/05/22 17:43:04 [crit] 795#0: *234697 mkdir() "/var/tmp/nginx/proxy_temp/6" failed (2: No such file or directory) while reading upstream, client: 98.216.247.110, server: cumulus.textplain.net, request: "GET /index.php/s/uNYcaXv0EFkvqtW/download HTTP/1.1", upstream: "https://192.168.1.220:443/index.php/s/uNYcaXv0EFkvqtW/download", host: "cumulus.textplain.net"

[Kruseltier]

Hi, I have exact the same problem.

owncloud Version: 8 Centos/Cloudlinux 6.6 Apache 2.4 + lsphp (litespeed) MySQL 5.5 SyncClient 1.8.1

It seems that it happen randomly but only when more than one desktop client is acessing the same ownCloud account.

Is there anything i can do to help for debugging ?

Greetings Dominik

[MiguelAlcaino]

Same issue here. I'm facing "Operation cancelled" as well

[netimpa]

Hi, I found the same problem, also found solution. My lab; 1) Server Owncloud 8.0 "CentOS 6.6" - "MySQL" 2) Client owncloud 1.8 - a)Ubuntu 14.10 64bits, b) MacOS 64bits Yosemite and c) Windows 7 64bits. 3) All client connected in different network vlan 3) Debug log enable in all client and owncloud-server 4) I use wireshark for scan network. 5) Upload 25 thousand files from client. I have all client display connect canceled and connect close for upload e download file.

Laboratory results; 1) My server does not show errors in the log

2) 06-02 15:38:09:260 0x7ff1eb47b9b0 OCC::SyncJournalDb::setDownloadInfo: "INSERT OR REPLACE INTO downloadinfo (path, tmpfile, etag, errorcount) VALUES ( ?1 , ?2, ?3, ?4 )" "MAC2/template_47119_mx648Ufrj227L577oZk6/screenshots/47119-1-origin.jpg" "MAC2/template_47119_mx648Ufrj227L577oZk6/screenshots/.47119-1-origin.jpg.~2eb59ad7" "da3fbda1964f19b06815d97478d13c9f" 0 06-02 15:38:09:260 0x7ff1eb47b9b0 OCC::SyncJournalDb::commitInternal: void OCC::SyncJournalDb::commitInternal(const QString &, bool) Transaction commit "download file start" and starting new transaction 06-02 15:38:09:261 0x7ff1eb47b9b0 OCC::GETFileJob::start: virtual void OCC::GETFileJob::start() OCC::BandwidthManager(0x7ff1ed9de4d8) false false 06-02 15:38:09:261 0x7ff1eb47b9b0 OCC::BandwidthManager::registerDownloadJob: void OCC::BandwidthManager::registerDownloadJob(OCC::GETFileJob *) OCC::GETFileJob(0x7ff1f0b29af0) 06-02 15:38:09:261 0x7ff1eb47b9b0 OCC::AbstractNetworkJob::start: !!! OCC::GETFileJob created for "https://owncloud.example.br" + "/MAC2/template_47119_mx648Ufrj227L577oZk6/screenshots/47119-1-origin.jpg" 06-02 15:38:09:269 0x7ff1eb47b9b0 OCC::AbstractNetworkJob::slotFinished: void OCC::AbstractNetworkJob::slotFinished() 2 "Ligação fechada" 06-02 15:38:09:269 0x7ff1eb47b9b0 OCC::PropagateDownloadFileQNAM::slotGetFinished: void OCC::PropagateDownloadFileQNAM::slotGetFinished() QUrl( "https://owncloud.example.br/remote.php/webdav/MAC2/template_47119_mx648Ufrj227L577oZk6/screenshots/47119-1-big.jpg" ) FINISHED WITH STATUS 2 "Ligação fechada" 06-02 15:38:09:269 0x7ff1eb47b9b0 OCC::SyncEngine::slotJobCompleted: void OCC::SyncEngine::slotJobCompleted(const OCC::SyncFileItem &) "MAC2/template_47119_mx648Ufrj227L577oZk6/screenshots/47119-1-big.jpg" INSTRUCTION_NEW 1 "Ligação fechada" 06-02 15:38:09:285 0x7ff1eb47b9b0 OCC::SocketApi::sendMessage: SocketApi: Sending message: "STATUS:ERROR:/Users/netadm/Documents/cubo/MAC2/template_47119_mx648Ufrj227L577oZk6/screenshots/47119-1-big.jpg" 06-02 15:38:09:286 0x7ff1eb47b9b0 OCC::AbstractNetworkJob::slotFinished: void OCC::AbstractNetworkJob::slotFinished() 2 "Ligação fechada" 06-02 15:38:09:286 0x7ff1eb47b9b0 OCC::PropagateDownloadFileQNAM::slotGetFinished: void OCC::PropagateDownloadFileQNAM::slotGetFinished() QUrl( "https://owncloud.example.br/remote.php/webdav/MAC2/template_47119_mx648Ufrj227L577oZk6/screenshots/47119-1-origin.jpg" ) FINISHED WITH STATUS 2 "Ligação fechada" 06-02 15:38:09:286 0x7ff1eb47b9b0 OCC::SyncJournalDb::setDownloadInfo: "DELETE FROM downloadinfo WHERE path=?1" "MAC2/template_47119_mx648Ufrj227L577oZk6/screenshots/47119-1-origin.jpg" 06-02 15:38:09:286 0x7ff1eb47b9b0 OCC::SyncEngine::slotJobCompleted: void OCC::SyncEngine::slotJobCompleted(const OCC::SyncFileItem &) "MAC2/template_47119_mx648Ufrj227L577oZk6/screenshots/47119-1-origin.jpg" INSTRUCTION_NEW 1 "Ligação fechada" 06-02 15:38:09:291 0x7ff1eb47b9b0 OCC::SocketApi::sendMessage: SocketApi: Sending message: "STATUS:ERROR:/Users/netadm/Documents/cubo/MAC2/template_47119_mx648Ufrj227L577oZk6/screenshots/47119-1-origin.jpg" 06-02 15:38:09:291 0x7ff1eb47b9b0 OCC::AbstractNetworkJob::slotFinished: void OCC::AbstractNetworkJob::slotFinished() 5 "Operation canceled" 06-02 15:38:09:292 0x7ff1eb47b9b0 OCC::PropagateDownloadFileQNAM::slotGetFinished: void OCC::PropagateDownloadFileQNAM::slotGetFinished() QUrl( "https://owncloud.example.br/remote.php/webdav/MAC2/template_47119_mx648Ufrj227L577oZk6/screenshots/47119-1-origin-1200.jpg" ) FINISHED WITH STATUS 5 "Operation canceled" 06-02 15:38:09:292 0x7ff1eb47b9b0 OCC::SyncJournalDb::setDownloadInfo: "DELETE FROM downloadinfo WHERE path=?1" "MAC2/template_47119_mx648Ufrj227L577oZk6/screenshots/47119-1-origin-1200.jpg" 06-02 15:38:09:292 0x7ff1eb47b9b0 OCC::SyncEngine::slotJobCompleted: void OCC::SyncEngine::slotJobCompleted(const OCC::SyncFileItem &) "MAC2/template_47119_mx648Ufrj227L577oZk6/screenshots/47119-1-origin-1200.jpg" INSTRUCTION_NEW 1 "Operation canceled" 06-02 15:38:09:297 0x7ff1eb47b9b0 OCC::SocketApi::sendMessage: SocketApi: Sending message: "STATUS:ERROR:/Users/netadm/Documents/cubo/MAC2/template_47119_mx648Ufrj227L577oZk6/screenshots/47119-1-origin-1200.jpg" 06-02 15:38:09:496 0x7ff1eb47b9b0 OCC::SyncJournalDb::walCheckpoint: void OCC::SyncJournalDb::walCheckpoint() took 0 msec 06-02 15:38:09:496 0x7ff1eb47b9b0 OCC::SyncJournalDb::commitInternal: void OCC::SyncJournalDb::commitInternal(const QString &, bool) Transaction commit "All Finished."

3) In WireShark show errors duplicate ACK. No. Time Source Destination Protocol Length Info 256220 2422.037584000 client-ip owncloud-ip-server TLSv1.2 117 Change Cipher Spec, Encrypted Handshake Message

Frame 256220: 117 bytes on wire (936 bits), 117 bytes captured (936 bits) on interface 0 Ethernet II, Src: Apple_56:9f:cf (b8:7c:12:56:9g:da), Dst: Cisco_a3:69:96 (54:75:d0:b3:59:89) Internet Protocol Version 4, Src: client-ip (client-ip), Dst: owncloud-ip-server (owncloud-ip-server) Transmission Control Protocol, Src Port: 50942 (50942), Dst Port: https (443), Seq: 581, Ack: 143, Len: 51 Secure Sockets Layer

No. Time Source Destination Protocol Length Info 256221 2422.074408000 client-ip owncloud-ip-server TCP 66 50941 > https [FIN, ACK] Seq=1024 Ack=143 Win=131168 Len=0 TSval=32227027 TSecr=7390526

Frame 256221: 66 bytes on wire (528 bits), 66 bytes captured (528 bits) on interface 0 Ethernet II, Src: Apple_56:9f:cf (b8:7c:12:56:9g:da), Dst: Cisco_a3:69:96 (54:75:d0:b3:59:89) Internet Protocol Version 4, Src: client-ip (client-ip), Dst: owncloud-ip-server (owncloud-ip-server) Transmission Control Protocol, Src Port: 50941 (50941), Dst Port: https (443), Seq: 1024, Ack: 143, Len: 0

No. Time Source Destination Protocol Length Info 256222 2422.077527000 owncloud-ip-server client-ip TCP 66 https > 50942 [ACK] Seq=143 Ack=632 Win=15744 Len=0 TSval=7390853 TSecr=32226991

Frame 256222: 66 bytes on wire (528 bits), 66 bytes captured (528 bits) on interface 0 Ethernet II, Src: Cisco_a3:69:96 (54:75:d0:b3:59:89), Dst: Apple_56:9f:cf (b8:7c:12:56:9g:da) Internet Protocol Version 4, Src: owncloud-ip-server (owncloud-ip-server), Dst: client-ip (client-ip) Transmission Control Protocol, Src Port: https (443), Dst Port: 50942 (50942), Seq: 143, Ack: 632, Len: 0

No. Time Source Destination Protocol Length Info 256223 2422.114628000 owncloud-ip-server client-ip TCP 66 https > 50941 [ACK] Seq=143 Ack=1025 Win=16896 Len=0 TSval=7390890 TSecr=32227027

Frame 256223: 66 bytes on wire (528 bits), 66 bytes captured (528 bits) on interface 0 Ethernet II, Src: Cisco_a3:69:96 (54:75:d0:b3:59:89), Dst: Apple_56:9f:cf (b8:7c:12:56:9g:da) Internet Protocol Version 4, Src: owncloud-ip-server (owncloud-ip-server), Dst: client-ip (client-ip) Transmission Control Protocol, Src Port: https (443), Dst Port: 50941 (50941), Seq: 143, Ack: 1025, Len: 0

No. Time Source Destination Protocol Length Info 256224 2422.114694000 client-ip owncloud-ip-server TCP 66 [TCP Dup ACK 256221#1] 50941 > https [ACK] Seq=1025 Ack=143 Win=131168 Len=0 TSval=32227067 TSecr=7390890

Frame 256224: 66 bytes on wire (528 bits), 66 bytes captured (528 bits) on interface 0 Ethernet II, Src: Apple_56:9f:cf (b8:7c:12:56:9g:da), Dst: Cisco_a3:69:96 (54:75:d0:b3:59:89) Internet Protocol Version 4, Src: client-ip (client-ip), Dst: owncloud-ip-server (owncloud-ip-server) Transmission Control Protocol, Src Port: 50941 (50941), Dst Port: https (443), Seq: 1025, Ack: 143, Len: 0

SOLUTION; 1) Server owncloud in file "/etc/sysctl.conf" net.core.somaxconn = 4096 net.ipv4.tcp_max_syn_backlog = 2048

net.ipv4.ip_local_port_range = 2048 64512 net.ipv4.tcp_tw_recycle = 1 net.ipv4.tcp_tw_reuse = 1 net.ipv4.tcp_fin_timeout = 10

net.core.somaxconn = 65536 net.ipv4.tcp_max_syn_backlog = 65536 net.netfilter.nf_conntrack_max = 262144

Server owncloud /etc/httpd/conf/httpd.conf KeepAlive On KeepAliveTimeout 2 MaxKeepAliveRequests 10 StartServers 100 MinSpareServers 100 MaxSpareServers 2000/ ServerLimit 6000/ MaxClients 6000

I hope it helps

[PVince81]

@nickvergessen had something similar with a specific PHP version. Then there was an update and the new version doesn't.

@nickvergessen can you post your version ?

[nickvergessen]

I had various connection issues on php 5.5.24 and 5.5.25, since the update to 5.5.26 last night everything is running smooth. Maybe that's worth a try for you as well.

[pdesanex]

@PVince81 I switched from php 5.4 to 5.6 via .htaccess -- I still get the same error. The client successfully synchronized the first 400 or so files (including many small files and a few large ones > 500MB). Now it repeatedly fails on a specific ~400MB file. After deselecting this file from synchronizing, the client downloads a few more files, then fails again.

[l0bster]

@netimpa Thanks for you suggestions.

i can confirm, tuning my apache server KeepAlive values solved my problems.

client: w8.1 oc client version 1.8.1

server: ownCloud 8.0.4 (stable) sqlite backend server local storage apache 2.2.15 release 39.el6.centos

/etc/httpd/conf/httpd.conf KeepAlive On KeepAliveTimeout 15

[pdesanex]

Hrm. It seems like I'm out of luck then ... I just received a reply from my shared hoster's customer support - the KeepAlive parameters are off-limits :-(

[guruz]

If you have a Mac or Windows machine, please give 1.8.3rc3 a try. https://owncloud.org/install/#testing-development

There has been a fix in the network stack (QtNetwork) that should fix this.

[pdesanex]

I just tested 1.8.3rc3 on xUbuntu_15.04 - it still doesn't work for me: Connection closed / Operation canceled. Please let me know how I can help to investigate this further.

[guruz]

@pdesanex That's why I meant Mac/Windows :-( Which Qt version do you use? Do you have your self-compiled Qt?

[pdesanex]

@guruz m( sorry about that. I'm not proficient with compiling things - but I could give it a try. xUbuntu_15.04 repositories provide Qt 4.8.6. and 5.4.1. Will test 1.8.3rc3 on Windows tomorrow.

[pdesanex]

@guruz a quick follow-up: 1) Confirmed: 1.8.3rc3 on Windows works fine! Not a single hiccup. 2) I built 1.8.3rc3 on Linux using Qt 5.4.1 - the problem persists.

[guruz]

@pdesanex Hey, thanks for trying out. Two questions..

1. Are you using download bandwidth limiting?
2. If you apply https://build.opensuse.org/package/view_file/windows:mingw:win32/mingw32-libqt5-qtbase/qt5-correctly-handle-connection-closed.patch?expand=1 to your Qt (might only apply when using 5.4.2 or when https://build.opensuse.org/package/view_file/windows:mingw:win32/mingw32-libqt5-qtbase/qt5-fix-upload-corruptions.patch?expand=1 is applied too) does it work then?

[pdesanex]

@guruz

1. Nope, no bandwidth limiting.
2. Sorry, I'm afraid this might be a bit beyond me :-( I will try to wrap my head around rolling my own Qt later today.

[jmeierk]

@pdsanex (and others who have their Owncloud instance running in a shared hosting environment such as allinkl.com)

Background

I had exactly the same problems: syncing many files to my clients (i.e., download) led to continuous "operation cancelled"/"connection closed" error messages (I think this started somewhere around February 2015). For the last months, it was practically impossible to sync the files because after one or two successfully synced files, the above error message kicked in. Sometimes the owncloud sync client even failed at the same files over and over again. In that way, syncing of 20GB of data would have taken months and a lot of manual interaction (hitting "retry sync" or "Pause"/"Resume" buttons over and over again). I updated to both the latest server and client version but the problem still persisted. I made this experience on two machines running Ubuntu 14.04 and one machine running OS X Lion, i.e., same errors on all of my machines.

Solution

In this thread I stumpled over some previous posts suggesting to adjust the Apache server's KeepAliveTimeout settings. Since my owncloud is running on an Allinkl server (all-inkl shared hosting), I had to contact them. Within 1-2 hours I got a reply from support and they told me that they had adjusted the KeepAliveTimeout from 2 seconds to 5 seconds (this is the Apache default anyway). After a restart of my sync client I was able to sync my 20GB WITHOUT any problem or error message. Heureka!

That is, if anyone else is experiencing this problem with Allinkl, I can strongly suggest to contact their support and ask for an adjustment of the KeepAliveTimeout but ONLY for the domain/subdomain the Owncloud is running on. They cannot change it globally on the whole server because other customers would also be affected by this change.

I'm currently running 8.0.3 server (stable) and 1.8.3 sync client (stable).

Question

Can someone with knowledge on the Apache please try to explain why an increase of the KeepAliveTimeout has such a big impact on the stability of the Owncloud-Sync? That is, why can a difference of 3 seconds have such as big impact (again, I'm not a pro on web server technology). In any case, the docs [1] say:

KeepAliveTimeout: The number of seconds Apache will wait for a subsequent request before closing the connection. Once a request has been received, the timeout value specified by the Timeout directive applies. Setting KeepAliveTimeout to a high value may cause performance problems in heavily loaded servers. The higher the timeout, the more server processes will be kept occupied waiting on connections with idle clients.

Perhaps my shared hoster (allinkl) reduced the timeout from 5 (default) to 2 seconds on that server to increase overall performance but if that was the original intention, they actually achieved the direct opposite ...

[1] http://httpd.apache.org/docs/2.2/mod/core.html#keepalivetimeout

[pdesanex]

@jmeierk I'm also with allinkl shared hosting - unfortunately, I was denied a change of the KeepAliveTimeout parameters. However, my sync issues were completely resolved with client 1.8.3 on Windows and Mac, without needing any changes on the server side whatsoever (unfortunately, the issues persist with the current Linux version of the client - see above). AFAICT, incrementing the KeepAliveTimeout fixes/obscures an issue on the client side?

[SHelfinger]

It still exists in Nightly Build

Tested EXE: ownCloud-2.0.0.5285-nightly20150718-setup.exe

On 1nd Server (Togo -> Germany) Works On 2nd Server (Localnet, LAN Connection, same Server configuration like Togo Server) connection abort.

[rpedrica]

I've been successfully syncing through the last 2 years on the same OC server which has gone through successive upgrades. Now at 8.0.4 and with client 1.8.4. I have about 8 syncs on my client configured and no issues. There are also other users on the same server with no issues.

Yesterday, I added a new folder sync to my client ( with just under 25k files ) and immediately it started throwing errors on some files. There was no pattern. Some were small files and others large files. Also, some file synced without issue.

Screen-shot of the client:

The client log shows the following:

07-20 01:06:06:596 0x1ca06a0 !!! OCC::PUTFileJob created for "https://xxx/" + "XStore/clients/3dxact/3d/3dXact/classes/commons/protection/PasswordProtection.inc.php" 07-20 01:06:06:596 0x1ca06a0 virtual void OCC::PropagateUploadFileQNAM::abort() OCC::PUTFileJob(0x2e314e0) "3dxact/3d/3dXact/classes/commons/protection/PasswordProtection.inc.php" 07-20 01:06:06:596 0x1ca06a0 void OCC::AbstractNetworkJob::slotFinished() 5 "Operation canceled" 07-20 01:06:06:597 0x1ca06a0 void OCC::PropagateUploadFileQNAM::slotPutFinished() QUrl( "https://xxx/remote.php/webdav/XStore/clients/3dxact/3d/3dXact/classes/commons/protection/PasswordProtection.inc.php" ) FINISHED WITH STATUS 5 "Operation canceled" QVariant(, ) QVariant(, ) 07-20 01:06:06:597 0x1ca06a0 "" 07-20 01:06:06:597 0x1ca06a0 void OCC::SyncEngine::slotJobCompleted(const OCC::SyncFileItem&) "3dxact/3d/3dXact/classes/commons/protection/PasswordProtection.inc.php" INSTRUCTION_NEW 1 "Operation canceled" 07-20 01:06:06:598 0x1ca06a0 QNetworkReplyImplPrivate::error: Internal problem, this method must only be called once. 07-20 01:06:06:598 0x1ca06a0 void OCC::AbstractNetworkJob::slotFinished() 5 "Operation canceled" 07-20 01:06:06:598 0x1ca06a0 void OCC::PropagateUploadFileQNAM::slotPutFinished() QUrl( "https://xxx/remote.php/webdav/XStore/clients/3dxact/3d/3dXact/classes/commons/protection/PasswordProtection.inc.php" ) FINISHED WITH STATUS 5 "Operation canceled" QVariant(, ) QVariant(, ) 07-20 01:06:06:599 0x1ca06a0 virtual void OCC::PropagateRemoteMkdir::start() "3dxact/3d/3dXact/classes/components/Form" 07-20 01:06:06:599 0x1ca06a0 !!! OCC::MkColJob created for "https://xxx/" + "XStore/clients/3dxact/3d/3dXact/classes/components/Form" 07-20 01:06:06:603 0x1ca06a0 void OCC::AbstractNetworkJob::slotFinished() 5 "Operation canceled" 07-20 01:06:06:604 0x1ca06a0 void OCC::PropagateRemoteMkdir::slotMkcolJobFinished() QUrl( "https://xxx/remote.php/webdav/XStore/clients/3dxact/3d/3dXact/classes/components/Form" ) FINISHED WITH STATUS 5 "Operation canceled" 07-20 01:06:06:604 0x1ca06a0 void OCC::SyncEngine::slotJobCompleted(const OCC::SyncFileItem&) "3dxact/3d/3dXact/classes/components/Form" INSTRUCTION_NEW 1 "Operation canceled"

Note while I was having issues with this sync, all my other old syncs were working fine.

I made the change to Apache:

KeepAlive on KeepAliveTimeout 15

And the issue is completely gone now! Perhaps this should be added as a prerequisite for apache config?

[jzee]

same problem with 1.8.4-2531 on OSX and OC 7.0.4 apache 2.4.10 was set tp KeepAliveTimeout 5, will now increase to 15, see what it gives

[guruz]

@rpedrica What was your settings before? @jzee Did you have any success?

[rpedrica]

@guruz I was using Centos 6.5 std install with base apache settings. ( KeepAlive is off ). This is in fact something I normally switch on but I must have missed it this time ... I have 15s as my setting now.

[jzee]

I will try with a large number of files this evening. just as a heads up to you, the performance tuning recommendations for the OC-server are a bit contradictory to the findings here and what's available elsewhere on the web:

For instance

a) https://doc.owncloud.org/server/8.0/admin_manual/configuration_server/performance_tuning.html says 2s for keep alive as a sensible setting

while other pages like this (random google result) says 100s(!) b) https://www.feedthebot.com/pagespeed/keep-alive.html

both without going into the details of why the respective setting would be good.

Anyway, I'll follow up with results as I have them

[guruz]

@SHelfinger Could you also tell us your Apache/nginx keep alive settings (on/off, requests count, timeout count) that made it fail?

[rpedrica]

@guruz The KeepAlive option enables persistent network connections in Apache. IE you get multiple requests over the same connection instead of starting a new connection for each request.

As you can imagine, this is far more efficient and can lead to significant performance improvements. The longer your time-out, the more requests per connection and the better the performance. There is a trade-off because it costs Apache to keep all this connection info in memory. The less load on your server i.t.o. connections, the longer you can make this - 60 - 120secs is common. 15sec should be very little issue for most ( even heavily loaded ) servers as long as you have enough memory - this needs to be monitored.

[SHelfinger]

@guruz Apache2.conf Timeout 800 KeepAliveTimeout 100 FCGI FcgidIdleTimeout 9000000 FcgidIOTimeout 9600 IdleTimeout 9000000 FcgidBusyTimeout 90000000000 BusyTimeout 90000000000 IPCConnectTimeout 9600 IPCCommTimeout 9600

I left my Laptop running for about 36 hours. At the end he upload everything. With multiple errors, but another files he is synchronizing.

So now I don't have a problem. It was just at the start. After I upload all the files, he stopped giving me Errors.

[jzee]

Uploaded roughly 300 files, between 30kB through 500MB in size.

Out of 300, Got 3x: Error downloading, expected filesize X found 0 Got 1x: Error downloading, Could not rename part file assembled from chunks Got 2x: Operation cancelled

so roughly 2% error. The server was doing nothing else at the time (except the cron jobs every 15m). Upload came from a Mac OSX10.10 through OC over to Dropbox

[jzee]

... and just after I posted this, I got the classic Timeout + 2x cancelled files afterwards. The timed out file was a ~100MB PDF, if that is of any help as an indicator.

All this using 15s as a timeout. I will increase to 100s as suggested elsewhere and retry tomorrow.

[guruz]

I was able to reproduce this now by having an Apache with a low Keep-Alive time of 250 msec. Then I can see that requests get "stuck" and our time out kills them after some time ("Operation Canceled").

OS X users: We have a nightly build to test. If everyone can confirm that this fixes the issue, we will move the patch also into the Windows Qt-build. http://download.owncloud.com/desktop/daily/ownCloud-1.8.4.2564-nightly20150724.pkg

(We don't know how the OpenSSL update in 1.8.4 could trigger this bug more likely..)