Closed MarvinFS closed 9 years ago
@bantu look, another case of Expected SSH_FXP_STATUS
@MarvinFS it looks like you're having timeout issues: Connection closed prematurely
@PVince81 yeah but it's a corporate local network - no timeouts here. on my side it looks like it filled the progress bar (time is different, depends on speed and file size) and then it's just hang for a very long time with a full bar doing nothing... then in just stops the upload progress and you left with the partial file on the server... nothing really useful in sftp server's logs (i'm in debug mode)
also the phpseclib's NET_SFTP_RESUME seems useful in case the connection drops and there are partial file on server
@MarvinFS It sounds like you're able to consistently reproduce the issue (possibly due to the large files), which is good. Please replace the contents of the owncloud/3rdparty/phpseclib/phpseclib
folder of https://download.owncloud.org/community/daily/owncloud-daily-master.tar.bz2 with the contents of the phpseclib-master
folder of https://github.com/phpseclib/phpseclib/archive/master.zip and see whether the issue persists. Thanks.
Possibly related to https://github.com/phpseclib/phpseclib/issues/319#issuecomment-45625669
done! right after apache restart a lot of garbage in logs haven't tried uploading yet
cat ./owncloud.log
{"reqId":"53c500820e612","app":"PHP","message":"Invalid size at \/var\/www\/owncloud\/3rdparty\/phpseclib\/phpseclib\/phpseclib\/Net\/SSH2.php#2704","level":3,"time":"2014-07-15T16:20:50+06:00","method":"GET","url":"\/owncloud\/index.php\/apps\/files\/ajax\/list.php?dir=%2F&sort=name&sortdirection=asc"}
{"reqId":"53c500820e612","app":"PHP","message":"Connection closed by server at \/var\/www\/owncloud\/3rdparty\/phpseclib\/phpseclib\/phpseclib\/Net\/SSH2.php#1028","level":3,"time":"2014-07-15T16:20:50+06:00","method":"GET","url":"\/owncloud\/index.php\/apps\/files\/ajax\/list.php?dir=%2F&sort=name&sortdirection=asc"}
{"reqId":"53c500820f24b","app":"PHP","message":"Invalid size at \/var\/www\/owncloud\/3rdparty\/phpseclib\/phpseclib\/phpseclib\/Net\/SSH2.php#2704","level":3,"time":"2014-07-15T16:20:50+06:00","method":"GET","url":"\/owncloud\/index.php\/apps\/files\/ajax\/scan.php?force=false&dir=&requesttoken=f06feb01eac59e487e78"}
{"reqId":"53c500820f24b","app":"PHP","message":"Connection closed by server at \/var\/www\/owncloud\/3rdparty\/phpseclib\/phpseclib\/phpseclib\/Net\/SSH2.php#1028","level":3,"time":"2014-07-15T16:20:50+06:00","method":"GET","url":"\/owncloud\/index.php\/apps\/files\/ajax\/scan.php?force=false&dir=&requesttoken=f06feb01eac59e487e78"}
{"reqId":"53c50083e1e5e","app":"PHP","message":"Invalid size at \/var\/www\/owncloud\/3rdparty\/phpseclib\/phpseclib\/phpseclib\/Net\/SSH2.php#2704","level":3,"time":"2014-07-15T16:20:51+06:00","method":"GET","url":"\/owncloud\/index.php\/apps\/files\/ajax\/list.php?dir=%2FSFTP&sort=name&sortdirection=asc"}
{"reqId":"53c50083e1e5e","app":"PHP","message":"Connection closed by server at \/var\/www\/owncloud\/3rdparty\/phpseclib\/phpseclib\/phpseclib\/Net\/SSH2.php#1028","level":3,"time":"2014-07-15T16:20:51+06:00","method":"GET","url":"\/owncloud\/index.php\/apps\/files\/ajax\/list.php?dir=%2FSFTP&sort=name&sortdirection=asc"}
{"reqId":"53c50083e1e5e","app":"PHP","message":"Invalid size at \/var\/www\/owncloud\/3rdparty\/phpseclib\/phpseclib\/phpseclib\/Net\/SSH2.php#2704","level":3,"time":"2014-07-15T16:20:51+06:00","method":"GET","url":"\/owncloud\/index.php\/apps\/files\/ajax\/list.php?dir=%2FSFTP&sort=name&sortdirection=asc"}
{"reqId":"53c50083e1e5e","app":"PHP","message":"Connection closed by server at \/var\/www\/owncloud\/3rdparty\/phpseclib\/phpseclib\/phpseclib\/Net\/SSH2.php#2959","level":3,"time":"2014-07-15T16:20:51+06:00","method":"GET","url":"\/owncloud\/index.php\/apps\/files\/ajax\/list.php?dir=%2FSFTP&sort=name&sortdirection=asc"}
{"reqId":"53c50083e1e5e","app":"PHP","message":"Expected SSH_FXP_ATTRS or SSH_FXP_STATUS at \/var\/www\/owncloud\/3rdparty\/phpseclib\/phpseclib\/phpseclib\/Net\/SFTP.php#1273","level":3,"time":"2014-07-15T16:20:51+06:00","method":"GET","url":"\/owncloud\/index.php\/apps\/files\/ajax\/list.php?dir=%2FSFTP&sort=name&sortdirection=asc"}
{"reqId":"53c500843aec3","app":"PHP","message":"Invalid size at \/var\/www\/owncloud\/3rdparty\/phpseclib\/phpseclib\/phpseclib\/Net\/SSH2.php#2704","level":3,"time":"2014-07-15T16:20:52+06:00","method":"GET","url":"\/owncloud\/index.php\/apps\/files\/ajax\/getstoragestats.php?dir=%2FSFTP"}
{"reqId":"53c500843aec3","app":"PHP","message":"Connection closed by server at \/var\/www\/owncloud\/3rdparty\/phpseclib\/phpseclib\/phpseclib\/Net\/SSH2.php#1028","level":3,"time":"2014-07-15T16:20:52+06:00","method":"GET","url":"\/owncloud\/index.php\/apps\/files\/ajax\/getstoragestats.php?dir=%2FSFTP"}
{"reqId":"53c500843aec3","app":"PHP","message":"Invalid size at \/var\/www\/owncloud\/3rdparty\/phpseclib\/phpseclib\/phpseclib\/Net\/SSH2.php#2704","level":3,"time":"2014-07-15T16:20:52+06:00","method":"GET","url":"\/owncloud\/index.php\/apps\/files\/ajax\/getstoragestats.php?dir=%2FSFTP"}
{"reqId":"53c500843aec3","app":"PHP","message":"Connection closed by server at \/var\/www\/owncloud\/3rdparty\/phpseclib\/phpseclib\/phpseclib\/Net\/SSH2.php#2959","level":3,"time":"2014-07-15T16:20:52+06:00","method":"GET","url":"\/owncloud\/index.php\/apps\/files\/ajax\/getstoragestats.php?dir=%2FSFTP"}
{"reqId":"53c500843aec3","app":"PHP","message":"Expected SSH_FXP_ATTRS or SSH_FXP_STATUS at \/var\/www\/owncloud\/3rdparty\/phpseclib\/phpseclib\/phpseclib\/Net\/SFTP.php#1273","level":3,"time":"2014-07-15T16:20:52+06:00","method":"GET","url":"\/owncloud\/index.php\/apps\/files\/ajax\/getstoragestats.php?dir=%2FSFTP"}
{"reqId":"53c50084e84c5","app":"PHP","message":"Invalid size at \/var\/www\/owncloud\/3rdparty\/phpseclib\/phpseclib\/phpseclib\/Net\/SSH2.php#2704","level":3,"time":"2014-07-15T16:20:52+06:00","method":"GET","url":"\/owncloud\/index.php\/apps\/files\/ajax\/list.php?dir=%2FSFTP%2Focloud&sort=name&sortdirection=asc"}
{"reqId":"53c50084e84c5","app":"PHP","message":"Connection closed by server at \/var\/www\/owncloud\/3rdparty\/phpseclib\/phpseclib\/phpseclib\/Net\/SSH2.php#1028","level":3,"time":"2014-07-15T16:20:52+06:00","method":"GET","url":"\/owncloud\/index.php\/apps\/files\/ajax\/list.php?dir=%2FSFTP%2Focloud&sort=name&sortdirection=asc"}
{"reqId":"53c50084e84c5","app":"PHP","message":"Invalid size at \/var\/www\/owncloud\/3rdparty\/phpseclib\/phpseclib\/phpseclib\/Net\/SSH2.php#2704","level":3,"time":"2014-07-15T16:20:52+06:00","method":"GET","url":"\/owncloud\/index.php\/apps\/files\/ajax\/list.php?dir=%2FSFTP%2Focloud&sort=name&sortdirection=asc"}
{"reqId":"53c50084e84c5","app":"PHP","message":"Connection closed by server at \/var\/www\/owncloud\/3rdparty\/phpseclib\/phpseclib\/phpseclib\/Net\/SSH2.php#2959","level":3,"time":"2014-07-15T16:20:52+06:00","method":"GET","url":"\/owncloud\/index.php\/apps\/files\/ajax\/list.php?dir=%2FSFTP%2Focloud&sort=name&sortdirection=asc"}
{"reqId":"53c50084e84c5","app":"PHP","message":"Expected SSH_FXP_ATTRS or SSH_FXP_STATUS at \/var\/www\/owncloud\/3rdparty\/phpseclib\/phpseclib\/phpseclib\/Net\/SFTP.php#1273","level":3,"time":"2014-07-15T16:20:52+06:00","method":"GET","url":"\/owncloud\/index.php\/apps\/files\/ajax\/list.php?dir=%2FSFTP%2Focloud&sort=name&sortdirection=asc"}
{"reqId":"53c500854f7cd","app":"PHP","message":"Invalid size at \/var\/www\/owncloud\/3rdparty\/phpseclib\/phpseclib\/phpseclib\/Net\/SSH2.php#2704","level":3,"time":"2014-07-15T16:20:53+06:00","method":"GET","url":"\/owncloud\/index.php\/apps\/files\/ajax\/getstoragestats.php?dir=%2FSFTP%2Focloud"}
{"reqId":"53c500854f7cd","app":"PHP","message":"Connection closed by server at \/var\/www\/owncloud\/3rdparty\/phpseclib\/phpseclib\/phpseclib\/Net\/SSH2.php#1028","level":3,"time":"2014-07-15T16:20:53+06:00","method":"GET","url":"\/owncloud\/index.php\/apps\/files\/ajax\/getstoragestats.php?dir=%2FSFTP%2Focloud"}
{"reqId":"53c500854f7cd","app":"PHP","message":"Invalid size at \/var\/www\/owncloud\/3rdparty\/phpseclib\/phpseclib\/phpseclib\/Net\/SSH2.php#2704","level":3,"time":"2014-07-15T16:20:53+06:00","method":"GET","url":"\/owncloud\/index.php\/apps\/files\/ajax\/getstoragestats.php?dir=%2FSFTP%2Focloud"}
{"reqId":"53c500854f7cd","app":"PHP","message":"Connection closed by server at \/var\/www\/owncloud\/3rdparty\/phpseclib\/phpseclib\/phpseclib\/Net\/SSH2.php#2959","level":3,"time":"2014-07-15T16:20:53+06:00","method":"GET","url":"\/owncloud\/index.php\/apps\/files\/ajax\/getstoragestats.php?dir=%2FSFTP%2Focloud"}
{"reqId":"53c500854f7cd","app":"PHP","message":"Expected SSH_FXP_ATTRS or SSH_FXP_STATUS at \/var\/www\/owncloud\/3rdparty\/phpseclib\/phpseclib\/phpseclib\/Net\/SFTP.php#1273","level":3,"time":"2014-07-15T16:20:53+06:00","method":"GET","url":"\/owncloud\/index.php\/apps\/files\/ajax\/getstoragestats.php?dir=%2FSFTP%2Focloud"}
{"reqId":"53c500867fa37","app":"PHP","message":"Invalid size at \/var\/www\/owncloud\/3rdparty\/phpseclib\/phpseclib\/phpseclib\/Net\/SSH2.php#2704","level":3,"time":"2014-07-15T16:20:54+06:00","method":"GET","url":"\/owncloud\/index.php\/apps\/files\/ajax\/list.php?dir=%2FSFTP%2Focloud%2Ftest4&sort=name&sortdirection=asc"}
{"reqId":"53c500867fa37","app":"PHP","message":"Connection closed by server at \/var\/www\/owncloud\/3rdparty\/phpseclib\/phpseclib\/phpseclib\/Net\/SSH2.php#1028","level":3,"time":"2014-07-15T16:20:54+06:00","method":"GET","url":"\/owncloud\/index.php\/apps\/files\/ajax\/list.php?dir=%2FSFTP%2Focloud%2Ftest4&sort=name&sortdirection=asc"}
{"reqId":"53c500867fa37","app":"PHP","message":"Invalid size at \/var\/www\/owncloud\/3rdparty\/phpseclib\/phpseclib\/phpseclib\/Net\/SSH2.php#2704","level":3,"time":"2014-07-15T16:20:54+06:00","method":"GET","url":"\/owncloud\/index.php\/apps\/files\/ajax\/list.php?dir=%2FSFTP%2Focloud%2Ftest4&sort=name&sortdirection=asc"}
{"reqId":"53c500867fa37","app":"PHP","message":"Connection closed by server at \/var\/www\/owncloud\/3rdparty\/phpseclib\/phpseclib\/phpseclib\/Net\/SSH2.php#2959","level":3,"time":"2014-07-15T16:20:54+06:00","method":"GET","url":"\/owncloud\/index.php\/apps\/files\/ajax\/list.php?dir=%2FSFTP%2Focloud%2Ftest4&sort=name&sortdirection=asc"}
{"reqId":"53c500867fa37","app":"PHP","message":"Expected SSH_FXP_ATTRS or SSH_FXP_STATUS at \/var\/www\/owncloud\/3rdparty\/phpseclib\/phpseclib\/phpseclib\/Net\/SFTP.php#1273","level":3,"time":"2014-07-15T16:20:54+06:00","method":"GET","url":"\/owncloud\/index.php\/apps\/files\/ajax\/list.php?dir=%2FSFTP%2Focloud%2Ftest4&sort=name&sortdirection=asc"}
{"reqId":"53c50086d1a2e","app":"PHP","message":"Invalid size at \/var\/www\/owncloud\/3rdparty\/phpseclib\/phpseclib\/phpseclib\/Net\/SSH2.php#2704","level":3,"time":"2014-07-15T16:20:54+06:00","method":"GET","url":"\/owncloud\/index.php\/apps\/files\/ajax\/getstoragestats.php?dir=%2FSFTP%2Focloud%2Ftest4"}
{"reqId":"53c50086d1a2e","app":"PHP","message":"Connection closed by server at \/var\/www\/owncloud\/3rdparty\/phpseclib\/phpseclib\/phpseclib\/Net\/SSH2.php#1028","level":3,"time":"2014-07-15T16:20:54+06:00","method":"GET","url":"\/owncloud\/index.php\/apps\/files\/ajax\/getstoragestats.php?dir=%2FSFTP%2Focloud%2Ftest4"}
{"reqId":"53c50086d1a2e","app":"PHP","message":"Invalid size at \/var\/www\/owncloud\/3rdparty\/phpseclib\/phpseclib\/phpseclib\/Net\/SSH2.php#2704","level":3,"time":"2014-07-15T16:20:54+06:00","method":"GET","url":"\/owncloud\/index.php\/apps\/files\/ajax\/getstoragestats.php?dir=%2FSFTP%2Focloud%2Ftest4"}
{"reqId":"53c50086d1a2e","app":"PHP","message":"Connection closed by server at \/var\/www\/owncloud\/3rdparty\/phpseclib\/phpseclib\/phpseclib\/Net\/SSH2.php#2959","level":3,"time":"2014-07-15T16:20:54+06:00","method":"GET","url":"\/owncloud\/index.php\/apps\/files\/ajax\/getstoragestats.php?dir=%2FSFTP%2Focloud%2Ftest4"}
{"reqId":"53c50086d1a2e","app":"PHP","message":"Expected SSH_FXP_ATTRS or SSH_FXP_STATUS at \/var\/www\/owncloud\/3rdparty\/phpseclib\/phpseclib\/phpseclib\/Net\/SFTP.php#1273","level":3,"time":"2014-07-15T16:20:54+06:00","method":"GET","url":"\/owncloud\/index.php\/apps\/files\/ajax\/getstoragestats.php?dir=%2FSFTP%2Focloud%2Ftest4"}
a lot if bad packet length messages like every second in sshd debug and then reconnects
Connection from 79.172.0.10 port 32169
debug1: Client protocol version 2.0; client software version phpseclib_0.3 (gmp)
debug1: no match: phpseclib_0.3 (gmp)
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_6.0p1 Debian-4+deb7u1
debug1: permanently_set_uid: 102/65534 [preauth]
debug1: list_hostkey_types: ssh-rsa,ssh-dss,ecdsa-sha2-nistp256 [preauth]
debug1: SSH2_MSG_KEXINIT sent [preauth]
Bad packet length 3116788250. [preauth]
Disconnecting: Packet corrupt [preauth]
debug1: do_cleanup [preauth]
debug1: monitor_read_log: child log fd closed
debug1: do_cleanup
debug1: Killing privsep child 9387
debug1: Forked child 9388.
Set /proc/self/oom_score_adj to -1000
debug1: rexec start in 4 out 4 newsock 4 pipe 6 sock 7
debug1: inetd sockets after dupping: 3, 3
Connection from 79.172.0.10 port 48170
@MarvinFS Okay thanks. Will look into that.
the last progress is: on my side the upload bar is full and stays the same for about 20 min (the progress bar filled like may be in 1 min or less - i have a 100mbit here) i was trying to upload a 900MB zip file. nothing new in the logs neither on sshd server nor on owncloud for that waiting time... i quit waiting... the last logs in owncloud were
{"reqId":"53c5017c41946","app":"PHP","message":"Connection closed by server at \/var\/www\/owncloud\/3rdparty\/phpseclib\/phpseclib\/phpseclib\/Net\/SSH2.php#2959","level":3,"time":"2014-07-15T16:25:00+06:00","method":"POST","url":"\/owncloud\/index.php\/apps\/files\/ajax\/upload.php"}
{"reqId":"53c5017c41946","app":"PHP","message":"Expected SSH_FXP_ATTRS or SSH_FXP_STATUS at \/var\/www\/owncloud\/3rdparty\/phpseclib\/phpseclib\/phpseclib\/Net\/SFTP.php#1273","level":3,"time":"2014-07-15T16:25:00+06:00","method":"POST","url":"\/owncloud\/index.php\/apps\/files\/ajax\/upload.php"}
{"reqId":"53c5017c41946","app":"PHP","message":"Invalid size at \/var\/www\/owncloud\/3rdparty\/phpseclib\/phpseclib\/phpseclib\/Net\/SSH2.php#2704","level":3,"time":"2014-07-15T16:25:00+06:00","method":"POST","url":"\/owncloud\/index.php\/apps\/files\/ajax\/upload.php"}
{"reqId":"53c5017c41946","app":"PHP","message":"Connection closed by server at \/var\/www\/owncloud\/3rdparty\/phpseclib\/phpseclib\/phpseclib\/Net\/SSH2.php#1028","level":3,"time":"2014-07-15T16:25:00+06:00","method":"POST","url":"\/owncloud\/index.php\/apps\/files\/ajax\/upload.php"}
{"reqId":"53c5017c41946","app":"PHP","message":"Invalid size at \/var\/www\/owncloud\/3rdparty\/phpseclib\/phpseclib\/phpseclib\/Net\/SSH2.php#2704","level":3,"time":"2014-07-15T16:25:00+06:00","method":"POST","url":"\/owncloud\/index.php\/apps\/files\/ajax\/upload.php"}
{"reqId":"53c5017c41946","app":"PHP","message":"Connection closed by server at \/var\/www\/owncloud\/3rdparty\/phpseclib\/phpseclib\/phpseclib\/Net\/SSH2.php#2959","level":3,"time":"2014-07-15T16:25:00+06:00","method":"POST","url":"\/owncloud\/index.php\/apps\/files\/ajax\/upload.php"}
{"reqId":"53c5017c41946","app":"PHP","message":"Expected SSH_FXP_ATTRS or SSH_FXP_STATUS at \/var\/www\/owncloud\/3rdparty\/phpseclib\/phpseclib\/phpseclib\/Net\/SFTP.php#1273","level":3,"time":"2014-07-15T16:25:00+06:00","method":"POST","url":"\/owncloud\/index.php\/apps\/files\/ajax\/upload.php"}
{"reqId":"53c5017c41946","app":"PHP","message":"Invalid size at \/var\/www\/owncloud\/3rdparty\/phpseclib\/phpseclib\/phpseclib\/Net\/SSH2.php#2704","level":3,"time":"2014-07-15T16:25:00+06:00","method":"POST","url":"\/owncloud\/index.php\/apps\/files\/ajax\/upload.php"}
{"reqId":"53c5017c41946","app":"PHP","message":"Connection closed by server at \/var\/www\/owncloud\/3rdparty\/phpseclib\/phpseclib\/phpseclib\/Net\/SSH2.php#2959","level":3,"time":"2014-07-15T16:25:00+06:00","method":"POST","url":"\/owncloud\/index.php\/apps\/files\/ajax\/upload.php"}
{"reqId":"53c5017c41946","app":"PHP","message":"Expected SSH_FXP_ATTRS or SSH_FXP_STATUS at \/var\/www\/owncloud\/3rdparty\/phpseclib\/phpseclib\/phpseclib\/Net\/SFTP.php#1273","level":3,"time":"2014-07-15T16:25:00+06:00","method":"POST","url":"\/owncloud\/index.php\/apps\/files\/ajax\/upload.php"}
@MarvinFS A patch has been accepted into phpseclib potentially fixing the latest issues you were having. Please retry with the latest phpseclib (same instructions as in https://github.com/owncloud/core/issues/9632#issuecomment-49013207). Thanks.
@bantu Dear Andreas, thanks for the fixes, sorry for the delay - was busy! errors changed in logs a bit, but still fails with the same symptoms: i was uploading a 1GB file it filled the progress bar in seconds, after that was just hanging, doing nothing no errors on both sides... for about 35 mins (again 100mb\sec channel) then failed, leaving partially uploaded file (see below the log) also i've measured the bandwidth usage on a server during upload - it was about 25-30KB\sec! which is really ODD! caz if i use any other method of file cope yo SFTP the speed is 6-9MB\sec in the same environment!
the logs from the server start till the end of upload
{"reqId":"53cdcb34f1ac7","app":"core","message":"OC_Image::loadFromFileHandle(): Trying","level":0,"time":"2014-07-22T08:23:48+06:00","method":"GET","url":"\/owncloud\/index.php\/core\/preview.png?file=%2FSFTP%2Focloud%2Fpustota.jpg&c=53c4b5fc45d53&x=36&y=36&forceIcon=0"}
{"reqId":"53cdcb34f1ac7","app":"core","message":"OC_Image->fixOrientation() No readable file path set.","level":0,"time":"2014-07-22T08:23:48+06:00","method":"GET","url":"\/owncloud\/index.php\/core\/preview.png?file=%2FSFTP%2Focloud%2Fpustota.jpg&c=53c4b5fc45d53&x=36&y=36&forceIcon=0"}
{"reqId":"53cdcb34f1ac7","app":"core","message":"OC_Image->fixOrientation() Orientation: -1","level":0,"time":"2014-07-22T08:23:48+06:00","method":"GET","url":"\/owncloud\/index.php\/core\/preview.png?file=%2FSFTP%2Focloud%2Fpustota.jpg&c=53c4b5fc45d53&x=36&y=36&forceIcon=0"}
{"reqId":"53cdcb385209f","app":"OC\\Files\\Cache\\Scanner","message":"!!! No reuse of etag for 'ocloud\/test4' !!! \ncache: Array\n(\n [fileid] => 27\n [storage] => sftp::ocloud@marvinfs.convex.ru\/\/\n [path] => ocloud\/test4\n [parent] => 17\n [name] => test4\n [mimetype] => httpd\/unix-directory\n [mimepart] => httpd\n [size] => 1636247838\n [mtime] => 1405404708\n [storage_mtime] => 1405404708\n [encrypted] => \n [unencrypted_size] => 0\n [etag] => 53c4cc49abc0e\n [permissions] => 31\n)\n \ndata: Array\n(\n [mimetype] => httpd\/unix-directory\n [mtime] => 1405419982\n [size] => -1\n [etag] => 53cdcb3851d91\n [storage_mtime] => 1405419982\n [permissions] => 31\n [parent] => 17\n)\n","level":0,"time":"2014-07-22T08:23:52+06:00","method":"GET","url":"\/owncloud\/index.php\/apps\/files\/ajax\/list.php?dir=%2FSFTP%2Focloud%2Ftest4&sort=name&sortdirection=asc"}
{"reqId":"53cdcf881c7d8","app":"OC\\Files\\Cache\\Scanner","message":"!!! Path 'thumbnails\/655' is not readable !!!","level":0,"time":"2014-07-22T08:42:16+06:00","method":"GET","url":"\/owncloud\/index.php\/apps\/files_versions\/preview?file=%2FSFTP%2Focloud%2Ftest4%2FDRP13-R399-DVD-Final.iso&version=1405406286"}
{"reqId":"53cdcf881c7d8","app":"core","message":"Generating preview for \"\/SFTP\/ocloud\/test4\/DRP13-R399-DVD-Final.iso.v1405406286\" with \"OC\\Preview\\Unknown\"","level":0,"time":"2014-07-22T08:42:16+06:00","method":"GET","url":"\/owncloud\/index.php\/apps\/files_versions\/preview?file=%2FSFTP%2Focloud%2Ftest4%2FDRP13-R399-DVD-Final.iso&version=1405406286"}
{"reqId":"53cdcf881c7d8","app":"core","message":"OC_Image->loadFromBase64, couldn't load","level":0,"time":"2014-07-22T08:42:16+06:00","method":"GET","url":"\/owncloud\/index.php\/apps\/files_versions\/preview?file=%2FSFTP%2Focloud%2Ftest4%2FDRP13-R399-DVD-Final.iso&version=1405406286"}
{"reqId":"53cdcf881c7d8","app":"core","message":"OC_Image->fixOrientation() Orientation: -1","level":0,"time":"2014-07-22T08:42:16+06:00","method":"GET","url":"\/owncloud\/index.php\/apps\/files_versions\/preview?file=%2FSFTP%2Focloud%2Ftest4%2FDRP13-R399-DVD-Final.iso&version=1405406286"}
{"reqId":"53cdd1f40687c","app":"PHP","message":"Connection closed prematurely at \/var\/www\/owncloud\/3rdparty\/phpseclib\/phpseclib\/phpseclib\/Net\/SSH2.php#3127","level":3,"time":"2014-07-22T08:52:36+06:00","method":"POST","url":"\/owncloud\/index.php\/apps\/files\/ajax\/upload.php"}
{"reqId":"53cdd1f40687c","app":"PHP","message":"Undefined variable: fp at \/var\/www\/owncloud\/3rdparty\/phpseclib\/phpseclib\/phpseclib\/Net\/SFTP.php#1874","level":3,"time":"2014-07-22T08:52:36+06:00","method":"POST","url":"\/owncloud\/index.php\/apps\/files\/ajax\/upload.php"}
{"reqId":"53cdd1f40687c","app":"PHP","message":"fclose() expects parameter 1 to be resource, null given at \/var\/www\/owncloud\/3rdparty\/phpseclib\/phpseclib\/phpseclib\/Net\/SFTP.php#1874","level":3,"time":"2014-07-22T08:52:36+06:00","method":"POST","url":"\/owncloud\/index.php\/apps\/files\/ajax\/upload.php"}
{"reqId":"53cdd1f40687c","app":"OC\\Files\\Cache\\Scanner","message":"!!! Path 'ocloud\/test4\/linuxmint-15-cinnamon-dvd-32bit.iso' is not readable !!!","level":0,"time":"2014-07-22T08:52:36+06:00","method":"POST","url":"\/owncloud\/index.php\/apps\/files\/ajax\/upload.php"}
{"reqId":"53cdd1f40687c","app":"OC\\Files\\Cache\\Scanner","message":"!!! No reuse of etag for 'ocloud\/test4' !!! \ncache: Array\n(\n [fileid] => 27\n [storage] => sftp::ocloud@marvinfs.convex.ru\/\/\n [path] => ocloud\/test4\n [parent] => 17\n [name] => test4\n [mimetype] => httpd\/unix-directory\n [mimepart] => httpd\n [size] => 2455009537\n [mtime] => -1\n [storage_mtime] => -1\n [encrypted] => \n [unencrypted_size] => 0\n [etag] => 53cdd1f46e605\n [permissions] => 31\n)\n \ndata: Array\n(\n [mimetype] => httpd\/unix-directory\n [mtime] => 1405995992\n [size] => -1\n [etag] => 53cdd1f4aa232\n [storage_mtime] => 1405995992\n [permissions] => 31\n [parent] => 17\n)\n","level":0,"time":"2014-07-22T08:52:36+06:00","method":"POST","url":"\/owncloud\/index.php\/apps\/files\/ajax\/upload.php"}
the other interesting issue that the partial fines uploaded to the STFP server are the very same size... 818761699, of cause the source files are different in sizes! (other partial files in that listing were uploaded previously with different settings, that's why the size differs a little bit i think)
root@marvinfs:/opt/ocloud/test4# ls -la
total 3197072
drwxr-xr-x 2 ocloud ocloud 4096 Jul 22 08:26 .
drwxrwxrwx 8 root root 4096 Jul 15 11:04 ..
-rw-r--r-- 1 ocloud ocloud 817790918 Jul 15 12:02 Delo.Rumjanceva.1955.DVDRip_[Youtracker]_by_AVP Studio.avi
-rw-r--r-- 1 ocloud ocloud 818376675 Jul 15 12:38 DRP13-R399-DVD-Final.iso
-rw-r--r-- 1 ocloud ocloud 11574 Jul 15 11:04 gover_server.ovf
-rw-r--r-- 1 ocloud ocloud 12350 Jul 15 11:04 gover_server.ovf_
-rw-r--r-- 1 ocloud ocloud 818761699 Jul 22 08:49 linuxmint-15-cinnamon-dvd-32bit.iso
-rw-r--r-- 1 ocloud ocloud 818761699 Jul 15 16:51 Video2Brain - Getting Started with Themes in WordPress.zip
-rw-r--r-- 1 ocloud ocloud 56321 Jul 15 11:04 Шаблон письма.docx
root@marvinfs:/opt/ocloud/test4#
Regards, Vladimir
@MarvinFS Please note that phpseclib SFTP does currently not support files larger than 4 GiB.
The following sounds like an issue:
{"reqId":"53cdd1f40687c","app":"PHP","message":"Connection closed prematurely at \/var\/www\/owncloud\/3rdparty\/phpseclib\/phpseclib\/phpseclib\/Net\/SSH2.php#3127","level":3,"time":"2014-07-22T08:52:36+06:00","method":"POST","url":"\/owncloud\/index.php\/apps\/files\/ajax\/upload.php"}
{"reqId":"53cdd1f40687c","app":"PHP","message":"Undefined variable: fp at \/var\/www\/owncloud\/3rdparty\/phpseclib\/phpseclib\/phpseclib\/Net\/SFTP.php#1874","level":3,"time":"2014-07-22T08:52:36+06:00","method":"POST","url":"\/owncloud\/index.php\/apps\/files\/ajax\/upload.php"}
{"reqId":"53cdd1f40687c","app":"PHP","message":"fclose() expects parameter 1 to be resource, null given at \/var\/www\/owncloud\/3rdparty\/phpseclib\/phpseclib\/phpseclib\/Net\/SFTP.php#1874","level":3,"time":"2014-07-22T08:52:36+06:00","method":"POST","url":"\/owncloud\/index.php\/apps\/files\/ajax\/upload.php"}
@bantu too bad! 4GB is nothing in modern world! :( but i was definitely trying 1gig file
@MarvinFS phpseclib will likely add support for files larger than 4 GiB. But since that is not a current feature, please try only with files smaller than 4 GiB. Thanks for testing, btw. :-)
Thanks for info! i'm eager to help improving such a nice product, despite that i can't provide any development, still i'm pretty good in beta testing everything, due to rich IT experience! :) Anyway i'm so much waiting for OC to work reliable with different external storages (btw webdav will be the second issue - i will update my opened issue after sftp), because i'm dead tired of evil google corp and wanna be sure about my privacy with personal cloud
@bantu can we expect any progress here?
@MarvinFS I am sorry this is taking so long. Could you give https://github.com/owncloud/core/issues/9632#issuecomment-49013207 another try. There have been some adjustments in phpseclib. Thanks.
@bantu Dear Andreas, thank you for the answer! i've just finished testing a new ver with clean setup - seem to work now, at least 950MB iso file uploaded successfully to remote SFTP server, i wanna try bigger files also, but the issue is speed now - it became just terrible! it took about 40-50 mins to upload a single 950MB file on a 100Mbit\sec network (the real net bandwidth is about 8-9MB\sec) here is the log
{"reqId":"5407f28db8bfe","app":"OC\\Files\\Cache\\Scanner","message":"!!! Path '' is not accessible or present !!!","level":0,"time":"2014-09-04T11:03:09+06:00","method":"GET","url":"\/owncloud\/index.php\/apps\/files\/ajax\/list.php?dir=%2F&sort=name&sortdirection=asc"}
{"reqId":"5407f2993176f","app":"webdav","message":"Sabre\\DAV\\Exception\\NotAuthenticated: No basic authentication headers were found","level":0,"time":"2014-09-04T11:03:21+06:00","method":"PROPFIND","url":"\/owncloud\/remote.php\/webdav\/"}
{"reqId":"5407f2bc870e8","app":"OC\\Files\\Cache\\Scanner","message":"!!! Path '' is not accessible or present !!!","level":0,"time":"2014-09-04T11:03:56+06:00","method":"GET","url":"\/owncloud\/index.php\/apps\/files\/ajax\/list.php?dir=%2F&sort=name&sortdirection=asc"}
{"reqId":"5407f2cfa9db7","app":"OC\\Files\\Cache\\Scanner","message":"!!! Path '' is not accessible or present !!!","level":0,"time":"2014-09-04T11:04:15+06:00","method":"GET","url":"\/owncloud\/index.php\/apps\/files\/ajax\/list.php?dir=%2F&sort=name&sortdirection=asc"}
{"reqId":"5407f2d332256","app":"webdav","message":"Sabre\\DAV\\Exception\\NotAuthenticated: No basic authentication headers were found","level":0,"time":"2014-09-04T11:04:19+06:00","method":"PROPFIND","url":"\/owncloud\/remote.php\/webdav\/"}
{"reqId":"5407f2df54416","app":"webdav","message":"Sabre\\DAV\\Exception\\NotAuthenticated: No basic authentication headers were found","level":0,"time":"2014-09-04T11:04:31+06:00","method":"PROPFIND","url":"\/owncloud\/remote.php\/webdav\/"}
{"reqId":"5407f2f32c4ca","app":"OC\\Files\\Cache\\Scanner","message":"!!! Path '' is not accessible or present !!!","level":0,"time":"2014-09-04T11:04:51+06:00","method":"GET","url":"\/owncloud\/index.php\/apps\/files\/ajax\/list.php?dir=%2F&sort=name&sortdirection=asc"}
{"reqId":"5407f30043377","app":"OC\\Files\\Cache\\Scanner","message":"!!! Path '' is not accessible or present !!!","level":0,"time":"2014-09-04T11:05:04+06:00","method":"GET","url":"\/owncloud\/index.php\/apps\/files\/ajax\/list.php?dir=%2F&sort=name&sortdirection=asc"}
{"reqId":"5407f30ae9ae9","app":"OC\\Files\\Cache\\Scanner","message":"!!! Path '' is not accessible or present !!!","level":0,"time":"2014-09-04T11:05:14+06:00","method":"GET","url":"\/owncloud\/index.php\/apps\/files\/ajax\/list.php?dir=%2F&sort=name&sortdirection=asc"}
{"reqId":"5407f311839eb","app":"webdav","message":"Sabre\\DAV\\Exception\\NotAuthenticated: No basic authentication headers were found","level":0,"time":"2014-09-04T11:05:21+06:00","method":"PROPFIND","url":"\/owncloud\/remote.php\/webdav\/"}
{"reqId":"5407f31b2a686","app":"webdav","message":"Sabre\\DAV\\Exception\\NotAuthenticated: No basic authentication headers were found","level":0,"time":"2014-09-04T11:05:31+06:00","method":"PROPFIND","url":"\/owncloud\/remote.php\/webdav\/"}
{"reqId":"5407f323d5a57","app":"webdav","message":"Sabre\\DAV\\Exception\\NotAuthenticated: No basic authentication headers were found","level":0,"time":"2014-09-04T11:05:39+06:00","method":"PROPFIND","url":"\/owncloud\/remote.php\/webdav\/"}
{"reqId":"5407f3402c326","app":"webdav","message":"Sabre\\DAV\\Exception\\NotAuthenticated: No basic authentication headers were found","level":0,"time":"2014-09-04T11:06:08+06:00","method":"PROPFIND","url":"\/owncloud\/remote.php\/webdav\/"}
{"reqId":"5407f347cb015","app":"OC\\Files\\Cache\\Scanner","message":"!!! Path 'thumbnails\/20' is not accessible or present !!!","level":0,"time":"2014-09-04T11:06:15+06:00","method":"GET","url":"\/owncloud\/index.php\/core\/preview.png?file=%2FSFTP%2Focloud%2Fpustota.jpg&c=5407f3475ccbb&x=36&y=36&forceIcon=0"}
{"reqId":"5407f347cb015","app":"core","message":"Generating preview for \"\/SFTP\/ocloud\/pustota.jpg\" with \"OC\\Preview\\Image\"","level":0,"time":"2014-09-04T11:06:15+06:00","method":"GET","url":"\/owncloud\/index.php\/core\/preview.png?file=%2FSFTP%2Focloud%2Fpustota.jpg&c=5407f3475ccbb&x=36&y=36&forceIcon=0"}
{"reqId":"5407f347cb015","app":"PHP","message":"file_get_contents(\/var\/www\/owncloud\/data\/admin\/files\/tmp\/497e0a422ad7883425c9e1e3394ed06d.jpg): failed to open stream: No such file or directory at \/var\/www\/owncloud\/lib\/private\/files\/storage\/local.php#156","level":3,"time":"2014-09-04T11:06:15+06:00","method":"GET","url":"\/owncloud\/index.php\/core\/preview.png?file=%2FSFTP%2Focloud%2Fpustota.jpg&c=5407f3475ccbb&x=36&y=36&forceIcon=0"}
{"reqId":"5407f347cb015","app":"PHP","message":"imagecreatefromstring(): Empty string or invalid image at \/var\/www\/owncloud\/lib\/private\/image.php#525","level":3,"time":"2014-09-04T11:06:15+06:00","method":"GET","url":"\/owncloud\/index.php\/core\/preview.png?file=%2FSFTP%2Focloud%2Fpustota.jpg&c=5407f3475ccbb&x=36&y=36&forceIcon=0"}
{"reqId":"5407f347cb015","app":"core","message":"OC_Image->loadFromFile, Default","level":0,"time":"2014-09-04T11:06:15+06:00","method":"GET","url":"\/owncloud\/index.php\/core\/preview.png?file=%2FSFTP%2Focloud%2Fpustota.jpg&c=5407f3475ccbb&x=36&y=36&forceIcon=0"}
{"reqId":"5407f347cb015","app":"core","message":"Generating preview for \"\/SFTP\/ocloud\/pustota.jpg\" with \"OC\\Preview\\Unknown\"","level":0,"time":"2014-09-04T11:06:15+06:00","method":"GET","url":"\/owncloud\/index.php\/core\/preview.png?file=%2FSFTP%2Focloud%2Fpustota.jpg&c=5407f3475ccbb&x=36&y=36&forceIcon=0"}
{"reqId":"5407f347cb015","app":"core","message":"OC_Image->loadFromBase64, couldn't load","level":0,"time":"2014-09-04T11:06:15+06:00","method":"GET","url":"\/owncloud\/index.php\/core\/preview.png?file=%2FSFTP%2Focloud%2Fpustota.jpg&c=5407f3475ccbb&x=36&y=36&forceIcon=0"}
{"reqId":"5407f347cb015","app":"core","message":"OC_Image->fixOrientation() Orientation: -1","level":0,"time":"2014-09-04T11:06:15+06:00","method":"GET","url":"\/owncloud\/index.php\/core\/preview.png?file=%2FSFTP%2Focloud%2Fpustota.jpg&c=5407f3475ccbb&x=36&y=36&forceIcon=0"}
{"reqId":"5407f4707ffd6","app":"OC\\Files\\Cache\\Scanner","message":"!!! No reuse of etag for 'ocloud' !!! \ncache: Array\n(\n [fileid] => 15\n [storage] => sftp::ocloud@marvinfs.convex.ru\/\/\n [path] => ocloud\n [parent] => 14\n [name] => ocloud\n [mimetype] => httpd\/unix-directory\n [mimepart] => httpd\n [size] => -1\n [mtime] => 1405400658\n [storage_mtime] => 1405400658\n [encrypted] => \n [unencrypted_size] => 0\n [etag] => 5407f345f209e\n [permissions] => 31\n)\n \ndata: Array\n(\n [mimetype] => httpd\/unix-directory\n [mtime] => 1409807483\n [size] => -1\n [etag] => 5407f4707fb8a\n [storage_mtime] => 1409807483\n [permissions] => 31\n)\n","level":0,"time":"2014-09-04T11:11:12+06:00","method":"GET","url":"\/owncloud\/index.php\/apps\/files\/ajax\/getstoragestats.php?dir=%2FSFTP%2Focloud"}
{"reqId":"5407fa0ddafd0","app":"OC\\Files\\Cache\\Scanner","message":"!!! No reuse of etag for 'ocloud\/linuxmint-15-cinnamon-dvd-32bit.iso' !!! \ncache: Array\n(\n [fileid] => 26\n [storage] => sftp::ocloud@marvinfs.convex.ru\/\/\n [path] => ocloud\/linuxmint-15-cinnamon-dvd-32bit.iso\n [parent] => 15\n [name] => linuxmint-15-cinnamon-dvd-32bit.iso\n [mimetype] => application\/octet-stream\n [mimepart] => application\n [size] => 116727750\n [mtime] => 1409807679\n [storage_mtime] => 1409807679\n [encrypted] => \n [unencrypted_size] => 0\n [etag] => 5407f4708785b\n [permissions] => 27\n)\n \ndata: Array\n(\n [mimetype] => application\/octet-stream\n [mtime] => 1409809117\n [size] => 973078528\n [etag] => 5407fa0ddaabb\n [storage_mtime] => 1409809117\n [permissions] => 27\n [parent] => 15\n)\n","level":0,"time":"2014-09-04T11:35:09+06:00","method":"POST","url":"\/owncloud\/index.php\/apps\/files\/ajax\/upload.php"}
{"reqId":"5407fa0ddafd0","app":"PHP","message":"call_user_func() expects parameter 1 to be a valid callback, class 'OCA\\Gallery\\Thumbnail' not found at \/var\/www\/owncloud\/lib\/private\/hook.php#78","level":3,"time":"2014-09-04T11:35:10+06:00","method":"POST","url":"\/owncloud\/index.php\/apps\/files\/ajax\/upload.php"}
@MarvinFS Do you have the mcrypt PHP extension installed and enabled?
@bantu as far as i can tell it is: (from the owncloud server)
PHP Version 5.5.9-1ubuntu4.3
System Linux ubuntu14 3.13.0-30-generic #55-Ubuntu SMP Fri Jul 4 21:40:53 UTC 2014 x86_64
Build Date Jul 7 2014 16:33:02
Server API Apache 2.0 Handler
Registered Stream Filters zlib.*, bzip2.*, convert.iconv.*, string.rot13, string.toupper, string.tolower, string.strip_tags, convert.*, consumed, dechunk, mcrypt.*, mdecrypt.*
mcrypt support enabled
mcrypt_filter support enabled
Version 2.5.8
Api No 20021217
Supported ciphers cast-128 gost rijndael-128 twofish arcfour cast-256 loki97 rijndael-192 saferplus wake blowfish-compat des rijndael-256 serpent xtea blowfish enigma rc2 tripledes
Supported modes cbc cfb ctr ecb ncfb nofb ofb stream
Need to put in a release note about the 4GB file size limitation mentioned above.
@MTRichards Good catch.
@bantu what should i do now? i guess this is a totally phpseclib issue... should i move in the phpseclib tree?
Even if phpseclib gets fixed, we still need a ticket here as a reminder to upgrade to the next fixed version of phpseclib.
added to release notes.
@bantu is there anything more we can do about this issue? Did we already update phpseclib properly? THX
Well we are on 0.3.6 (Feb 2014) and current is 0.3.9 (Nov 2014)
on stable7 we are hopelessly out dated with respect to your 3rdparty dependencies: https://www.versioneye.com/user/projects/54bf83f36c0035c592000048
master/oc8 doesn't look that bad: https://www.versioneye.com/user/projects/54a548e697427553100000b9
So from my understanding we should update phpseclib for stable7 @bantu can I ask you to update this in the 3rdparty repo stable7 and apply proper testing? Many thanks
@bantu ping
@DeepDiver1975 Not sure there is anything that can be done about this ticket in particular, but if PHP 5.6.1+ compatibility is desired (I think it is), we should upgrade to phpseclib 0.3.9 for PHP 5.6 improvements. Besides that, the improvements to Net_SSH2 are nice to have for our SFTP support. Also see https://github.com/phpseclib/phpseclib/blob/master/CHANGELOG.md
@MarvinFS Could you please tell us a bit more about your SFTP server? Which operating system is it running. Is it running OpenSSH? Which version?
@bantu you dont think updating seclib from 0.3.6 to 0.3.9 will help?
@nickvergessen I don't know. stable7 should receive a phpseclib upgrade for PHP 5.6 compatibility. @MarvinFS here claims to be using master, which already has a newer version of phpseclib (at least right now).
just to get a clear understanding of the current situation: on master we can upload files to sftp with a max size of ??? on stable7 we can upload files to sftp with a max size of ???
@bantu master uses 0.3.6 I just verified it in the repo
@nickvergessen Then https://www.versioneye.com/user/projects/54a548e697427553100000b9 lies.
@bantu appearantly it does, maybe it's confused by the "*" requirement we have. Current 3rdparty master: https://github.com/owncloud/3rdparty/blob/master/composer.lock#L1278-L1279 Current commit of oc master: https://github.com/owncloud/3rdparty/blob/a32d3924bd0012a5410fff4666131cbdfdec2001/composer.lock#L1278-L1279
Also:
nickv@nickv-lt-mint ~/ownCloud/Master/3rdparty $ ~/Tools/composer.phar update phpseclib/phpseclib
Loading composer repositories with package information
Updating dependencies (including require-dev)
- Removing phpseclib/phpseclib (0.3.6)
- Installing phpseclib/phpseclib (0.3.9)
Downloading: 100%
Writing lock file
Generating autoload files
@DeepDiver1975 I've not actually interpreted this ticket to be about addressing any potentially remaining filesize limitations. Rather, I've always focussed on the "Expected SSH_FXP_STATUS" / "Connection closed prematurely" errors, which were encountered when uploading large files (say 1.8 GiB). I've not been able to reproduce those errors so far.
The version of phpseclib stable7 is using does not clamp filesizes to 4 GiB any more, as per https://github.com/owncloud/core/issues/11432#issuecomment-59217651. As such, there should be no practical filesize limit on 64bit PHP environments.
On 32bit PHP the situation is a bit different. A call to $sftp->put() will result in filesize() being called by phpseclib, which as we know might lie. As such, the limit on 32 bit PHP is 2 GiB. A solution to this part is to extract the filesize tricks into its own library and also use that in phpseclib. I've been meaning to do that.
As per nickvergessen, stable7 and master currently use the same version.
@DeepDiver1975 Additionally, I believe the web interface can not upload files larger than 2 GiB at all on PHP versions below 5.6 (see http://cweiske.de/tagebuch/php-large-file-uploads.htm). And chunking still needs to be investigated on 32bit platforms.
Guys, i'm still greatly concerned about this issue, thank you very much for not letting it go, but i'm on a ski resort right now, will be back on sunday evening and from monday's morning i'll be able to test any new builds (if any) and will provide all the info requested, ok?
@DeepDiver1975 From IRC:
bantu: move to next version I'd say. I haven't been able to reproduce this problem in particular.
Move to 8.1 then
@bantu
Colleagues, what particular test build i may use to see if these issues are still exists? I'm ready to test 'em with latest debian and ubuntu LTS including sftp transfer and large files upload
@bantu i've just tried again following your previous instructions (replace the contents of the owncloud/3rdparty/phpseclib/phpseclib folder of https://download.owncloud.org/community/daily/owncloud-daily-master.tar.bz2 with the contents of the phpseclib-master folder of https://github.com/phpseclib/phpseclib/archive/master.zip which is 0.3.9 apparently ) and in doesn't work anymore - ocloud is not starting - giving me
{"reqId":"dmJdtH1soUKmfzuP3l3E","remoteAddr":"192.168.1.101","app":"PHP","message":"Class 'phpseclib\Crypt\Rijndael' not found at \/var\/www\/owncloud\/3rdparty\/phpseclib\/phpseclib\/phpseclib\/Crypt\/AES.php#58","level":3,"time":"2015-02-10T22:43:59+05:00"}
with original phpseclib from latest ocloud daily master the behavior is still the same - uploads about 818MB to the SFTP server and disconnects leaving partially uploaded file
@MarvinFS Branches in phpseclib have been reorganised. You will have to use https://github.com/phpseclib/phpseclib/archive/1.0.zip instead.
@bantu ok i finished the testing with new 0.3.9 branch (all the testing were over public internet) source server: at office, vmware cluster, ubuntu 14.04 LTS: ocloud daily master clean + 0.3.9. phpaseclib (external storage support app enabled)
dest server: at home, debian 7.8
upload 2.5GB file from source to dest by SFTP external storage - SUCCESS speed 1.3-1.6MB/sec upload 4.5GB file from source to dest by FTP external storage - SUCCESS speed 11.2 - 11.5 MB\sec
in the logfiles nothing useful - seems everything fine.
Whay else can i do?
you will be laughing but now if i try to download that uploaded file from FTP server though ocloud interface it downloads ONLY the ocloud size which is 359.4 MB, if i try to download the same file from remote FTP server directly is downloads the whole 4.5 gigs... md5 matches - i checked it
@MarvinFS if you uploaded through the Web UI that goes through POST, in which case PHP will first store a temp file before giving OC access to it. This might explain the delay. If you try with WebDAV it should be streamed directly.
@PVince81 Got it! makes sense! but the ocloud interface FTP file size issue after upload still remains - should i open other tread? it's not related to phpseclib due to this is plain FTP.
now will try the maximum size upload through SFTP using HPN-SSH on ubuntu - such low transfer speed of standard openssh drives me nuts
Yes, please make another ticket for FTP. Thanks.
uploaded 9.2GB file to SFTP external storage just fine! success at last! next will try 16G. i used HPN-SSH as a SFTP external target server - performance improved DRASTICALLY - 5-10 times! (from 1.5 MB\sec to 5-11MB\sec) i used this manual for Ubuntu https://spoutcraft.org/threads/blazing-fast-sftp-ssh-transfer.7682/ although i failed to find a ready repo for Debian
Regarding latest #9605 SFTP fixes, i'm trying the latest nightly https://download.owncloud.org/community/daily/owncloud-daily-master.tar.bz2 new DB new everything
Steps to reproduce
Expected behaviour
Actual behaviour
Server configuration
Ubuntu 14LTS
Web server: Apache 2.4
Database: MySQL
PHP version: PHP 5.5.9-1ubuntu4.3
ownCloud version: (see ownCloud admin page) july 15 nightly
Updated from an older ownCloud or fresh install: the very fresh
List of activated apps: only defaults and external storages app
The content of config/config.php:
Are you using external storage, if yes which one: sftp
Are you using encryption: yes/no no
Client configuration
Browser: firefox, ie11, chrome Operating system: windows 7, 8.1
Logs
ownCloud log (data/owncloud.log)
php options in .htaccess