search

nextcloud / server

☁️ Nextcloud server, a safe home for all your data
https://nextcloud.com
GNU Affero General Public License v3.0
27.42k stars 4.07k forks source link

Failure with multiple files upload (1 of 10 regularly fails) with S3 primary storage #18176

Closed bbelky closed 3 years ago

bbelky commented 4 years ago

Steps to reproduce

  1. Try to upload 10 large files with 5GB size using web interface.
  2. 9 files were uploaded successfully.
  3. 1 file was not uploaded.

Expected behaviour

All files uploaded.

Actual behaviour

One file was not uploaded.

Server configuration

OS: RedHat 7.7 Nextcloud: 16.0.4

Signing status: No errors have been found.

Nextcloud configuration: <?php $CONFIG = array ( 'objectstore' => array ( 'class' => '\OC\Files\ObjectStore\S3', 'arguments' => array ( 'bucket' => 'driveline-mgb-2019-10-21', 'autocreate' => true, 'key' => ‘xxx', 'secret' => ‘xxx', 'hostname' => ‘s3.company.com', 'use_ssl' => true, 'region' => 'nord1', ), ), 'passwordsalt' => ‘xxx', 'secret' => ‘xxx', 'trusted_domains' => array ( 0 => 'localhost', 1 => 'disk.company.com',

), 'datadirectory' => '/var/www/nextcloud/data', 'dbtype' => 'pgsql', 'version' => '16.0.4.1', 'overwrite.cli.url' => 'http://localhost', 'dbname' => 'nextcloud', 'dbhost' => '127.0.0.1', 'dbport' => '', 'dbtableprefix' => 'oc_', 'dbuser' => 'nextcloud', 'dbpassword' => ‘xxx', 'installed' => true, 'instanceid' => ‘xxx', 'memcache.local' => '\OC\Memcache\APCu', 'maintenance' => false, );

Are you using external storage, if yes which one: local/smb/sftp/... S3 Are you using encryption: yes/no No Are you using an external user-backend, if yes which one: LDAP/ActiveDirectory/Webdav/... No

Client configuration

Browser: Chrome 78.0.3904.108 Operating system: MacOS Catalina

Logs

Aws\S3\Exception\S3MultipartUploadException: An exception occurred while uploading parts to a multipart upload. The following parts had errors: - Part 6: Error executing "UploadPart" on "https://driveline-ieng5ziej-2019-09-13.s3.company.com/urn%3Aoid%3A9981?partNumber=6&uploadId=MDA2ODkxMTUxNTc0OTcxNTY2NjY0"; AWS HTTP error: cURL error 55: TCP connection reset by peer (see http://curl.haxx.se/libcurl/c/libcurl-errors.html) (server): 100 Continue - {"reqId":"GYOVa4RkFp5xIL6EYPbm","level":3,"time":"2019-11-28T20:09:55+00:00","remoteAddr":"46.39.51.247","user":"Sergey","app":"objectstore","method":"MOVE","url":"\/remote.php\/dav\/uploads\/Sergey\/web-file-upload-f32499651494b748361dfc2776e57665-1574967551501\/.file","message":{"Exception":"Aws\S3\Exception\S3MultipartUploadException","Message":"An exception occurred while uploading parts to a multipart upload. The following parts had errors:\n- Part 6: Error executing \"UploadPart\" on \"https:\/\/driveline-ieng5ziej-2019-09-13.s3.company.com\/urn%3Aoid%3A9981?partNumber=6&uploadId=MDA2ODkxMTUxNTc0OTcxNTY2NjY0\"; AWS HTTP error: cURL error 55: TCP connection reset by peer (see http:\/\/curl.haxx.se\/libcurl\/c\/libcurl-errors.html) (server): 100 Continue - \n","Code":0,"Trace":[{"function":"Aws\Multipart\{closure}","class":"Aws\Multipart\AbstractUploadManager","type":"->","args":[" sensitive parameters replaced "]},{"file":"\/var\/www\/nextcloud\/3rdparty\/guzzlehttp\/promises\/src\/Coroutine.php","line":122,"function":"send","class":"Generator","type":"->","args":[null]},{"file":"\/var\/www\/nextcloud\/3rdparty\/guzzlehttp\/promises\/src\/Promise.php","line":203,"function":"_handleSuccess","class":"GuzzleHttp\Promise\Coroutine","type":"->","args":[null]},{"file":"\/var\/www\/nextcloud\/3rdparty\/guzzlehttp\/promises\/src\/Promise.php","line":156,"function":"callHandler","class":"GuzzleHttp\Promise\Promise","type":"::","args":[1,null,[{"class":"GuzzleHttp\Promise\Promise"},[{"class":"GuzzleHttp\Promise\Coroutine"},"_handleSuccess"],[{"class":"GuzzleHttp\Promise\Coroutine"},"_handleFailure"]]]},{"file":"\/var\/www\/nextcloud\/3rdparty\/guzzlehttp\/promises\/src\/TaskQueue.php","line":47,"function":"GuzzleHttp\Promise\{closure}","class":"GuzzleHttp\Promise\Promise","type":"::","args":[" sensitive parameters replaced "]},{"file":"\/var\/www\/nextcloud\/3rdparty\/guzzlehttp\/guzzle\/src\/Handler\/CurlMultiHandler.php","line":98,"function":"run","class":"GuzzleHttp\Promise\TaskQueue","type":"->","args":[]},{"file":"\/var\/www\/nextcloud\/3rdparty\/guzzlehttp\/guzzle\/src\/Handler\/CurlMultiHandler.php","line":125,"function":"tick","class":"GuzzleHttp\Handler\CurlMultiHandler","type":"->","args":[]},{"file":"\/var\/www\/nextcloud\/3rdparty\/guzzlehttp\/promises\/src\/Promise.php","line":246,"function":"execute","class":"GuzzleHttp\Handler\CurlMultiHandler","type":"->","args":[" sensitive parameter replaced "]},{"file":"\/var\/www\/nextcloud\/3rdparty\/guzzlehttp\/promises\/src\/Promise.php","line":223,"function":"invokeWaitFn","class":"GuzzleHttp\Promise\Promise","type":"->","args":[]},{"file":"\/var\/www\/nextcloud\/3rdparty\/guzzlehttp\/promises\/src\/Promise.php","line":267,"function":"waitIfPending","class":"GuzzleHttp\Promise\Promise","type":"->","args":[]},{"file":"\/var\/www\/nextcloud\/3rdparty\/guzzlehttp\/promises\/src\/Promise.php","line":225,"function":"invokeWaitList","class":"GuzzleHttp\Promise\Promise","type":"->","args":[]},{"file":"\/var\/www\/nextcloud\/3rdparty\/guzzlehttp\/promises\/src\/Promise.php","line":267,"function":"waitIfPending","class":"GuzzleHttp\Promise\Promise","type":"->","args":[]},{"file":"\/var\/www\/nextcloud\/3rdparty\/guzzlehttp\/promises\/src\/Promise.php","line":225,"function":"invokeWaitList","class":"GuzzleHttp\Promise\Promise","type":"->","args":[]},{"file":"\/var\/www\/nextcloud\/3rdparty\/guzzlehttp\/promises\/src\/Promise.php","line":62,"function":"waitIfPending","class":"GuzzleHttp\Promise\Promise","type":"->","args":[]},{"file":"\/var\/www\/nextcloud\/3rdparty\/guzzlehttp\/promises\/src\/Coroutine.php","line":65,"function":"wait","class":"GuzzleHttp\Promise\Promise","type":"->","args":[]},{"file":"\/var\/www\/nextcloud\/3rdparty\/guzzlehttp\/promises\/src\/Promise.php","line":246,"function":"GuzzleHttp\Promise\{closure}","class":"GuzzleHttp\Promise\Coroutine","type":"->","args":[" sensitive parameters replaced "]},{"file":"\/var\/www\/nextcloud\/3rdparty\/guzzlehttp\/promises\/src\/Promise.php","line":223,"function":"invokeWaitFn","class":"GuzzleHttp\Promise\Promise","type":"->","args":[]},{"file":"\/var\/www\/nextcloud\/3rdparty\/guzzlehttp\/promises\/src\/Promise.php","line":267,"function":"waitIfPending","class":"GuzzleHttp\Promise\Promise","type":"->","args":[]},{"file":"\/var\/www\/nextcloud\/3rdparty\/guzzlehttp\/promises\/src\/Promise.php","line":225,"function":"invokeWaitList","class":"GuzzleHttp\Promise\Promise","type":"->","args":[]},{"file":"\/var\/www\/nextcloud\/3rdparty\/guzzlehttp\/promises\/src\/Promise.php","line":62,"function":"waitIfPending","class":"GuzzleHttp\Promise\Promise","type":"->","args":[]},{"file":"\/var\/www\/nextcloud\/3rdparty\/aws\/aws-sdk-php\/src\/Multipart\/AbstractUploadManager.php","line":83,"function":"wait","class":"GuzzleHttp\Promise\Promise","type":"->","args":[]},{"file":"\/var\/www\/nextcloud\/lib\/private\/Files\/ObjectStore\/S3ObjectTrait.php","line":92,"function":"upload","class":"Aws\Multipart\AbstractUploadManager","type":"->","args":[]},{"file":"\/var\/www\/nextcloud\/lib\/private\/Files\/ObjectStore\/ObjectStoreStorage.php","line":452,"function":"writeObject","class":"OC\Files\ObjectStore\S3","type":"->","args":["urn:oid:9981",null]},{"file":"\/var\/www\/nextcloud\/lib\/private\/Files\/Storage\/Wrapper\/Wrapper.php","line":630,"function":"writeStream","class":"OC\Files\ObjectStore\ObjectStoreStorage","type":"->","args":["files\/2019-11-28 21-54-41 \u2014 \u043a\u043e\u043f\u0438\u044f \u2014 \u043a\u043e\u043f\u0438\u044f (2) \u2014 \u043a\u043e\u043f\u0438\u044f.mov",null,null]},{"file":"\/var\/www\/nextcloud\/lib\/private\/Files\/Storage\/Wrapper\/Wrapper.php","line":630,"function":"writeStream","class":"OC\Files\Storage\Wrapper\Wrapper","type":"->","args":["files\/2019-11-28 21-54-41 \u2014 \u043a\u043e\u043f\u0438\u044f \u2014 \u043a\u043e\u043f\u0438\u044f (2) \u2014 \u043a\u043e\u043f\u0438\u044f.mov",null,null]},{"file":"\/var\/www\/nextcloud\/apps\/dav\/lib\/Connector\/Sabre\/File.php","line":191,"function":"writeStream","class":"OC\Files\Storage\Wrapper\Wrapper","type":"->","args":["files\/2019-11-28 21-54-41 \u2014 \u043a\u043e\u043f\u0438\u044f \u2014 \u043a\u043e\u043f\u0438\u044f (2) \u2014 \u043a\u043e\u043f\u0438\u044f.mov",null]},{"file":"\/var\/www\/nextcloud\/apps\/dav\/lib\/Connector\/Sabre\/Directory.php","line":156,"function":"put","class":"OCA\DAV\Connector\Sabre\File","type":"->","args":[null]},{"file":"\/var\/www\/nextcloud\/3rdparty\/sabre\/dav\/lib\/DAV\/Tree.php","line":316,"function":"createFile","class":"OCA\DAV\Connector\Sabre\Directory","type":"->","args":["2019-11-28 21-54-41 \u2014 \u043a\u043e\u043f\u0438\u044f \u2014 \u043a\u043e\u043f\u0438\u044f (2) \u2014 \u043a\u043e\u043f\u0438\u044f.mov",null]},{"file":"\/var\/www\/nextcloud\/3rdparty\/sabre\/dav\/lib\/DAV\/Tree.php","line":130,"function":"copyNode","class":"Sabre\DAV\Tree","type":"->","args":[{"class":"OCA\DAV\Upload\FutureFile"},{"class":"OCA\DAV\Files\FilesHome"},"2019-11-28 21-54-41 \u2014 \u043a\u043e\u043f\u0438\u044f \u2014 \u043a\u043e\u043f\u0438\u044f (2) \u2014 \u043a\u043e\u043f\u0438\u044f.mov"]},{"file":"\/var\/www\/nextcloud\/3rdparty\/sabre\/dav\/lib\/DAV\/Tree.php","line":161,"function":"copy","class":"Sabre\DAV\Tree","type":"->","args":["uploads\/Sergey\/web-file-upload-f32499651494b748361dfc2776e57665-1574967551501\/.file","files\/Sergey\/2019-11-28 21-54-41 \u2014 \u043a\u043e\u043f\u0438\u044f \u2014 \u043a\u043e\u043f\u0438\u044f (2) \u2014 \u043a\u043e\u043f\u0438\u044f.mov"]},{"file":"\/var\/www\/nextcloud\/3rdparty\/sabre\/dav\/lib\/DAV\/CorePlugin.php","line":642,"function":"move","class":"Sabre\DAV\Tree","type":"->","args":["uploads\/Sergey\/web-file-upload-f32499651494b748361dfc2776e57665-1574967551501\/.file","files\/Sergey\/2019-11-28 21-54-41 \u2014 \u043a\u043e\u043f\u0438\u044f \u2014 \u043a\u043e\u043f\u0438\u044f (2) \u2014 \u043a\u043e\u043f\u0438\u044f.mov"]},{"function":"httpMove","class":"Sabre\DAV\CorePlugin","type":"->","args":[{"absoluteUrl":"https:\/\/disk.mediabg.ru\/remote.php\/dav\/uploads\/Sergey\/web-file-upload-f32499651494b748361dfc2776e57665-1574967551501\/.file","class":"Sabre\HTTP\Request"},{"class":"Sabre\HTTP\Response"}]},{"file":"\/var\/www\/nextcloud\/3rdparty\/sabre\/event\/lib\/EventEmitterTrait.php","line":105,"function":"call_user_func_array","args":[[{"class":"Sabre\DAV\CorePlugin"},"httpMove"],[{"absoluteUrl":"https:\/\/disk.mediabg.ru\/remote.php\/dav\/uploads\/Sergey\/web-file-upload-f32499651494b748361dfc2776e57665-1574967551501\/.file","class":"Sabre\HTTP\Request"},{"class":"Sabre\HTTP\Response"}]]},{"file":"\/var\/www\/nextcloud\/3rdparty\/sabre\/dav\/lib\/DAV\/Server.php","line":479,"function":"emit","class":"Sabre\Event\EventEmitter","type":"->","args":["method:MOVE",[{"absoluteUrl":"https:\/\/disk.mediabg.ru\/remote.php\/dav\/uploads\/Sergey\/web-file-upload-f32499651494b748361dfc2776e57665-1574967551501\/.file","class":"Sabre\HTTP\Request"},{"class":"Sabre\HTTP\Response"}]]},{"file":"\/var\/www\/nextcloud\/3rdparty\/sabre\/dav\/lib\/DAV\/Server.php","line":254,"function":"invokeMethod","class":"Sabre\DAV\Server","type":"->","args":[{"absoluteUrl":"https:\/\/disk.mediabg.ru\/remote.php\/dav\/uploads\/Sergey\/web-file-upload-f32499651494b748361dfc2776e57665-1574967551501\/.file","class":"Sabre\HTTP\Request"},{"class":"Sabre\HTTP\Response"}]},{"file":"\/var\/www\/nextcloud\/apps\/dav\/lib\/Server.php","line":316,"function":"exec","class":"Sabre\DAV\Server","type":"->","args":[]},{"file":"\/var\/www\/nextcloud\/apps\/dav\/appinfo\/v2\/remote.php","line":35,"function":"exec","class":"OCA\DAV\Server","type":"->","args":[]},{"file":"\/var\/www\/nextcloud\/remote.php","line":163,"args":["\/var\/www\/nextcloud\/apps\/dav\/appinfo\/v2\/remote.php"],"function":"require_once"}],"File":"\/var\/www\/nextcloud\/3rdparty\/aws\/aws-sdk-php\/src\/Multipart\/AbstractUploadManager.php","Line":134,"CustomMessage":"Could not create object urn:oid:9981 for files\/2019-11-28 21-54-41 \u2014 \u043a\u043e\u043f\u0438\u044f \u2014 \u043a\u043e\u043f\u0438\u044f (2) \u2014 \u043a\u043e\u043f\u0438\u044f.mov"},"userAgent":"Mozilla\/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit\/537.36 (KHTML, like Gecko) Chrome\/78.0.3904.108 Safari\/537.36","version":"16.0.4.1"}

szaimen commented 3 years ago

Is this Issue still valid in NC21.0.2? If not, please close this issue. Thanks! :)

ghost commented 3 years ago

This issue has been automatically marked as stale because it has not had recent activity and seems to be missing some essential information. It will be closed if no further activity occurs. Thank you for your contributions.