SMB/CIFS Problem: Could not rename part file to final file

Lumm23 commented 6 years ago

Steps to reproduce

Connect a SMB/CIFS External Storage to cloud
Uplaod any File you want

Expected behaviour

Tell us what should happen

Upload should begin and finish with no issue

Actual behaviour

Tell us what happens instead

The file is trying to upload but exits with the error on top:

Could not rename part file to final file

Server configuration

Operating system:

Ubuntu 18.04 LTS

Web server:

Apache 2

Database:

MariaDB

PHP version:

PHP 7.1

Nextcloud version: (see Nextcloud admin page)

NC 14.0.0.19

Updated from an older Nextcloud/ownCloud or fresh install:

Updated from 13.0.6

Where did you install Nextcloud from:

Tech and me prepared VM installation

Signing status:

Signing status

``` Login as admin user into your Nextcloud and access http://example.com/index.php/settings/integrity/failed paste the results here. No errors have been found. ```

List of activated apps:

App list

- accessibility: 1.0.1 - activity: 2.7.0 - admin_audit: 1.4.0 - bruteforcesettings: 1.1.0 - cloud_federation_api: 0.0.1 - comments: 1.4.0 - dav: 1.6.0 - federatedfilesharing: 1.4.0 - files: 1.9.0 - files_external: 1.5.0 - files_pdfviewer: 1.3.2 - files_rightclick: 0.8.4 - files_sharing: 1.6.2 - files_texteditor: 2.6.0 - files_trashbin: 1.4.1 - files_versions: 1.7.1 - files_videoplayer: 1.3.0 - gallery: 18.1.0 - groupfolders: 1.3.3 - logreader: 2.0.0 - lookup_server_connector: 1.2.0 - metadata: 0.7.0 - nextcloud_announcements: 1.3.0 - notifications: 2.2.1 - oauth2: 1.2.1 - password_policy: 1.4.0 - previewgenerator: 1.1.0 - provisioning_api: 1.4.0 - serverinfo: 1.4.0 - sharebymail: 1.4.0 - support: 1.0.0 - survey_client: 1.2.0 - systemtags: 1.4.0 - theming: 1.5.0 - twofactor_backupcodes: 1.3.1 - updatenotification: 1.4.1 - user_ldap: 1.4.0 - workflowengine: 1.4.0 Disabled: - admin_notifications - announcementcenter - checksum - contacts - encryption - federation - files_accesscontrol - files_downloadactivity - files_retention - firstrunwizard - fulltextsearch - ldaporg - quota_warning - ransomware_protection - tasks - unsplash - user_external - w2g2 ``` If you have access to your command line run e.g.: sudo -u www-data php occ app:list from within your Nextcloud installation folder ```

Nextcloud configuration:

Config report

{ "system": { "passwordsalt": "***REMOVED SENSITIVE VALUE***", "secret": "***REMOVED SENSITIVE VALUE***", "trusted_domains": [ "localhost", "192.168.50.28", "nextcloud", "cloud.gertex.de", "gertex.net", "192.168.50.47" ], "datadirectory": "***REMOVED SENSITIVE VALUE***", "overwrite.cli.url": "https:\/\/cloud.gertex.de\/", "dbtype": "mysql", "version": "14.0.0.19", "dbname": "***REMOVED SENSITIVE VALUE***", "dbhost": "***REMOVED SENSITIVE VALUE***", "dbport": "", "dbtableprefix": "oc_", "mysql.utf8mb4": true, "dbuser": "***REMOVED SENSITIVE VALUE***", "dbpassword": "***REMOVED SENSITIVE VALUE***", "installed": true, "instanceid": "***REMOVED SENSITIVE VALUE***", "quota_include_external_storage": false, "maintenance": false, "mail_smtpmode": "smtp", "log_rotate_size": "10485760", "memcache.local": "\\OC\\Memcache\\Redis", "filelocking.enabled": true, "memcache.distributed": "\\OC\\Memcache\\Redis", "memcache.locking": "\\OC\\Memcache\\Redis", "redis": { "host": "***REMOVED SENSITIVE VALUE***", "port": 0, "timeout": 0, "dbindex": 0, "password": "***REMOVED SENSITIVE VALUE***" }, "htaccess.RewriteBase": "\/", "ldapIgnoreNamingRules": false, "ldapProviderFactory": "\\OCA\\User_LDAP\\LDAPProviderFactory", "mail_from_address": "***REMOVED SENSITIVE VALUE***", "mail_smtpauthtype": "LOGIN", "mail_domain": "***REMOVED SENSITIVE VALUE***", "mail_smtphost": "***REMOVED SENSITIVE VALUE***", "loglevel": 1, "updater.release.channel": "production", "theme": "" } }

Are you using external storage, if yes which one: local/smb/sftp/...

SMB

Are you using encryption: yes/no

No

Are you using an external user-backend, if yes which one: LDAP/ActiveDirectory/Webdav/...

LDAP

LDAP configuration (delete this part if not used)

LDAP config

Client configuration

Browser: Google Chrome Version 69.0.3497.100

Operating system:

Windows 10 Version 1803 Build 17134.286

Logs

Nextcloud log (data/nextcloud.log)

Nextcloud log

Fatal | webdav | Sabre\DAV\Exception: Could not rename part file to final file/var/www/nextcloud/apps/dav/lib/Connector/Sabre/Directory.php - line 156:OCA\DAV\Connector\Sabre\File->put(null)/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php - line 1096:OCA\DAV\Connector\Sabre\Directory->createFile("nix.png", null)/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/CorePlugin.php - line 529:Sabre\DAV\Server->createFile("zwissTEX Me ... g", null, null)Sabre\DAV\CorePlugin->httpPut(Sabre\HTTP\R ... "}, Sabre\HTTP\Response {})/var/www/nextcloud/3rdparty/sabre/event/lib/EventEmitterTrait.php - line 105:call_user_func_array([ Sabre\DAV\ ... "], [ Sabre\HTTP ... }])/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php - line 479:Sabre\Event\EventEmitter->emit("method:PUT", [ Sabre\HTTP ... }])/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php - line 254:Sabre\DAV\Server->invokeMethod(Sabre\HTTP\R ... "}, Sabre\HTTP\Response {})/var/www/nextcloud/apps/dav/appinfo/v1/webdav.php - line 80:Sabre\DAV\Server->exec()/var/www/nextcloud/remote.php - line 163:require_once("/var/www/ne ... p") -- | -- | -- Error | webdav | renaming part file to final file failed ($run: false, $renameOkay: false, $fileExists: false) -- | -- | -- Error | PHP | Undefined variable: run at /var/www/nextcloud/apps/dav/lib/Connector/Sabre/File.php#223 -- | -- | -- Warning | no app in context | Icewind\SMB\Exception\NotFoundException: Invalid request for /0005a_Managementsystem%20Mexico/nix.png (NotFoundException)/var/www/nextcloud/apps/files_external/3rdparty/icewind/smb/src/Native/NativeState.php - line 61:Icewind\SMB\Exception\Exception::fromMap({ 1: "Icewin ... "}, 2, "/0005a_Mana ... g")/var/www/nextcloud/apps/files_external/3rdparty/icewind/smb/src/Native/NativeState.php - line 73:Icewind\SMB\Native\NativeState->handleError("/0005a_Mana ... g")/var/www/nextcloud/apps/files_external/3rdparty/icewind/smb/src/Native/NativeState.php - line 136:Icewind\SMB\Native\NativeState->testResult(false, "smb://192.1 ... g")/var/www/nextcloud/apps/files_external/3rdparty/icewind/smb/src/Native/NativeShare.php - line 166:Icewind\SMB\Native\NativeState->rename("smb://192.1 ... t", "smb://192.1 ... g")/var/www/nextcloud/apps/files_external/lib/Lib/Storage/SMB.php - line 237:Icewind\SMB\Native\NativeShare->rename("/0005a_Mana ... t", "/0005a_Mana ... g")/var/www/nextcloud/lib/private/Files/Storage/Wrapper/Wrapper.php - line 276:OCA\Files_External\Lib\Storage\SMB->rename("nix.png.ocT ... t", "nix.png")/var/www/nextcloud/lib/private/Files/Storage/Wrapper/PermissionsMask.php - line 88:OC\Files\Storage\Wrapper\Wrapper->rename("nix.png.ocT ... t", "nix.png")/var/www/nextcloud/lib/private/Files/Storage/Wrapper/Wrapper.php - line 276:OC\Files\Storage\Wrapper\PermissionsMask->rename("nix.png.ocT ... t", "nix.png")/var/www/nextcloud/lib/private/Files/Storage/Wrapper/Availability.php - line 299:OC\Files\Storage\Wrapper\Wrapper->rename("nix.png.ocT ... t", "nix.png")/var/www/nextcloud/apps/files_trashbin/lib/Storage.php - line 163:OC\Files\Storage\Wrapper\Availability->rename("nix.png.ocT ... t", "nix.png")/var/www/nextcloud/lib/private/Files/Storage/Wrapper/Wrapper.php - line 570:OCA\Files_Trashbin\Storage->rename("nix.png.ocT ... t", "nix.png")/var/www/nextcloud/apps/dav/lib/Connector/Sabre/File.php - line 220:OC\Files\Storage\Wrapper\Wrapper->moveFromStorage(OCA\Files_Tr ... l}, "nix.png.ocT ... t", "nix.png")/var/www/nextcloud/apps/dav/lib/Connector/Sabre/Directory.php - line 156:OCA\DAV\Connector\Sabre\File->put(null)/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php - line 1096:OCA\DAV\Connector\Sabre\Directory->createFile("nix.png", null)/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/CorePlugin.php - line 529:Sabre\DAV\Server->createFile("zwissTEX Me ... g", null, null)Sabre\DAV\CorePlugin->httpPut(Sabre\HTTP\R ... "}, Sabre\HTTP\Response {})/var/www/nextcloud/3rdparty/sabre/event/lib/EventEmitterTrait.php - line 105:call_user_func_array([ Sabre\DAV\ ... "], [ Sabre\HTTP ... }])/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php - line 479:Sabre\Event\EventEmitter->emit("method:PUT", [ Sabre\HTTP ... }])/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php - line 254:Sabre\DAV\Server->invokeMethod(Sabre\HTTP\R ... "}, Sabre\HTTP\Response {})/var/www/nextcloud/apps/dav/appinfo/v1/webdav.php - line 80:Sabre\DAV\Server->exec()/var/www/nextcloud/remote.php - line 163:require_once("/var/www/ne ... p") -- | -- | --

nextcloud-bot commented 6 years ago

GitMate.io thinks possibly related issues are https://github.com/nextcloud/server/issues/9152 (Problem renaming folders), https://github.com/nextcloud/server/issues/9000 (Problem rewriting SMB files), https://github.com/nextcloud/server/issues/8942 (Uploading to shared public upload-only Folder fails: Could not rename part file to final file), https://github.com/nextcloud/server/issues/9091 ( server replied: Internal Server Error (Could not rename part file to final file)), and https://github.com/nextcloud/server/issues/490 (Rename file logging from "ownCloud" to "file").

Lumm23 commented 6 years ago

Does no one has any idea what causes this error or how ti fix it ? I cant upload any file to the external storage. This is very important because there are over 200 users waiting for it. I searched through all the forums i could find and also the things the nextcloud bot suggested me. There is no fix that could have helped me. Could someone please look through this ? Thanks very much.

Lumm23 commented 6 years ago

Does no one else have this problem ?

Lumm23 commented 6 years ago

I really need your help .. just give me any point where i can start searching for the problem ..

mwitschel commented 6 years ago

We had a similiar problem with NetApp as backend for CIFS/SMB external storage. Seems the CIFS implementation on NetApp and Samba/smbclient negotiated SMB1 as protocol but tried to upload with data chunks > 64k which does not work in SMB1 resulting in a butched upload. To fix this behaviour add the following to global section of /etc/samba/smb.conf (path may differ on your system) on your nextcloud server:

client min protocol = SMB2
client max protocol = SMB2
min protocol = SMB2
max protocol = SMB2

Pls let me know if this was of any help.

Source: https://unix.stackexchange.com/questions/367202/setting-smb-2-0-as-default-for-mount-operations

Lumm23 commented 6 years ago

Did not change anything.. i checked the whole smb.conf file there wasnt anything like that in there so i copied that in the file. I tried rebooting the whole machine but it did not affect the error at all ..

mwitschel commented 6 years ago

It may well be that this won't fix your problem. But just to make sure, you put those lines into the global section of smb.conf? It is prefixed by a line "[global]". Otherwise those parameters wont do anything.

Lumm23 commented 6 years ago

That's where i pastet them.

mwitschel commented 6 years ago

seems perfect... then sorry for not beeing helpfull in your case :(

Lumm23 commented 6 years ago

no problem but thanks for tryin .. :/

Lumm23 commented 6 years ago

still not working after 14.0.3 Update ....

egena-its commented 6 years ago

Same here; Tech&Me VM 14.0.3 Exact same issue.

Lumm23 commented 6 years ago

Are you using Netapp with a CIFS share?

egena-its commented 6 years ago

Are you using Netapp with a CIFS share?

I'm using an internally self-hosted Tech&Me VM, updated to 14.03 using the External Storage SMB/CIFS share configuration to a Windows 2012 Std (Not R2) share. Creating a file within Nextcloud works fine but uploads do not, this includes the sync clients on any platform.

I have done a workaround by mounting the windows share using fstab and using "Local" as storage type. This means the share is accessed using one set of windows credentials rather than the LDAP authentication (preferred) but this works very well - and much quicker!

addaitrs commented 5 years ago

Hi Guys,

Just to let you know of a workaround for this. If I create the share as a hidden share this works eg; \server1\Data$ and connect your external storage to that.

No idea why, but it works!

LukeOwlclaw commented 5 years ago

I had the same problem. It turned out to be a permission problem. User "www-data" could not write to the "data/media/files" folder.

dulinux commented 5 years ago

I also got this problem when trying to upload files to a directory I had read access but not write access. Double check if the folder you are trying to upload to is owned by the Nextcloud user you are logged in and is writable.

NarchiX commented 5 years ago

Hi!

Don't want to create new topic, so will post in here:

Nextcloud 14.0.4 Centos 7, with updated kernel to work with SMB3

I have mounted Azure cifs share with such command: sudo mount -t cifs //xxxxxxx.core.windows.net/folder /opt/data/folder -o vers=3.0,username=username,password=passwor,dir_mode=0777,file_mode=0777,sec=ntlmssp,uid=apache_user_id,gid=apache_grp_id

share mounts with no error and syncs. I can create files and folders and new files in those folders with no problems with Nextcloud. Also they nicely syncs to Azure. BUT I can't upload any file using Nextcloud or NC WebDav to that folder or subfolders.

All those files and folders are owned by apache : apache

In error log there is such error:

` Fatal | webdav | Sabre\DAV\Exception: Could not rename part file to final file Error | webdav | renaming part file to final file failed ($run: false, $renameOkay: false, $fileExists: false) Error | PHP | Undefined variable: run at /var/www/html/nextcloud/apps/dav/lib/Connector/Sabre/File.php#223 Error | PHP | rename(/opt/data/folder/imgpsh_fullsize.png.ocTransferId583196258.part,/opt/data/folder/imgpsh_fullsize.png): Resource temporarily unavailable at /var/www/html/nextcloud/lib/private/Files/Storage/Local.php#272

`

Any advice, please?!

david-pace commented 5 years ago

I fixed this error by adding the line

writeable = yes

to the configuration of the corresponding Samba shares. Also make sure that the Samba user which is used by Nextcloud is in the write list of the share. If multiple users are in the write list, make sure they all have file system access to the physical folder or use the force user directive to specify a user which has file system access.

A complete Samba configuration might look like this:

[MyShare]
path = /path/to/my/share
write list = bob,nextcloud
valid users = bob,nextcloud
force user = bob
writeable = yes

In this example, nextcloud is the user utilized by Nextcloud whereas bob is the Linux user which owns the /path/to/my/share directory.

jammon commented 4 years ago

I get this error with filenames containing characters like ":", "/" or "?". All other files sync normally.

skjnldsv commented 4 years ago

I get this error with filenames containing characters like ":", "/" or "?". All other files sync normally.

Reproductible every time? cc @icewind1991

jammon commented 4 years ago

@skjnldsv Yes!

valeriop commented 4 years ago

We had a similiar problem with NetApp as backend for CIFS/SMB external storage. Seems the CIFS implementation on NetApp and Samba/smbclient negotiated SMB1 as protocol but tried to upload with data chunks > 64k which does not work in SMB1 resulting in a butched upload. To fix this behaviour add the following to global section of /etc/samba/smb.conf (path may differ on your system) on your nextcloud server:
client min protocol = SMB2
client max protocol = SMB2
min protocol = SMB2
max protocol = SMB2
Pls let me know if this was of any help.

Source: https://unix.stackexchange.com/questions/367202/setting-smb-2-0-as-default-for-mount-operations

To anyone using Netapp: 1) change smb.conf according above instructions 2) Login to Netapp admin web GUI (ONTAP System Manager) Storage->Shares and edit "Share" options you want to connect/mount as External Storage. You will find a "Symbolic links" section. Change from ¨Symlinks and widelinks" to "Symlinks" Alternatively you can issue the relative cmdline : vserver cifs share modify -vserver _vservername -share-name _sharename -symlink-properties symlinks Hope this helps.

schorschii commented 4 years ago

Thanks @valeriop , you saved my day :) We're also using Netapp. I thought it was a Samba issue until I found your post. After changing the symlinks option I was able to rename files on our Netapp share via GVFS (Nemo file manager).

iburadempa commented 4 years ago

Failing to rename files/dirs with smbclient/libsmbclient for protocols > SMB1 appears to be a samba bug, cf.: https://bugzilla.samba.org/show_bug.cgi?id=13599 https://bugzilla.samba.org/show_bug.cgi?id=14169 There is also a Debian bug: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=949394

In my case with nextcloud on Debian buster trying to connect to a samba server on Debian stretch I could work around the issue with 1) letting icewind use the wrapper for smbclient by removing libsmbclient-php (apt remove php-smbclient) and 2) patching apps/files_external/3rdparty/icewind/smb/src/Wrapped/Share.php in line 84, replacing the smbclient command invokation '%s %s%s -t %s %s %s %s', with '%s %s%s -t %s -m NT1 %s %s %s', so that all SMB connections to the server have the maximum protocol version NT1. Of course, the server must support the NT1 dialect. To check if NT1 is supported, use: smbclient -d 10 -m NT1 -U nextcloud_user //SERVER/SHARE

gismomtl commented 4 years ago

FYI, with Version 19.0.1 July 17 2020, issue was still occuring with SMB share during upload of file.. and not where creating subfolder. Modifying /etc/samba/smb.conf with client min protocol = SMB2 client max protocol = SMB2 min protocol = SMB2 max protocol = SMB2

was not working but client min protocol = NT1 client max protocol = NT1 min protocol = NT1 max protocol = NT1

is working. No need to patch any other file

iburadempa commented 4 years ago

client min protocol = NT1 client max protocol = NT1 min protocol = NT1 max protocol = NT1

is working. No need to patch any other file

... if you are fine with other samba clients connecting through a very unsafe protocol of if you have no other clients.

gismomtl commented 4 years ago

Do we have a solution for SMB2 or SMB3 ? especially if NAS parameters cannot be changed or is not NetAPP...

ntinti commented 3 years ago

Problem still persist. Going back to NT1 is no solution. Problem is Nextcloud / icewind, not the samba-client

Tschortsch666 commented 3 years ago

client min protocol = NT1 client max protocol = NT1 min protocol = NT1 max protocol = NT1 is working. No need to patch any other file

... if you are fine with other samba clients connecting through a very unsafe protocol of if you have no other clients.

I used a Windows Server 2016 as AD Controller and this fixed it temporary. As I use NextCloud in a Docker Container, the change is not persistant. Is there any solution in the meanwhile? Even simple file operation do not work like renaming, deleting files.

Tschortsch666 commented 3 years ago

Problem still persist. Going back to NT1 is no solution. Problem is Nextcloud / icewind, not the samba-client

Yes, I also think so.

ynott commented 3 years ago

@Tschortsch666

As I use NextCloud in a Docker Container, the change is not persistant. Is there any solution in the meanwhile?

This isn't the main topic, but How about browsing samba.conf on the Docker host side from within the Docker container?

like this.

mkdir /docker-host/samba/
cp /etc/samba/smb.conf /docker-host/samba/
docker run nextcloud:stable -v /docker-host/samba:/etc/samba

Tschortsch666 commented 3 years ago

@ynott: Yes, as I am currently no docker-expert, I did not think about this solution. Will add this to the Dockerfile. Anyway - a solution should be provided as I think that the Setup AD-Controller <> DFS-Shares <> NextCloud is Enterprise-like ;)

Cheers, Tschortsch

coderReview commented 2 years ago

https://bugzilla.samba.org/show_bug.cgi?id=14169 This has been fixed on latest version of samba. Some other issues related to the SAMBA library that had issues with Nextcloud and Owncloud were fixed as well.

szaimen commented 1 year ago

Hi, please update to 24.0.8 or better 25.0.2 and report back if it fixes the issue. Thank you!

Tschortsch666 commented 1 year ago

Hi,

As far as I can reproduce it now, it is fixed for us here.

BR

From: Simon L. @.> Sent: Monday, January 9, 2023 12:50 PM To: nextcloud/server @.> Cc: Eichhorn, Georg @.>; Mention @.> Subject: Re: [nextcloud/server] SMB/CIFS Problem: Could not rename part file to final file (#11295)

Hi, please update to 24.0.8 or better 25.0.2 and report back if it fixes the issue. Thank you!

— Reply to this email directly, view it on GitHubhttps://smex-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2fgithub.com%2fnextcloud%2fserver%2fissues%2f11295%23issuecomment%2d1375509794&umid=bd12afad-084c-4c8f-9b0b-23ee1483d378&auth=6bd9a9a560409b64e076cdbf5a16c4c84dfa12ca-7d72a5775840e872ae872dc54d80c50f24f37f82, or unsubscribehttps://smex-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2fgithub.com%2fnotifications%2funsubscribe%2dauth%2fASTYOGC64C2TEGG63URUQWTWRP3QJANCNFSM4FWBDOQQ&umid=bd12afad-084c-4c8f-9b0b-23ee1483d378&auth=6bd9a9a560409b64e076cdbf5a16c4c84dfa12ca-206d3f12c5fc8dd338949331781f2b86f639968f. You are receiving this because you were mentioned.Message ID: @.**@.>>

szaimen commented 1 year ago

Great! Thanks for verifying!

akarabach commented 8 months ago

@szaimen It should be reopened, [Nextcloud Hub 7] (28.0.4)

nextcloud / server