[Bug]: Unlocking Office files fails since 3.11.1 (VFS)

Mika-6-9 commented 8 months ago

⚠️ Before submitting, please verify the following: ⚠️

[X] This is a bug, not a question or a configuration issue.
[X] This issue is not already reported on Github (I've searched it).
[x] Nextcloud Server and Desktop Client are up to date. See Server Maintenance and Release Schedule and Desktop Releases for supported versions.
[X] I agree to follow Nextcloud's Code of Conduct

Bug description

Since updating the client to version 3.11.1, office files lock properly when opened from Windows. When closing files, they no longer unlock (or not always). Too bad, the function had just worked more or less correctly.

Steps to reproduce

open and close Office file

Expected behavior

When the file is closed, the file unlocks

Which files are affected by this bug

Office files

Operating system

Windows

Which version of the operating system you are running.

Windows 11 et Windows 10

Package

Official SNAP package

Nextcloud Server version

27.1.1

Nextcloud Desktop Client version

3.11.1

Is this bug present after an update or on a fresh install?

Updated from a minor version (ex. 3.4.2 to 3.4.4)

Are you using the Nextcloud Server Encryption module?

Encryption is Disabled

Are you using an external user-backend?

[ ] Default internal user-backend
[X] LDAP/ Active Directory
[ ] SSO - SAML
[ ] Other

Nextcloud Server logs

No response

Additional info

No response

mgallien commented 8 months ago

@Mika-6-9 could you try again and send us a debug archive ? if you need, I can provide a share link for you to upload it only privately instead of uploading it on github

Mika-6-9 commented 8 months ago

@mgallien Hello I reverted to version 3.11.0. I can redo tests in 3.11.1 and send you the files privately if you give me a link. I will ask collaborators if they can also do tests to have several logs

forsgear commented 8 months ago

I confirm the problem. the same issue on my side. 3.11.0 works fine.

BoMaGithop commented 7 months ago

Same problem, not only with office files, also Xmind...

JohanBoshoffWebAware commented 7 months ago

Same problem on my side. I was using Nextcloud 26, upgraded to 27 and the problem persists

Ornanovitch commented 7 months ago

I can confirm this problem since 3.11.1

Ornanovitch commented 7 months ago

And in my case, the NC desktop logs are spammed with this kind of msg for a locked file (which should be unlocked):

2024-02-13 17:22:30:439 [ info nextcloud.sync.discovery /usr/src/debug/nextcloud-client/nextcloud-client/src/libsync/discovery.cpp:466 ]:   Processing "*************.txt" | (db/local/remote) | valid: true/db/true | mtime: 1707749204/0/1707749204 | size: 10474/0/10474 | etag: "65ca2f640613f"//"65ca2f640613f" | checksum: "SHA1:cbbfe9b75e0057e1e4e35edb7f094749027cd23e"//"SHA1:cbbfe9b75e0057e1e4e35edb7f094749027cd23e" | perm: "WDNVRm"//"WDNVRm" | fileid: "01915906ocane5i2ktyc"//"01915906ocane5i2ktyc" | type: CSyncEnums::ItemTypeFile/CSyncEnums::ItemTypeSkip/CSyncEnums::ItemTypeFile | e2ee: false/false | e2eeMangledName: ""/"" | file lock: locked//locked | metadata missing: /false/
2024-02-13 17:23:11:269 [ info nextcloud.sync.discovery /usr/src/debug/nextcloud-client/nextcloud-client/src/libsync/discovery.cpp:466 ]:   Processing "*************.txt" | (db/local/remote) | valid: true/db/true | mtime: 1707749204/0/1707749204 | size: 10474/0/10474 | etag: "65ca2f640613f"//"65ca2f640613f" | checksum: "SHA1:cbbfe9b75e0057e1e4e35edb7f094749027cd23e"//"SHA1:cbbfe9b75e0057e1e4e35edb7f094749027cd23e" | perm: "WDNVRm"//"WDNVRm" | fileid: "01915906ocane5i2ktyc"//"01915906ocane5i2ktyc" | type: CSyncEnums::ItemTypeFile/CSyncEnums::ItemTypeSkip/CSyncEnums::ItemTypeFile | e2ee: false/false | e2eeMangledName: ""/"" | file lock: locked//locked | metadata missing: /false/
2024-02-13 17:23:26:564 [ info nextcloud.sync.discovery /usr/src/debug/nextcloud-client/nextcloud-client/src/libsync/discovery.cpp:466 ]:   Processing "*************.txt" | (db/local/remote) | valid: true/db/true | mtime: 1707749204/0/1707749204 | size: 10474/0/10474 | etag: "65ca2f640613f"//"65ca2f640613f" | checksum: "SHA1:cbbfe9b75e0057e1e4e35edb7f094749027cd23e"//"SHA1:cbbfe9b75e0057e1e4e35edb7f094749027cd23e" | perm: "WDNVRm"//"WDNVRm" | fileid: "01915906ocane5i2ktyc"//"01915906ocane5i2ktyc" | type: CSyncEnums::ItemTypeFile/CSyncEnums::ItemTypeSkip/CSyncEnums::ItemTypeFile | e2ee: false/false | e2eeMangledName: ""/"" | file lock: locked//locked | metadata missing: /false/
2024-02-13 17:23:41:860 [ info nextcloud.sync.discovery /usr/src/debug/nextcloud-client/nextcloud-client/src/libsync/discovery.cpp:466 ]:   Processing "*************.txt" | (db/local/remote) | valid: true/db/true | mtime: 1707749204/0/1707749204 | size: 10474/0/10474 | etag: "65ca2f640613f"//"65ca2f640613f" | checksum: "SHA1:cbbfe9b75e0057e1e4e35edb7f094749027cd23e"//"SHA1:cbbfe9b75e0057e1e4e35edb7f094749027cd23e" | perm: "WDNVRm"//"WDNVRm" | fileid: "01915906ocane5i2ktyc"//"01915906ocane5i2ktyc" | type: CSyncEnums::ItemTypeFile/CSyncEnums::ItemTypeSkip/CSyncEnums::ItemTypeFile | e2ee: false/false | e2eeMangledName: ""/"" | file lock: locked//locked | metadata missing: /false/
2024-02-13 17:23:57:054 [ info nextcloud.sync.discovery /usr/src/debug/nextcloud-client/nextcloud-client/src/libsync/discovery.cpp:466 ]:   Processing "*************.txt" | (db/local/remote) | valid: true/db/true | mtime: 1707749204/0/1707749204 | size: 10474/0/10474 | etag: "65ca2f640613f"//"65ca2f640613f" | checksum: "SHA1:cbbfe9b75e0057e1e4e35edb7f094749027cd23e"//"SHA1:cbbfe9b75e0057e1e4e35edb7f094749027cd23e" | perm: "WDNVRm"//"WDNVRm" | fileid: "01915906ocane5i2ktyc"//"01915906ocane5i2ktyc" | type: CSyncEnums::ItemTypeFile/CSyncEnums::ItemTypeSkip/CSyncEnums::ItemTypeFile | e2ee: false/false | e2eeMangledName: ""/"" | file lock: locked//locked | metadata missing: /false/
2024-02-13 17:24:00:111 [ info nextcloud.sync.discovery /usr/src/debug/nextcloud-client/nextcloud-client/src/libsync/discovery.cpp:466 ]:   Processing "*************.txt" | (db/local/remote) | valid: true/db/true | mtime: 1707749204/0/1707749204 | size: 10474/0/10474 | etag: "65ca2f640613f"//"65ca2f640613f" | checksum: "SHA1:cbbfe9b75e0057e1e4e35edb7f094749027cd23e"//"SHA1:cbbfe9b75e0057e1e4e35edb7f094749027cd23e" | perm: "WDNVRm"//"WDNVRm" | fileid: "01915906ocane5i2ktyc"//"01915906ocane5i2ktyc" | type: CSyncEnums::ItemTypeFile/CSyncEnums::ItemTypeSkip/CSyncEnums::ItemTypeFile | e2ee: false/false | e2eeMangledName: ""/"" | file lock: locked//locked | metadata missing: /false/

vagner-dias commented 7 months ago

same issue here. automatic unlocking is not working anymore. Tested on: Desktop client: V3.12.0 NC: 27.1.5 files_lock: 27.0.4

JohanBoshoffWebAware commented 7 months ago

I just tested on 3.12.0 and the problem seems to be resolved with this update. I will test with the rest of our team tomorrow morning and post an update if not working. So far so good.

Ornanovitch commented 7 months ago

Just tested on 3.12, unsolved for me :/

Mika-6-9 commented 7 months ago

Hello. Same for me, 3.12 unsolved. Is there not a link with PR? https://github.com/nextcloud/desktop/pull/6347

P.S : @mgallien I'm waiting for the link to upload my logs if you still need them

pschopen commented 7 months ago

For me it was solved with the files_lock update to 28.0.2

smehrbrodt commented 7 months ago

For me it was solved with the files_lock update to 28.0.2

Confirmed, updating files_lock to 28.0.2 fixes the problem here too.

Ornanovitch commented 7 months ago

Updating files_lock to 27.0.4 does not fix it, so I guess this is a NC 27 problem now

timaba commented 7 months ago

I second that issue with following setup Nextcloud Server 28.0.2 Temporary Files Lock 28.0.2 Clients 3.11.1 and 3.12.0 in Win10 and Win11.

Interesting fact: Not all file types are affected, but typical Office files such as docs/xlsx. As described above, the save/close event does not seem to be communicated correctly. After unlocking the file via the context menu, the sync releases the files globally.

AbeilleVerte commented 7 months ago

hello Same issue on Linux Mint, did someone find a way to fix this?

Nextcloud Server 27.1.6 Temporary Files Lock 27.0.4 Clients 3.12.0 on Mint.

dirk80 commented 7 months ago

I have the same issue with Nextcloud 27.1.5 Enterprise Temporary Files Lock 27.0.2 Windows Desktop-Client 3.12.0

Ornanovitch commented 7 months ago

This is more and more a serious issue as users are progressively upgrading to 3.11.1 or 3.12 :/

JohanBoshoffWebAware commented 7 months ago

Most of our problems are resolved on 3.12.0 client and using Nextcloud 27.0.4. When using Redis for memory caching, make sure to double check that redis is running correctly, especially if configured using socket. While I was busy debugging the locking issue prior to the bug release, I found that redis was running, but my sock file was using the wrong directory.

However, we have thousands of files that are still locked. I have no locks in my oc_file_locks table and I cannot clear it as it is an empty set.

Does anyone know if the app Temporary File Lock stores the locks in another table that I can clear? I just want to be able to clear all the files that still shows as locked. Users are having a difficult time unlocking and wastes their time. Even files the users have not worked on for years are now suddenly locked.

Any ideas please? To prevent to conflicted copy, I am using the Temporary File Lock app. Transactional file locking is enabled in nextcloud config and I use redis for that. Restarting the server or services also does not remove the locks.

Ornanovitch commented 7 months ago

Okay, thanks Johan, problem solved but I don't know exactly how/why:

I checked that redis and php-redis were running correctly (and they were), and then I realized that since my last system upgrade, I had php8.3 running next to php8.2. Nextcloud is using php8.2 but after stopping php8.3, uninstalling it and restarting all the services, I am not experiencing this problem anymore :0

JohanBoshoffWebAware commented 7 months ago

That is great news that your problem is resolved. I just have the problem that we have hundreds of thousands of files that are still locked and I have no way of unlocking them. I do use the Temporary Files Lock app so that users are prompted that the file is locked. It works better than having to sort out conflicted files.

As explained, the oc_file_locks table is empty so I do not know where the app stores these locks. Do you maybe know?

Ornanovitch commented 7 months ago

Did you try to disable the Temporary Files Lock app, then to restart php-fpm, then to re-enable the app and restart php-fpm again? I think I had this kind of problem just after the NC desktop upgrade, and that I managed to solve it that way, but I'm not sure

JohanBoshoffWebAware commented 7 months ago

Thank you for the suggestion. I have just done that and saw no log entries, and just as I got happy, they started appearing. You must see my log scrolling of locked issues

Ornanovitch commented 7 months ago

Too bad... I'll keep an eye open on this particular problem as I'm not sure it completely disappeared for all my users

Mika-6-9 commented 7 months ago

With Nextcloud server 27.1.1 and Nextcloud desktop 3.12 AND Nextcloud files_lock 27.0.4 After the update to files_lock 27.0.4, no more problem unlocking office files. I'm going to close this issue, but I'll still leave it open for a while so people with Nextcloud server 28 can say if it works for them

Mika-6-9 commented 7 months ago

https://github.com/nextcloud/files_lock/issues/240

JohanBoshoffWebAware commented 7 months ago

Hi Mika. I am sorry, but how can you close the issue it if you state that it is working with an older version 27.1.1. We are on version 27.1.6 and still have the issue. Yes, it did work on the previous versions. The newest versions are causing the issues.

vagner-dias commented 7 months ago

same issue here. automatic unlocking is not working anymore. Tested on: Desktop client: V3.12.0 NC: 27.1.5 _fileslock: 27.0.4

UPDATE: I've deactivated the file lock application, uninstalled it, and subsequently re-downloaded and re-enabled it. This appears to have resolved the issue. I'll continue to monitor it and provide further updates.

JohanBoshoffWebAware commented 7 months ago

I tried that, but existing locks does not get unlocked.

vagner-dias commented 7 months ago

I tried that, but existing locks does not get unlocked. same here... @mgallien - Any ideas on how to release all locks to start from scratch ?

JohanBoshoffWebAware commented 7 months ago

This is also exactly what I need to know. I cannot believe there is no way to unlock all files that have been locked. Since it does not use the database for locking, how does it keep track of files that are locked?

According to the Temporary Files Lock app there are ways to unlock or lock a file one at a time by knowing the ID of the file, but we have hundreds of thousands of locked files.

AlxxxlA commented 7 months ago

Hi, we got a similar issue here. We tried some updates in the database to unlock the files.

You can remove the locks on the database :

TRUNCATE TABLE oc_files_lock

This update unlock all the files but some desktop clients keep the files lock. You have to edit the file manually on the web interface and apply an update to force a resync of the desktop client.

Another way is to pass the value of the type to 0 to allowed the desktop client or the web interface to unlock the file ignoring the token :

UPDATE oc_files_lock SET type = 0 WHERE type = 2

I think this is the best way to clear the lock file as it just allow users to unlock on their side manually.

If you need the more informations on files lock into the database :

SELECT oc_files_lock.user_id, FROM_UNIXTIME(oc_files_lock.creation), oc_filecache.path, oc_filecache.name FROM oc_files_lock INNER JOIN oc_filecache ON oc_files_lock.file_id = oc_filecache.fileid;

Mika-6-9 commented 7 months ago

With Nextcloud server 27.1.1 and Nextcloud desktop 3.12 AND Nextcloud files_lock 27.0.4 After the update to files_lock 27.0.4, no more problem unlocking office files. I'm going to close this issue, but I'll still leave it open for a while so people with Nextcloud server 28 can say if it works for them

I spoke too quickly, I have quite a few files that don't unlock, but not all the time ?!??

Mika-6-9 commented 7 months ago

Another point, when you do "save as" from an open Office document, the "original" file does not unlock and the new one created does not lock even though it is open and you are going to work in it.

JohanBoshoffWebAware commented 7 months ago

@Mika-6-9 - I am glad you can leave open this thread as there are loads of bugs relating to the locking and unlocking and we can do it so wait for a fixed Desktop Client, or a command on the Temporary File Locks app to be able to unlock all locked files.

vagner-dias commented 7 months ago

With Nextcloud server 27.1.1 and Nextcloud desktop 3.12 AND Nextcloud files_lock 27.0.4 After the update to files_lock 27.0.4, no more problem unlocking office files. I'm going to close this issue, but I'll still leave it open for a while so people with Nextcloud server 28 can say if it works for them

I spoke too quickly, I have quite a few files that don't unlock, but not all the time ?!??

I spoke too quickly too. Some old files which were lock are not unlocking anymore. I am unable to unlock them even if I go to the web interface of the user who locked them and force the unlock operation. Files are stuck now.

Mika-6-9 commented 7 months ago

For information. With version 3.12, I have many more unlock errors with Excel than Word.

allexzander commented 7 months ago

FYI. If you lock a file manually (via the context menu in Windows Explorer or in the Web UI), then open that file with Word/Excel/etc., and then close it, it will not unlock automatically if the lock was imposed manually.

vagner-dias commented 7 months ago

FYI. If you lock a file manually (via the context menu in Windows Explorer or in the Web UI), then open that file with Word/Excel/etc., and then close it, it will not unlock automatically if the lock was imposed manually.

That´s the exact problem. Using:Desktop client: V3.12.0 NC: 27.1.5 files_lock: 27.0.4

everyone is experiencing cases where the file is locked by opening excel, word, etc, and after the file is saved and closed, thus deleting the lock file, the NC lock state is not removed. the file gets locked indefinetivley. In my case this is always happening on files inside group folders

marcotrevisan commented 7 months ago

@vagner-dias that does not seem the exact same use case @allexzander is describing. Seems like you're experiencing it:

regardless the files were previously manually locked,
in group folders.

Ornanovitch commented 7 months ago

I realize this problem happens in group folders in my case too. I thought I had solved it, but I still have problems with users upgrading to 3.11.1 or 3.12. I had to disable the File Lock app

mgallien commented 7 months ago

I am pretty much convinced that this issue has been fixed in the latest releases if you want to test and report the status, please ensure you are running 3.12.1 release of desktop client and release 27.0.4 of files_lock server application or 28.0.2 of files_lock server application

desktop client release 3.12.0 is known to have an issue with files_lock release older than 27.0.4 or 28.0.2, no need to provide feedback on this

Mika-6-9 commented 7 months ago

Hi. Thanks, I'll test this early next week.

Mika-6-9 commented 6 months ago

Hi. Unfortunately, there are still some files that do not unlock when closing the file (mostly Excel) We are continuing the tests.

Other feedback?

vagner-dias commented 6 months ago

I encounter a similar issue, particularly when the Excel file remains open for an extended period. When a user opens and closes the office file within 1-2 hours, everything functions smoothly. However, if the user opens the file in the morning and closes it in the evening, the automatic unlock process fails to occur.

ynott commented 6 months ago

It seems that in some cases the lock does not unlock. If I start and end the edit in Windows Explorer, it will be unlocked successfully. But, it will not unlock when "Edit Locally" from the Nextcloud UI.

marcotrevisan commented 6 months ago

Hi all. I just noticed there's a new release of the Temporary Files Lock app. The changelog looks promising...

ynott commented 6 months ago

I have already tried it with the new desktop client(v3.12.2) and the new Temporary Files Lock(v27.0.5) app. So far it is working perfectly. Even If I select "Edit Locally" from the WebUI, it will still unlock correctly when I finish editing.

But files that are already locked need to be unlocked manually. (This works from the WebUI or from the Desktop client.)

Thanks to all the developers for your hard work!

JohanBoshoffWebAware commented 6 months ago

Looks decent to me also. Is there no command to remove all locks that are currently active for all users with an OCC command? I am sure that will sort out most issues and clear all locked files that was locked incorrectly. Please, developers, that feature will help a lot of users, especially users where there are thousands of locked files. It is impossible to find them one by one and unlock them.

ZID-TU-Graz-Collab commented 6 months ago

Same issue here, In our case the files are only locked when inside a groupfolder AND the groupfolder is shared to another person. On top of that, only under those specific circumstances: When creating a new Excel/Word file locally by rightclicking->new Word Document (then give a chosen name) It creates the new Word Document and another one with the generic name "Microsoft Word-document (new)" Appereantly it instantly snycs the new File to the cloud and is seeing the "renamend" version as a new file. Client 3.12.2 files_lock 27.0.4 (we will give an update when upgraded to 27.0.5) NC 27.1.7 OS (install of Client) Windows

nextcloud / desktop