Open dodofox81 opened 1 year ago
cc @artonge
I have try to update my installation to 26.0.1 The problem persist....
@artonge is that also because groupfolders app is installed?
Hi, I have the same problem in NC 26.0.1 and I also have :
Same, any ETA for a fix? Quite problematic.
I have try to disable "file access control" and "group folder", but don't work...
Hi, i am wondering if this issue is not due to this fix : #37768 ?
In the code, there is this line :
$groupFolderOrS3VersioningInstalled = $this->appManager->isInstalled('groupfolders') || $this->appManager->isInstalled('files_versions_s3');
In my installation, the S3 plugin integration it's not installed. So I think that this isn't the problem. But thanks.
I have a docker installation (Nextcloud Hub 4 (26.0.0)
) where it is working correctly.
And I have a native installation (26.0.2
) where it is not working.
Note that on the later, the files get versioned correctly, but the UI does not show any version.
Note 2: On the native version I just tested NC 27.0.0
but it still is not working.
Is there an update to fix this very BIG problem????
Just discovered this same behavior in my environment. I don't use versions very much, but I was just checking on a file that I have made many changes to over the years and was curious how many versions were available, but the versions tab was originally empty. After refreshing, I could see two versions -- my current version and the initial version. In my files_versions folder, I can see many more versions of this same file from the past couple of years, but they are not showing up in the GUI.
Running native installed Nextcloud 26.0.2, all apps are up-to-date.
Hey, it seems like the issue is still not identified. Can some of you check their NC server logs and browser logs looking for related errors?
I just discovered that I am also unable to use Versions correctly after updating to NC26 in a docker container. Anytime an already existing file is saved and synchronized, I get one of these errors in my NC log: nc_log.txt
The file in question I'm looking at is showing it's "initial version" but no edits after that in the Versions tab of the detail view in the web client.
I went and checked a file that I update and re-save all the time and it is also not showing any of it's Versions, only the "initial version" with a date 14 days ago.
My install uses NC26.0.3, MariaDB 10.5.21 backend in docker containers, Linode S3 as Primary Storage, and I do not use File Access Control or Group Folders. I had none of these types of errors prior to upgrading from NC25.
I'm on Nextcloud 27 in Docker and have some versions for all files but they are all broken. Still seems to be a problem.
I did some testing and investigation on my server and found the problem on my deployment. I upgraded to v27 and still had the same issue.
When creating a new file, it seemed versioning was working, so this only seems to be affecting old files that have accumulated versions.
In the databases, there is the main filecache table for storing information about files in the file system (which includes pointers to each file version) and there is a table specifically for versions that is used to track metadata of each version of a corresponding file.
I found that while oc_filescache table had all of the versions listed, the oc_files_versions table was missing all but the most recent couple of files.
Example using a file named "Hardware Tracking.xlsx":
mysql> SELECT fileid,storage,name,size,mtime,storage_mtime FROM oc_filecache WHERE name like '%Hardware Tracking%';
+---------+---------+------------------------------------+--------+------------+---------------+
| fileid | storage | name | size | mtime | storage_mtime |
+---------+---------+------------------------------------+--------+------------+---------------+
| 252721 | 2 | Hardware Tracking.xlsx | 127002 | 1689016203 | 1689016203 |
| 382944 | 2 | Hardware Tracking.xlsx.v1600404344 | 116807 | 1621620936 | 1621620936 |
| 383877 | 2 | Hardware Tracking.xlsx.v1621620937 | 116928 | 1623385161 | 1623385161 |
| 384437 | 2 | Hardware Tracking.xlsx.v1623385161 | 117113 | 1624159722 | 1624159722 |
| 384964 | 2 | Hardware Tracking.xlsx.v1624592501 | 117440 | 1625417783 | 1625417783 |
| 386513 | 2 | Hardware Tracking.xlsx.v1625603092 | 117578 | 1627331254 | 1627331254 |
| 479865 | 2 | Hardware Tracking.xlsx.v1627331523 | 117630 | 1651436950 | 1651436950 |
| 480195 | 2 | Hardware Tracking.xlsx.v1651436950 | 117832 | 1651983043 | 1651983043 |
| 1976615 | 2 | Hardware Tracking.xlsx.v1652129851 | 117611 | 1663464261 | 1663464261 |
| 2407212 | 2 | Hardware Tracking.xlsx.v1663866300 | 117736 | 1665625288 | 1665625288 |
| 2418357 | 2 | Hardware Tracking.xlsx.v1665808816 | 118036 | 1680752167 | 1680752167 |
| 2422603 | 2 | Hardware Tracking.xlsx.v1680752164 | 161873 | 1688265437 | 1688265437 |
| 2449763 | 2 | Hardware Tracking.xlsx.v1688265437 | 126984 | 1689016203 | 1689016203 |
+---------+---------+------------------------------------+--------+------------+---------------+
But here is the results from the oc_files_versions table for the fileid 252721 (the real file id, not a version file id):
mysql> select * from oc_files_versions where file_id='252721';
+-----+---------+------------+--------+----------+----------+
| id | file_id | timestamp | size | mimetype | metadata |
+-----+---------+------------+--------+----------+----------+
| 759 | 252721 | 1680752164 | 161873 | 48 | [] |
| 761 | 252721 | 1688265437 | 126984 | 48 | [] |
+-----+---------+------------+--------+----------+----------+
2 rows in set (0.00 sec)
So just on a whim, I decided to try re-inserting the missing versions. For the timestamp in the oc_files_versions table, I used the timestamp from the filename (i.e. Hardware Tracking.xlsx.v1600404344
INSERT INTO oc_files_versions SET file_id="252721", timestamp='1600404344', size='116807', mimetype=48, metadata="[]";
After this, I refreshed the Nextcloud file list and the newly added version was listed. I then proceeded to re-add all of the missing versions and now I have a complete and working list of file versions for this specific file. This would have to be repeated for any other files.
I was also seeing broken images for preview icons on my word/excel files in the versions list, but this was resolved by enabling the preview option under the ONLYOFFICE app settings.
Hopefully this helps someone else. Maybe someone can use this to develop a script that can find all of the missing file version info and re-add them.
NOTE: Looking at my db backups from a couple of months ago, likely before upgrading to v26, I do not see the oc_files_versions table. Guessing this table was added during the upgrade and did not get populated completely or something.
Can confirm, I ugraded from v25 to 26 on May 1. I did a db backup just before the upgrade and in this db backup (v25) oc_files_versions does not exist. The next back up I have is from May 5 (post v26 upgrade) in which oc_files_versions DOES exist. This table only has a total of 20 entries in it at the time with the last having an id of 23. As of today, there are near 800 entries in that table.
Also wanted to follow up on my previous message and share the final output from mysql after inserting the missing file versions:
mysql> SELECT fileid,name,mimetype,size,mtime,storage_mtime FROM oc_filecache WHERE name LIKE '%Hardware Tracking%';
+---------+------------------------------------+----------+--------+------------+---------------+
| fileid | name | mimetype | size | mtime | storage_mtime |
+---------+------------------------------------+----------+--------+------------+---------------+
| 252721 | Hardware Tracking.xlsx | 48 | 127002 | 1689016203 | 1689016203 |
| 382944 | Hardware Tracking.xlsx.v1600404344 | 48 | 116807 | 1621620936 | 1621620936 |
| 383877 | Hardware Tracking.xlsx.v1621620937 | 48 | 116928 | 1623385161 | 1623385161 |
| 384437 | Hardware Tracking.xlsx.v1623385161 | 48 | 117113 | 1624159722 | 1624159722 |
| 384964 | Hardware Tracking.xlsx.v1624592501 | 48 | 117440 | 1625417783 | 1625417783 |
| 386513 | Hardware Tracking.xlsx.v1625603092 | 48 | 117578 | 1627331254 | 1627331254 |
| 479865 | Hardware Tracking.xlsx.v1627331523 | 48 | 117630 | 1651436950 | 1651436950 |
| 480195 | Hardware Tracking.xlsx.v1651436950 | 48 | 117832 | 1651983043 | 1651983043 |
| 1976615 | Hardware Tracking.xlsx.v1652129851 | 48 | 117611 | 1663464261 | 1663464261 |
| 2407212 | Hardware Tracking.xlsx.v1663866300 | 48 | 117736 | 1665625288 | 1665625288 |
| 2418357 | Hardware Tracking.xlsx.v1665808816 | 48 | 118036 | 1680752167 | 1680752167 |
| 2422603 | Hardware Tracking.xlsx.v1680752164 | 48 | 161873 | 1688265437 | 1688265437 |
| 2449763 | Hardware Tracking.xlsx.v1688265437 | 48 | 126984 | 1689016203 | 1689016203 |
+---------+------------------------------------+----------+--------+------------+---------------+
13 rows in set (0.34 sec)
mysql> SELECT * FROM oc_files_versions WHERE file_id = '252721';
+-----+---------+------------+--------+----------+----------+
| id | file_id | timestamp | size | mimetype | metadata |
+-----+---------+------------+--------+----------+----------+
| 781 | 252721 | 1600404344 | 116807 | 48 | [] |
| 780 | 252721 | 1621620937 | 116928 | 48 | [] |
| 779 | 252721 | 1623385161 | 117113 | 48 | [] |
| 778 | 252721 | 1624592501 | 117440 | 48 | [] |
| 777 | 252721 | 1625603092 | 117578 | 48 | [] |
| 776 | 252721 | 1627331523 | 117630 | 48 | [] |
| 775 | 252721 | 1651436950 | 117832 | 48 | [] |
| 774 | 252721 | 1652129851 | 117611 | 48 | [] |
| 773 | 252721 | 1663866300 | 117736 | 48 | [] |
| 771 | 252721 | 1665808816 | 118036 | 48 | [] |
| 759 | 252721 | 1680752164 | 161873 | 48 | [] |
| 761 | 252721 | 1688265437 | 126984 | 48 | [] |
| 772 | 252721 | 1689016203 | 127002 | 48 | [] |
+-----+---------+------------+--------+----------+----------+
13 rows in set (0.00 sec)
I think the issue is that the files_versions
table only gets populated fully if it's empty for a file when listing its versions. But thinking of it now, entries can be inserted in files_versions
in other places, which would break the former initialisation process.
One solution would be to always check the DB and the FS when listing versions, and to add missing entries in the DB. But I would avoid that if possible, as it would slow down the process a bit.
A second solution would be to join files_versions
with filecache
based on path. This could help with listing all the versions, and we would prob the FS only if some entries are empty. This would also give us the fileid
of the versions which will help for further improvements.
Third solution, would be to prob the DB in places where we create a new DB entries, and to populate it if it is empty.
For now, I think that solution 2 would be the best. Maybe in combination with solution 3. Need to investigate further.
The problem for me it's that the version dir of the user it's empty...
it looks like the versions didn't reflect some changes in the core. If I try "occ versions:expire" I wee error message in SQL Statements:
bash-4.4$ php occ versions:expire A0A2B321-ECEF-D701-80F8-0003BA090E1A Remove deleted files of A0A2B321-ECEF-D701-80F8-0003BA090E1A
In DbalException.php line 71:
An exception occurred while executing a query: SQLSTATE[42S22]: Column not found: 1054 Unknown column 'file_id' in 'where clause'
In ExceptionConverter.php line 69:
An exception occurred while executing a query: SQLSTATE[42S22]: Column not found: 1054 Unknown column 'file_id' in 'where clause'
In Exception.php line 30:
SQLSTATE[42S22]: Column not found: 1054 Unknown column 'file_id' in 'where clause'
In Statement.php line 101:
SQLSTATE[42S22]: Column not found: 1054 Unknown column 'file_id' in 'where clause'
versions:expire [
bash-4.4$
With Nextcloud 27.1.1, I have this strange behaviour : For some (mostly old) files, the file version tab shows multiple versions but only some of them really exist in the user folder. Real versions with corresponding existent files have their preview icon visible, others are blank :
In Onlyoffice history pane, all versions are also listed, and only versions that have an existent corresponding file can be opened :
If I try to open a non existent version, I get this error message which freeze the browser tab :
I suppose that missing files have expired according files_versions policy and have been deleted but the database information has not been updated. But on some files, nearly zero version have been kept, so I suppose there is another bug somewhere...
oc_filecache and oc_files_versions tables are inconsistent. oc_filecache table seems to reflect exactly what is really on disk, and should be reliable to reconstruct the files_versions table which seems to be used to display the available versions to the user on the Nextcloud interface AND inside onlyoffice. Is the maintainer of file_versions code planning to publish a new version cleaning this mess or should we try to update the database ourselves ?
Our installation suffers from inconsistent oc_filecache and oc_files versions. How can we recover from this state?
I needed to disable the versions app for now. It is unusable and confuses our users.
In addition, this problem impairs the activity app: While the versions app is active, only the initial and the first ever change of a file gets listed in the activities tab. All subsequent changes do not appear there (and have dysfunctional entries in the versions tab). As soon as I disabled the versions app, the activities tab works as intended and shows every change of the file. Since files, versions and activities are featured apps of Nextcloud, this is a severe problem and should be fixed immediately.
@ruedigerkupper If you have the ability in your infrastructure to roll back to NC25, Versions still works there. This problem first "appeared" for me when NC26 was tagged stable in Docker and I upgraded. I highly recommend this for you if Versions plays a major role in your use case (as it does in mine). I think it's been broken since March or April 2023. Also, track #38431.
Hello, could those that experienced the issue tell me if they used the occ versions:cleanup
command?
If so, then this PR should fix the issue: https://github.com/nextcloud/server/pull/43263
@artonge I was holding my NC instance at NC25 until I saw some action on this issue, and I spun up a test instance of stable (looks like 27.1.6 in this test) to test out Versions again. Unfortunately, I don't have an instance with the broken file versions to test the occ command, but I think that's OK as the issue I was experiencing here was not expected behavior regardless of the use of occ. It looks like it is behaving as expected in stable, except that I cannot Download or Restore the Initial Version. It looks like this has already been reported, in issue #43262.
Thanks for your work on this issue, I will run a few rounds of testing and we may migrate up to stable if this isn't too much of a problem.
⚠️ This issue respects the following points: ⚠️
Bug description
Hi, I have try to install on a fresh server (debian 11 - apache2 - php8.1 - mariadb) , nextcloud 26. But I have this problem: file versioning don’t work. I have try to create a new doc file with some changed but there isn’t nothing in versioning tab. I have try to disable and re-enable the app… not work. And there isn’t any error in log file. I have try to upgrade another machine with NC25 to NC26. in NC25 versioning work. but after that I have upgraded, file versioning not works.
Steps to reproduce
Expected behavior
I have checked in /var/www/html/data/user/files_versions and it's empty.
Installation method
Fresh install
Nextcloud Server version
26
Operating system
Debian 11
PHP engine version
8.1
Web server
Apache2
Database engine version
MariaDB
Is this bug present after an update or on a fresh install?
Fresh install and upgrade
Are you using the Nextcloud Server Encryption module?
None
What user-backends are you using?
Configuration report
List of activated Apps
Nextcloud Signing status
Nextcloud Logs
Additional info
No response