search

nextcloud / groupfolders

📁👩‍👩‍👧‍👦 Admin-configured folders shared by everyone in a group. https://github.com/nextcloud-releases/groupfolders
https://apps.nextcloud.com/apps/groupfolders
284 stars 87 forks source link

Backends provided no user object / Duplicate entry for key 'gf_versions_uniq_index' #2559

Closed vidlb closed 9 months ago

vidlb commented 1 year ago

Steps to reproduce

Error appears in the logs with cron jobs, or when using docker run (...) occ groupfolders:expire -v

=> That was the only step to reproduce the "backends provided no user object" error (prior to update 15.3.2 - 16.0.1).
Regarding "duplicate entry for key", see 4 comments starting from https://github.com/nextcloud/groupfolders/issues/2559#issuecomment-1881314205

Actual behaviour

In Root.php line 364:
  [OC\User\NoUserException] Backends provided no user object

Server configuration

Operating system: Debian 12 / Docker 24

Web server: Nginx reverse proxy

Nextcloud version: 27.1

Group folders version: 15.3

Updated from an older Nextcloud/ownCloud or fresh install: Updated from 27.0

Where did you install Nextcloud from: Nextcloud AIO

Are you using external storage, if yes which one: local/s3/smb/sftp/... No

Are you using encryption: yes/no No

Are you using an external user-backend, if yes which one: LDAP/ActiveDirectory/Webdav/Saml/... No

Logs (docker run)

Exception trace:
  at /var/www/html/lib/private/Files/Node/Root.php:364
 OC\Files\Node\Root->getUserFolder() at n/a:n/a
 call_user_func_array() at /var/www/html/lib/private/Files/Node/LazyFolder.php:74
 OC\Files\Node\LazyFolder->__call() at /var/www/html/lib/private/Files/Node/LazyRoot.php:40
 OC\Files\Node\LazyRoot->getUserFolder() at /var/www/html/custom_apps/groupfolders/lib/Versions/VersionsBackend.php:73
 OCA\GroupFolders\Versions\VersionsBackend->getVersionsForFile() at /var/www/html/custom_apps/groupfolders/lib/Versions/GroupVersionsExpireManager.php:72
 OCA\GroupFolders\Versions\GroupVersionsExpireManager->expireFolder() at /var/www/html/custom_apps/groupfolders/lib/Versions/GroupVersionsExpireManager.php:59
 OCA\GroupFolders\Versions\GroupVersionsExpireManager->expireAll() at /var/www/html/custom_apps/groupfolders/lib/Command/ExpireGroup/ExpireGroupVersions.php:66
 OCA\GroupFolders\Command\ExpireGroup\ExpireGroupVersions->execute() at /var/www/html/custom_apps/groupfolders/lib/Command/ExpireGroup/ExpireGroupVersionsTrash.php:54
 OCA\GroupFolders\Command\ExpireGroup\ExpireGroupVersionsTrash->execute() at /var/www/html/3rdparty/symfony/console/Command/Command.php:298
 Symfony\Component\Console\Command\Command->run() at /var/www/html/core/Command/Base.php:177
 OC\Core\Command\Base->run() at /var/www/html/3rdparty/symfony/console/Application.php:1040
 Symfony\Component\Console\Application->doRunCommand() at /var/www/html/3rdparty/symfony/console/Application.php:301
 Symfony\Component\Console\Application->doRun() at /var/www/html/3rdparty/symfony/console/Application.php:171
 Symfony\Component\Console\Application->run() at /var/www/html/lib/private/Console/Application.php:211
 OC\Console\Application->run() at /var/www/html/console.php:100
 require_once() at /var/www/html/occ:11

Logs (cron errors in nextcloud admin panel)

Error   core
OC\User\NoUserException: Backends provided no user object
    <<closure>>
    OC\Files\Node\Root->getUserFolder("")

    /var/www/html/lib/private/Files/Node/LazyFolder.php - line 74:
    call_user_func_array([ [ "OC\\Fil ... "], [ ""])

    /var/www/html/lib/private/Files/Node/LazyRoot.php - line 40:
    OC\Files\Node\LazyFolder->__call("getUserFolder", [ ""])

    /var/www/html/custom_apps/groupfolders/lib/Versions/VersionsBackend.php - line 73:
    OC\Files\Node\LazyRoot->getUserFolder("")

    /var/www/html/custom_apps/groupfolders/lib/Versions/GroupVersionsExpireManager.php - line 72:
    OCA\GroupFolders\Versions\VersionsBackend->getVersionsForFile([ "OC\\User\\User"], [ "OC\\Files\\FileInfo"])

    /var/www/html/custom_apps/groupfolders/lib/Versions/GroupVersionsExpireManager.php - line 59:
    OCA\GroupFolders\Versions\GroupVersionsExpireManager->expireFolder([ 2,"LaTeleS ... "])

    /var/www/html/custom_apps/groupfolders/lib/BackgroundJob/ExpireGroupVersions.php - line 43:
    OCA\GroupFolders\Versions\GroupVersionsExpireManager->expireAll()

    /var/www/html/lib/public/BackgroundJob/Job.php - line 81:
    OCA\GroupFolders\BackgroundJob\ExpireGroupVersions->run(null)

    /var/www/html/lib/public/BackgroundJob/TimedJob.php - line 103:
    OCP\BackgroundJob\Job->start([ "OC\\BackgroundJob\\JobList"])

    /var/www/html/lib/public/BackgroundJob/TimedJob.php - line 93:
    OCP\BackgroundJob\TimedJob->start([ "OC\\BackgroundJob\\JobList"])

    /var/www/html/cron.php - line 152:
    OCP\BackgroundJob\TimedJob->execute([ "OC\\BackgroundJob\\JobList"], [ "OC\\Log"])

The bug

It seems the error is due to this line where getVersionsForFile is called with a dummy user, which isn't implemented (yet ?) since getVersionsForFile is clearly requiring a real user, because it will call $user->getUID() which will return NULL and then cause the exception with OC\Files\Node\Root->getUserFolder("")

I couldn't find existing issues, but may be this is something already fixed in master / targeting nextcloud 28 ?
Will it disappear when AIO switch to NC 28 ?

VVD commented 10 months ago

Did several more tests - sometimes it work, but in most cases not. Couldn't identify the dependency. But if it's subfolder groupfolder in other groupfolder and have "more than 10 files", then sync doesn't work. Even if I change 1 file 10 bytes size.

VVD commented 10 months ago

Sync of files in subfolder groupfolder in other groupfolder (parent groupfolder) is happening after something changed in parent groupfolder. Try the scenario:

  1. Create groupfolder /Test1, then /Test1/Test2 and configure access.
  2. Turn on in Nextcloud Desktop Client sync of the /Test1 and /Test1/Test2.
  3. In Nextclod web interface create file /Test1/Test2/test.txt.
  4. Check local storage connected via Nextcloud Desktop Client - no new file /Test1/Test2/test.txt.
  5. In Nextclod web interface create file /Test1/test.txt.
  6. Check local storage connected via Nextcloud Desktop Client - both files are here /Test1/Test2/test.txt and /Test1/test.txt.
rakekniven commented 10 months ago

Did several more tests - sometimes it work, but in most cases not. Couldn't identify the dependency. But if it's subfolder groupfolder in other groupfolder and have "more than 10 files", then sync doesn't work. Even if I change 1 file 10 bytes size.

From my understanding this is not related to this issue. Please open a new after searching the existing ones.

VVD commented 10 months ago

From my understanding this is not related to this issue. Please open a new after searching the existing ones.

Sorry for noise, I had issue with:

An exception occurred while executing a query: SQLSTATE[23505]: Unique violation: 7 ERROR: duplicate key value violates unique constraint "gf_versions_uniq_index"

too and I thought it was somehow connected.

I found correct issues: 2 issues here and 1 at Nextcloud Desktop Client project.

TomasPopovas commented 10 months ago

The same problem - after updating to version 27.1.4, I have the same problem as in the topic - At the same time, two-factor authorization was reset for most users, and not from the first time users could log into their profile on the page again. Many were blocked and after unblocking, everyone logged in without problems after 1-2 attempts. Whether there are 1 or 2 of you, this is not a problem - but when there are 100 of you or more, these are fun days for IT support)) updating to version 27.1.5 did not help

nursoda commented 10 months ago

For me deleting my file trashbin was the solution. / Cannot confirm that.

I confirm that I did NOT IMMEDIATELY get new error messages by manually running cron.php on my production instances after deleting ALL trashbins using sudo 33 php /PATH/TO/CLOUD/occ trashbin:cleanup --all-users.

In that state, this trashbin deletion also had the somewhat unexpected side-effect that the /settings/admin/logging now loads immediately on those instances that suffered from logreader issue 1073.

However, consecutive cron.php runs brought back the errors in the log and the logreader stall issue – even without ANY interaction in the corresponding NC instance!

Adding @solracsf 's logger statement does not produce additional output in nextcloud.log, neither on the above 'deleted trasbins' instance nor on one that still show the error upon running cron.php.

BTW1: Might groupfolder issue 2466 also be related? BTW2: Might server issue 39273 also be related?

BTW3: It took me quite a while to figure out which is the issue to track. It might be a good idea to include "Duplicate entry gf_versions_uniq_index" in the issue subject line.

rakekniven commented 10 months ago

The same problem

No. Please see first initial posting. It is about error messages, not about blocked users.

D3nnis3n commented 9 months ago

I also experience this issue, one log entry every hour, as others described. (And I only noticed because Nextcloud 28 now shows amount of errors in the overview admin page)

tossy-yossy commented 9 months ago

I also had the "Integrity constraint violation" error in my instance. For me deleting my file trashbin was the solution.

By emptying the trash bin, the error disappeared once, but when I moved the files in the group folder to the trash bin, the error started appearing again.

BJKle commented 9 months ago

I also had the "Integrity constraint violation" error in my instance. For me deleting my file trashbin was the solution.

By emptying the trash bin, the error disappeared once, but when I moved the files in the group folder to the trash bin, the error started appearing again.

I can confirm. I setup a new NC and get the error log filled as well.

mokkin commented 9 months ago

I cleaned all trashbins but it didn't help to fix the issue NC 28.0.1

vidlb commented 9 months ago

I can confirm this duplicate key error with NC 27.1.5, while I didn't update anything manually or changed settings.
Looks like something was missing in the migrations, is there a command line we can run to manually fix this corrupted index ?

le-patenteux commented 9 months ago

Hello, as of this moment, is there a workaround to make this work again? (The trashbin trick did not work for me) Is there a way to execute the command as a certain user to make it work again? My file versioning has started crushing the space available on my server.

NC 28.0.1

vasyugan commented 9 months ago

Hello, as of this moment, is there a workaround to make this work again?

I have found no workaround mentioned in the forum or anywhere else. Which is a problem given that nobody seems to be working on this, even though this has been breaking major functionality since September. I wonder if we are going to have to put up a bounty in order to get this fixed.

marcotrevisan commented 9 months ago

Hello, as of this moment, is there a workaround to make this work again?

I have found no workaround mentioned in the forum or anywhere else. Which is a problem given that nobody seems to be working on this, even though this has been breaking major functionality since September. I wonder if we are going to have to put up a bounty in order to get this fixed.

I just checked, there's work underway for this issue. https://github.com/nextcloud/groupfolders/pull/2720 Hopefully it will be resolved soon.

vasyugan commented 9 months ago

Hello, as of this moment, is there a workaround to make this work again?

I have found no workaround mentioned in the forum or anywhere else. Which is a problem given that nobody seems to be working on this, even though this has been breaking major functionality since September. I wonder if we are going to have to put up a bounty in order to get this fixed.

I just checked, there's work underway for this issue. #2720 Hopefully it will be resolved soon.

Thanks! I just tested the changed lib/Versions/VersionsBackend.php but unfortunately, running occ groupfolders:expire I got exactly the same error message as before.

marcotrevisan commented 9 months ago

Thanks! I just tested the changed lib/Versions/VersionsBackend.php but unfortunately, running occ groupfolders:expire I got exactly the same error message as before.

Are you getting the exact same stack trace as reported i.e. line 88 of VersionsBackend.php, and a subsequent call to insert()? If so, you might need to remove the old code in your edited file...

Coolsero commented 9 months ago

Hello,

I have the same problem. I have attached two images here with the error messages before I upload the file lib/Versions/VersionsBackend.php

I replaced it completely and then.

Coolsero

New PHP File New-Version

Old PHP File Old-Version

vasyugan commented 9 months ago

Thanks! I just tested the changed lib/Versions/VersionsBackend.php but unfortunately, running occ groupfolders:expire I got exactly the same error message as before.

Are you getting the exact same stack trace as reported i.e. line 88 of VersionsBackend.php, and a subsequent call to insert()? If so, you might need to remove the old code in your edited file...

I haven't edited but replaced the file.

This is the output I got with the old file:

Expiring version in '*****'

In DbalException.php line 71:

  [OC\DB\Exceptions\DbalException (1062)]                                                                                                            
  An exception occurred while executing a query: SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry '18854-1689327178' for key '  
  gf_versions_uniq_index'                                                                                                                            

Exception trace:
  at /var/www/nextcloud/lib/private/DB/Exceptions/DbalException.php:71
 OC\DB\Exceptions\DbalException::wrap() at /var/www/nextcloud/lib/private/DB/QueryBuilder/QueryBuilder.php:328
 OC\DB\QueryBuilder\QueryBuilder->executeStatement() at /var/www/nextcloud/lib/public/AppFramework/Db/QBMapper.php:137
 OCP\AppFramework\Db\QBMapper->insert() at /var/www/nextcloud/apps/groupfolders/lib/Versions/VersionsBackend_OLD.php:88
 OCA\GroupFolders\Versions\VersionsBackend->getVersionsForFile() at /var/www/nextcloud/apps/groupfolders/lib/Versions/GroupVersionsExpireManager.php:72
 OCA\GroupFolders\Versions\GroupVersionsExpireManager->expireFolder() at /var/www/nextcloud/apps/groupfolders/lib/Versions/GroupVersionsExpireManager.php:59
 OCA\GroupFolders\Versions\GroupVersionsExpireManager->expireAll() at /var/www/nextcloud/apps/groupfolders/lib/Command/ExpireGroup/ExpireGroupVersions.php:66
 OCA\GroupFolders\Command\ExpireGroup\ExpireGroupVersions->execute() at /var/www/nextcloud/apps/groupfolders/lib/Command/ExpireGroup/ExpireGroupVersionsTrash.php:54
 OCA\GroupFolders\Command\ExpireGroup\ExpireGroupVersionsTrash->execute() at /var/www/nextcloud/3rdparty/symfony/console/Command/Command.php:298
 Symfony\Component\Console\Command\Command->run() at /var/www/nextcloud/core/Command/Base.php:177
 OC\Core\Command\Base->run() at /var/www/nextcloud/3rdparty/symfony/console/Application.php:1040
 Symfony\Component\Console\Application->doRunCommand() at /var/www/nextcloud/3rdparty/symfony/console/Application.php:301
 Symfony\Component\Console\Application->doRun() at /var/www/nextcloud/3rdparty/symfony/console/Application.php:171
 Symfony\Component\Console\Application->run() at /var/www/nextcloud/lib/private/Console/Application.php:206
 OC\Console\Application->run() at /var/www/nextcloud/console.php:100
 require_once() at /var/www/nextcloud/occ:11

In ExceptionConverter.php line 62:

  [Doctrine\DBAL\Exception\UniqueConstraintViolationException (1062)]                                                                                
  An exception occurred while executing a query: SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry '18854-1689327178' for key '  
  gf_versions_uniq_index'                                                                                                                            

Exception trace:
  at /var/www/nextcloud/3rdparty/doctrine/dbal/src/Driver/API/MySQL/ExceptionConverter.php:62
 Doctrine\DBAL\Driver\API\MySQL\ExceptionConverter->convert() at /var/www/nextcloud/3rdparty/doctrine/dbal/src/Connection.php:1938
 Doctrine\DBAL\Connection->handleDriverException() at /var/www/nextcloud/3rdparty/doctrine/dbal/src/Connection.php:1880
 Doctrine\DBAL\Connection->convertExceptionDuringQuery() at /var/www/nextcloud/3rdparty/doctrine/dbal/src/Connection.php:1208
 Doctrine\DBAL\Connection->executeStatement() at /var/www/nextcloud/lib/private/DB/Connection.php:294
 OC\DB\Connection->executeStatement() at /var/www/nextcloud/3rdparty/doctrine/dbal/src/Query/QueryBuilder.php:386
 Doctrine\DBAL\Query\QueryBuilder->execute() at /var/www/nextcloud/lib/private/DB/QueryBuilder/QueryBuilder.php:280
 OC\DB\QueryBuilder\QueryBuilder->execute() at /var/www/nextcloud/lib/private/DB/QueryBuilder/QueryBuilder.php:326
 OC\DB\QueryBuilder\QueryBuilder->executeStatement() at /var/www/nextcloud/lib/public/AppFramework/Db/QBMapper.php:137
 OCP\AppFramework\Db\QBMapper->insert() at /var/www/nextcloud/apps/groupfolders/lib/Versions/VersionsBackend_OLD.php:88
 OCA\GroupFolders\Versions\VersionsBackend->getVersionsForFile() at /var/www/nextcloud/apps/groupfolders/lib/Versions/GroupVersionsExpireManager.php:72
 OCA\GroupFolders\Versions\GroupVersionsExpireManager->expireFolder() at /var/www/nextcloud/apps/groupfolders/lib/Versions/GroupVersionsExpireManager.php:59
 OCA\GroupFolders\Versions\GroupVersionsExpireManager->expireAll() at /var/www/nextcloud/apps/groupfolders/lib/Command/ExpireGroup/ExpireGroupVersions.php:66
 OCA\GroupFolders\Command\ExpireGroup\ExpireGroupVersions->execute() at /var/www/nextcloud/apps/groupfolders/lib/Command/ExpireGroup/ExpireGroupVersionsTrash.php:54
 OCA\GroupFolders\Command\ExpireGroup\ExpireGroupVersionsTrash->execute() at /var/www/nextcloud/3rdparty/symfony/console/Command/Command.php:298
 Symfony\Component\Console\Command\Command->run() at /var/www/nextcloud/core/Command/Base.php:177
 OC\Core\Command\Base->run() at /var/www/nextcloud/3rdparty/symfony/console/Application.php:1040
 Symfony\Component\Console\Application->doRunCommand() at /var/www/nextcloud/3rdparty/symfony/console/Application.php:301
 Symfony\Component\Console\Application->doRun() at /var/www/nextcloud/3rdparty/symfony/console/Application.php:171
 Symfony\Component\Console\Application->run() at /var/www/nextcloud/lib/private/Console/Application.php:206
 OC\Console\Application->run() at /var/www/nextcloud/console.php:100
 require_once() at /var/www/nextcloud/occ:11

In Exception.php line 28:

  [Doctrine\DBAL\Driver\PDO\Exception (1062)]                                                                                
  SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry '18854-1689327178' for key 'gf_versions_uniq_index'  

Exception trace:
  at /var/www/nextcloud/3rdparty/doctrine/dbal/src/Driver/PDO/Exception.php:28
 Doctrine\DBAL\Driver\PDO\Exception::new() at /var/www/nextcloud/3rdparty/doctrine/dbal/src/Driver/PDO/Statement.php:132
 Doctrine\DBAL\Driver\PDO\Statement->execute() at /var/www/nextcloud/3rdparty/doctrine/dbal/src/Connection.php:1202
 Doctrine\DBAL\Connection->executeStatement() at /var/www/nextcloud/lib/private/DB/Connection.php:294
 OC\DB\Connection->executeStatement() at /var/www/nextcloud/3rdparty/doctrine/dbal/src/Query/QueryBuilder.php:386
 Doctrine\DBAL\Query\QueryBuilder->execute() at /var/www/nextcloud/lib/private/DB/QueryBuilder/QueryBuilder.php:280
 OC\DB\QueryBuilder\QueryBuilder->execute() at /var/www/nextcloud/lib/private/DB/QueryBuilder/QueryBuilder.php:326
 OC\DB\QueryBuilder\QueryBuilder->executeStatement() at /var/www/nextcloud/lib/public/AppFramework/Db/QBMapper.php:137
 OCP\AppFramework\Db\QBMapper->insert() at /var/www/nextcloud/apps/groupfolders/lib/Versions/VersionsBackend_OLD.php:88
 OCA\GroupFolders\Versions\VersionsBackend->getVersionsForFile() at /var/www/nextcloud/apps/groupfolders/lib/Versions/GroupVersionsExpireManager.php:72
 OCA\GroupFolders\Versions\GroupVersionsExpireManager->expireFolder() at /var/www/nextcloud/apps/groupfolders/lib/Versions/GroupVersionsExpireManager.php:59
 OCA\GroupFolders\Versions\GroupVersionsExpireManager->expireAll() at /var/www/nextcloud/apps/groupfolders/lib/Command/ExpireGroup/ExpireGroupVersions.php:66
 OCA\GroupFolders\Command\ExpireGroup\ExpireGroupVersions->execute() at /var/www/nextcloud/apps/groupfolders/lib/Command/ExpireGroup/ExpireGroupVersionsTrash.php:54
 OCA\GroupFolders\Command\ExpireGroup\ExpireGroupVersionsTrash->execute() at /var/www/nextcloud/3rdparty/symfony/console/Command/Command.php:298
 Symfony\Component\Console\Command\Command->run() at /var/www/nextcloud/core/Command/Base.php:177
 OC\Core\Command\Base->run() at /var/www/nextcloud/3rdparty/symfony/console/Application.php:1040
 Symfony\Component\Console\Application->doRunCommand() at /var/www/nextcloud/3rdparty/symfony/console/Application.php:301
 Symfony\Component\Console\Application->doRun() at /var/www/nextcloud/3rdparty/symfony/console/Application.php:171
 Symfony\Component\Console\Application->run() at /var/www/nextcloud/lib/private/Console/Application.php:206
 OC\Console\Application->run() at /var/www/nextcloud/console.php:100
 require_once() at /var/www/nextcloud/occ:11

In Statement.php line 130:

  [PDOException (23000)]                                                                                                     
  SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry '18854-1689327178' for key 'gf_versions_uniq_index'  

Exception trace:
  at /var/www/nextcloud/3rdparty/doctrine/dbal/src/Driver/PDO/Statement.php:130
 PDOStatement->execute() at /var/www/nextcloud/3rdparty/doctrine/dbal/src/Driver/PDO/Statement.php:130
 Doctrine\DBAL\Driver\PDO\Statement->execute() at /var/www/nextcloud/3rdparty/doctrine/dbal/src/Connection.php:1202
 Doctrine\DBAL\Connection->executeStatement() at /var/www/nextcloud/lib/private/DB/Connection.php:294
 OC\DB\Connection->executeStatement() at /var/www/nextcloud/3rdparty/doctrine/dbal/src/Query/QueryBuilder.php:386
 Doctrine\DBAL\Query\QueryBuilder->execute() at /var/www/nextcloud/lib/private/DB/QueryBuilder/QueryBuilder.php:280
 OC\DB\QueryBuilder\QueryBuilder->execute() at /var/www/nextcloud/lib/private/DB/QueryBuilder/QueryBuilder.php:326
 OC\DB\QueryBuilder\QueryBuilder->executeStatement() at /var/www/nextcloud/lib/public/AppFramework/Db/QBMapper.php:137
 OCP\AppFramework\Db\QBMapper->insert() at /var/www/nextcloud/apps/groupfolders/lib/Versions/VersionsBackend_OLD.php:88
 OCA\GroupFolders\Versions\VersionsBackend->getVersionsForFile() at /var/www/nextcloud/apps/groupfolders/lib/Versions/GroupVersionsExpireManager.php:72
 OCA\GroupFolders\Versions\GroupVersionsExpireManager->expireFolder() at /var/www/nextcloud/apps/groupfolders/lib/Versions/GroupVersionsExpireManager.php:59
 OCA\GroupFolders\Versions\GroupVersionsExpireManager->expireAll() at /var/www/nextcloud/apps/groupfolders/lib/Command/ExpireGroup/ExpireGroupVersions.php:66
 OCA\GroupFolders\Command\ExpireGroup\ExpireGroupVersions->execute() at /var/www/nextcloud/apps/groupfolders/lib/Command/ExpireGroup/ExpireGroupVersionsTrash.php:54
 OCA\GroupFolders\Command\ExpireGroup\ExpireGroupVersionsTrash->execute() at /var/www/nextcloud/3rdparty/symfony/console/Command/Command.php:298
 Symfony\Component\Console\Command\Command->run() at /var/www/nextcloud/core/Command/Base.php:177
 OC\Core\Command\Base->run() at /var/www/nextcloud/3rdparty/symfony/console/Application.php:1040
 Symfony\Component\Console\Application->doRunCommand() at /var/www/nextcloud/3rdparty/symfony/console/Application.php:301
 Symfony\Component\Console\Application->doRun() at /var/www/nextcloud/3rdparty/symfony/console/Application.php:171
 Symfony\Component\Console\Application->run() at /var/www/nextcloud/lib/private/Console/Application.php:206
 OC\Console\Application->run() at /var/www/nextcloud/console.php:100
 require_once() at /var/www/nextcloud/occ:11

groupfolders:expire [--output [OUTPUT]]

And here with the new file: 

Expiring version in '***'

In DbalException.php line 71:

  [OC\DB\Exceptions\DbalException (1062)]                                                                                                            
  An exception occurred while executing a query: SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry '18854-1689327184' for key '  
  gf_versions_uniq_index'                                                                                                                            

Exception trace:
  at /var/www/nextcloud/lib/private/DB/Exceptions/DbalException.php:71
 OC\DB\Exceptions\DbalException::wrap() at /var/www/nextcloud/lib/private/DB/QueryBuilder/QueryBuilder.php:328
 OC\DB\QueryBuilder\QueryBuilder->executeStatement() at /var/www/nextcloud/lib/public/AppFramework/Db/QBMapper.php:137
 OCP\AppFramework\Db\QBMapper->insert() at /var/www/nextcloud/apps/groupfolders/lib/Versions/VersionsBackend_NEW.php:122
 OCA\GroupFolders\Versions\VersionsBackend->getVersionsForFile() at /var/www/nextcloud/apps/groupfolders/lib/Versions/GroupVersionsExpireManager.php:72
 OCA\GroupFolders\Versions\GroupVersionsExpireManager->expireFolder() at /var/www/nextcloud/apps/groupfolders/lib/Versions/GroupVersionsExpireManager.php:59
 OCA\GroupFolders\Versions\GroupVersionsExpireManager->expireAll() at /var/www/nextcloud/apps/groupfolders/lib/Command/ExpireGroup/ExpireGroupVersions.php:66
 OCA\GroupFolders\Command\ExpireGroup\ExpireGroupVersions->execute() at /var/www/nextcloud/apps/groupfolders/lib/Command/ExpireGroup/ExpireGroupVersionsTrash.php:54
 OCA\GroupFolders\Command\ExpireGroup\ExpireGroupVersionsTrash->execute() at /var/www/nextcloud/3rdparty/symfony/console/Command/Command.php:298
 Symfony\Component\Console\Command\Command->run() at /var/www/nextcloud/core/Command/Base.php:177
 OC\Core\Command\Base->run() at /var/www/nextcloud/3rdparty/symfony/console/Application.php:1040
 Symfony\Component\Console\Application->doRunCommand() at /var/www/nextcloud/3rdparty/symfony/console/Application.php:301
 Symfony\Component\Console\Application->doRun() at /var/www/nextcloud/3rdparty/symfony/console/Application.php:171
 Symfony\Component\Console\Application->run() at /var/www/nextcloud/lib/private/Console/Application.php:206
 OC\Console\Application->run() at /var/www/nextcloud/console.php:100
 require_once() at /var/www/nextcloud/occ:11

In ExceptionConverter.php line 62:

  [Doctrine\DBAL\Exception\UniqueConstraintViolationException (1062)]                                                                                
  An exception occurred while executing a query: SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry '18854-1689327184' for key '  
  gf_versions_uniq_index'                                                                                                                            

Exception trace:
  at /var/www/nextcloud/3rdparty/doctrine/dbal/src/Driver/API/MySQL/ExceptionConverter.php:62
 Doctrine\DBAL\Driver\API\MySQL\ExceptionConverter->convert() at /var/www/nextcloud/3rdparty/doctrine/dbal/src/Connection.php:1938
 Doctrine\DBAL\Connection->handleDriverException() at /var/www/nextcloud/3rdparty/doctrine/dbal/src/Connection.php:1880
 Doctrine\DBAL\Connection->convertExceptionDuringQuery() at /var/www/nextcloud/3rdparty/doctrine/dbal/src/Connection.php:1208
 Doctrine\DBAL\Connection->executeStatement() at /var/www/nextcloud/lib/private/DB/Connection.php:294
 OC\DB\Connection->executeStatement() at /var/www/nextcloud/3rdparty/doctrine/dbal/src/Query/QueryBuilder.php:386
 Doctrine\DBAL\Query\QueryBuilder->execute() at /var/www/nextcloud/lib/private/DB/QueryBuilder/QueryBuilder.php:280
 OC\DB\QueryBuilder\QueryBuilder->execute() at /var/www/nextcloud/lib/private/DB/QueryBuilder/QueryBuilder.php:326
 OC\DB\QueryBuilder\QueryBuilder->executeStatement() at /var/www/nextcloud/lib/public/AppFramework/Db/QBMapper.php:137
 OCP\AppFramework\Db\QBMapper->insert() at /var/www/nextcloud/apps/groupfolders/lib/Versions/VersionsBackend_NEW.php:122
 OCA\GroupFolders\Versions\VersionsBackend->getVersionsForFile() at /var/www/nextcloud/apps/groupfolders/lib/Versions/GroupVersionsExpireManager.php:72
 OCA\GroupFolders\Versions\GroupVersionsExpireManager->expireFolder() at /var/www/nextcloud/apps/groupfolders/lib/Versions/GroupVersionsExpireManager.php:59
 OCA\GroupFolders\Versions\GroupVersionsExpireManager->expireAll() at /var/www/nextcloud/apps/groupfolders/lib/Command/ExpireGroup/ExpireGroupVersions.php:66
 OCA\GroupFolders\Command\ExpireGroup\ExpireGroupVersions->execute() at /var/www/nextcloud/apps/groupfolders/lib/Command/ExpireGroup/ExpireGroupVersionsTrash.php:54
 OCA\GroupFolders\Command\ExpireGroup\ExpireGroupVersionsTrash->execute() at /var/www/nextcloud/3rdparty/symfony/console/Command/Command.php:298
 Symfony\Component\Console\Command\Command->run() at /var/www/nextcloud/core/Command/Base.php:177
 OC\Core\Command\Base->run() at /var/www/nextcloud/3rdparty/symfony/console/Application.php:1040
 Symfony\Component\Console\Application->doRunCommand() at /var/www/nextcloud/3rdparty/symfony/console/Application.php:301
 Symfony\Component\Console\Application->doRun() at /var/www/nextcloud/3rdparty/symfony/console/Application.php:171
 Symfony\Component\Console\Application->run() at /var/www/nextcloud/lib/private/Console/Application.php:206
 OC\Console\Application->run() at /var/www/nextcloud/console.php:100
 require_once() at /var/www/nextcloud/occ:11

In Exception.php line 28:

  [Doctrine\DBAL\Driver\PDO\Exception (1062)]                                                                                
  SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry '18854-1689327184' for key 'gf_versions_uniq_index'  

Exception trace:
  at /var/www/nextcloud/3rdparty/doctrine/dbal/src/Driver/PDO/Exception.php:28
 Doctrine\DBAL\Driver\PDO\Exception::new() at /var/www/nextcloud/3rdparty/doctrine/dbal/src/Driver/PDO/Statement.php:132
 Doctrine\DBAL\Driver\PDO\Statement->execute() at /var/www/nextcloud/3rdparty/doctrine/dbal/src/Connection.php:1202
 Doctrine\DBAL\Connection->executeStatement() at /var/www/nextcloud/lib/private/DB/Connection.php:294
 OC\DB\Connection->executeStatement() at /var/www/nextcloud/3rdparty/doctrine/dbal/src/Query/QueryBuilder.php:386
 Doctrine\DBAL\Query\QueryBuilder->execute() at /var/www/nextcloud/lib/private/DB/QueryBuilder/QueryBuilder.php:280
 OC\DB\QueryBuilder\QueryBuilder->execute() at /var/www/nextcloud/lib/private/DB/QueryBuilder/QueryBuilder.php:326
 OC\DB\QueryBuilder\QueryBuilder->executeStatement() at /var/www/nextcloud/lib/public/AppFramework/Db/QBMapper.php:137
 OCP\AppFramework\Db\QBMapper->insert() at /var/www/nextcloud/apps/groupfolders/lib/Versions/VersionsBackend_NEW.php:122
 OCA\GroupFolders\Versions\VersionsBackend->getVersionsForFile() at /var/www/nextcloud/apps/groupfolders/lib/Versions/GroupVersionsExpireManager.php:72
 OCA\GroupFolders\Versions\GroupVersionsExpireManager->expireFolder() at /var/www/nextcloud/apps/groupfolders/lib/Versions/GroupVersionsExpireManager.php:59
 OCA\GroupFolders\Versions\GroupVersionsExpireManager->expireAll() at /var/www/nextcloud/apps/groupfolders/lib/Command/ExpireGroup/ExpireGroupVersions.php:66
 OCA\GroupFolders\Command\ExpireGroup\ExpireGroupVersions->execute() at /var/www/nextcloud/apps/groupfolders/lib/Command/ExpireGroup/ExpireGroupVersionsTrash.php:54
 OCA\GroupFolders\Command\ExpireGroup\ExpireGroupVersionsTrash->execute() at /var/www/nextcloud/3rdparty/symfony/console/Command/Command.php:298
 Symfony\Component\Console\Command\Command->run() at /var/www/nextcloud/core/Command/Base.php:177
 OC\Core\Command\Base->run() at /var/www/nextcloud/3rdparty/symfony/console/Application.php:1040
 Symfony\Component\Console\Application->doRunCommand() at /var/www/nextcloud/3rdparty/symfony/console/Application.php:301
 Symfony\Component\Console\Application->doRun() at /var/www/nextcloud/3rdparty/symfony/console/Application.php:171
 Symfony\Component\Console\Application->run() at /var/www/nextcloud/lib/private/Console/Application.php:206
 OC\Console\Application->run() at /var/www/nextcloud/console.php:100
 require_once() at /var/www/nextcloud/occ:11

In Statement.php line 130:

  [PDOException (23000)]                                                                                                     
  SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry '18854-1689327184' for key 'gf_versions_uniq_index'  

Exception trace:
  at /var/www/nextcloud/3rdparty/doctrine/dbal/src/Driver/PDO/Statement.php:130
 PDOStatement->execute() at /var/www/nextcloud/3rdparty/doctrine/dbal/src/Driver/PDO/Statement.php:130
 Doctrine\DBAL\Driver\PDO\Statement->execute() at /var/www/nextcloud/3rdparty/doctrine/dbal/src/Connection.php:1202
 Doctrine\DBAL\Connection->executeStatement() at /var/www/nextcloud/lib/private/DB/Connection.php:294
 OC\DB\Connection->executeStatement() at /var/www/nextcloud/3rdparty/doctrine/dbal/src/Query/QueryBuilder.php:386
 Doctrine\DBAL\Query\QueryBuilder->execute() at /var/www/nextcloud/lib/private/DB/QueryBuilder/QueryBuilder.php:280
 OC\DB\QueryBuilder\QueryBuilder->execute() at /var/www/nextcloud/lib/private/DB/QueryBuilder/QueryBuilder.php:326
 OC\DB\QueryBuilder\QueryBuilder->executeStatement() at /var/www/nextcloud/lib/public/AppFramework/Db/QBMapper.php:137
 OCP\AppFramework\Db\QBMapper->insert() at /var/www/nextcloud/apps/groupfolders/lib/Versions/VersionsBackend_NEW.php:122
 OCA\GroupFolders\Versions\VersionsBackend->getVersionsForFile() at /var/www/nextcloud/apps/groupfolders/lib/Versions/GroupVersionsExpireManager.php:72
 OCA\GroupFolders\Versions\GroupVersionsExpireManager->expireFolder() at /var/www/nextcloud/apps/groupfolders/lib/Versions/GroupVersionsExpireManager.php:59
 OCA\GroupFolders\Versions\GroupVersionsExpireManager->expireAll() at /var/www/nextcloud/apps/groupfolders/lib/Command/ExpireGroup/ExpireGroupVersions.php:66
 OCA\GroupFolders\Command\ExpireGroup\ExpireGroupVersions->execute() at /var/www/nextcloud/apps/groupfolders/lib/Command/ExpireGroup/ExpireGroupVersionsTrash.php:54
 OCA\GroupFolders\Command\ExpireGroup\ExpireGroupVersionsTrash->execute() at /var/www/nextcloud/3rdparty/symfony/console/Command/Command.php:298
 Symfony\Component\Console\Command\Command->run() at /var/www/nextcloud/core/Command/Base.php:177
 OC\Core\Command\Base->run() at /var/www/nextcloud/3rdparty/symfony/console/Application.php:1040
 Symfony\Component\Console\Application->doRunCommand() at /var/www/nextcloud/3rdparty/symfony/console/Application.php:301
 Symfony\Component\Console\Application->doRun() at /var/www/nextcloud/3rdparty/symfony/console/Application.php:171
 Symfony\Component\Console\Application->run() at /var/www/nextcloud/lib/private/Console/Application.php:206
 OC\Console\Application->run() at /var/www/nextcloud/console.php:100
 require_once() at /var/www/nextcloud/occ:11

groupfolders:expire [--output [OUTPUT]]
VVD commented 9 months ago

I fixed errors using steps:

  1. sudo -u www php occ groupfolders:expire
  2. select * from oc_filecache where fileid = ID;
  3. in web client: rename this file to "new name"
  4. in web client: copy this file to "old name"
  5. in web client: remove file "new name"
  6. Repeat from step 1.
vasyugan commented 9 months ago

2. select * from oc_filecache where fileid = ID;

MariaDB [nextcloud]> select * from oc_filecache where 'fileid' = 'ID';
Empty set (0,001 sec)
bobab12 commented 9 months ago

In my case sudo -u www php occ groupfolders:expire fails with the following errors:

In DbalException.php line 71:

  An exception occurred while executing a query: SQLSTATE[23505]: Unique violation: 7 ERROR:  duplicate key value violates unique constra  
  int "gf_versions_uniq_index"                                                                                                             
  DETAIL:  Key (file_id, "timestamp")=(3306056, 1658591737) already exists.                                                                

In ExceptionConverter.php line 53:

  An exception occurred while executing a query: SQLSTATE[23505]: Unique violation: 7 ERROR:  duplicate key value violates unique constra  
  int "gf_versions_uniq_index"                                                                                                             
  DETAIL:  Key (file_id, "timestamp")=(3306056, 1658591737) already exists.                                                                

In Exception.php line 28:

  SQLSTATE[23505]: Unique violation: 7 ERROR:  duplicate key value violates unique constraint "gf_versions_uniq_index"  
  DETAIL:  Key (file_id, "timestamp")=(3306056, 1658591737) already exists.                                             

In Statement.php line 130:

  SQLSTATE[23505]: Unique violation: 7 ERROR:  duplicate key value violates unique constraint "gf_versions_uniq_index"  
  DETAIL:  Key (file_id, "timestamp")=(3306056, 1658591737) already exists.

The file_id in this instance is 3306056, so the query would become:

select * from oc_filecache where fileid = 3306056;

This query should return a record, which includes a path that can be cross-referenced in the web client. However, in this specific case, the path appears to be linked to a deleted file: __groupfolders/trash/1/DSCPDC_0001_BURST20220427083413292_COVER.JPG.d1658591829.

I'm uncertain about the appropriate steps to take moving forward.

VVD commented 9 months ago
  1. select * from oc_filecache where fileid = ID;
MariaDB [nextcloud]> select * from oc_filecache where 'fileid' = 'ID';
Empty set (0,001 sec)

ID from error message from 1. For example here:

In DbalException.php line 71:

  An exception occurred while executing a query: SQLSTATE[23505]: Unique violation: 7 ERROR:  duplicate key value violates unique constra  
  int "gf_versions_uniq_index"                                                                                                             
  DETAIL:  Key (file_id, "timestamp")=(3306056, 1658591737) already exists.

ID = 3306056

ernolf commented 9 months ago
  1. select * from oc_filecache where fileid = ID;
MariaDB [nextcloud]> select * from oc_filecache where 'fileid' = 'ID';
Empty set (0,001 sec)

You should of coures replace ID with the fileid in question!

Here an example:

This is the messsage: SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry '8826002-1702996424' for key 'gf_versions_uniq_index'

In other words:

--> If you are curious about unix time <-- This is how to convert unixtime to human readable: Paste this function in your terminal: ```sh u2d(){ date -d "@${1}" "+%Y-%m-%d %H:%M:%S";} ``` then simply enter `u2d $unixtime_string`, so in my example that results in 2023-12-19 15:33:44

So I checked fileid 8826002:

MariaDB [nextcloud]> SELECT path FROM oc_filecache WHERE fileid = 8826002;
+-----------------------------------------------------+
| path                                                |
+-----------------------------------------------------+
| __groupfolders/trash/9/Spring_83_Em.mp3.d1704291095 |
+-----------------------------------------------------+
1 row in set (0,002 sec)

This query provides the information that the file is located in the trashbin from groupfolder 9. So I emptied trashbin for groupfolder 9:

occ groupfolders:trashbin:cleanup 9
Are you sure you want to empty the trashbin of your group folder with id 9, this can not be undone (y/N).y

then:

sudo -u www-data php -f /var/www/nextcloud/occ groupfolders:expire

That procedure did fix it for me in all the cases where this message appeared

Much luck, ernolf

bobab12 commented 9 months ago

This query provides the information that the file is located in the trashbin from groupfolder 9. So I emptied trashbin for groupfolder 9:

occ groupfolders:trashbin:cleanup 9
Are you sure you want to empty the trashbin of your group folder with id 9, this can not be undone (y/N).y

then:

sudo -u www-data php -f /var/www/nextcloud/occ groupfolders:expire

That fixed it for me as well. :+1:

vidlb commented 9 months ago

I can confirm this duplicate key error with NC 27.1.5

I was able to fix the problem using @ernolf solution: manually removed the only problematic file (from user folder then trash) - using occ so the db / index can be updated. Thanks !

Chartman123 commented 9 months ago

Same here, let's see if the error returns. Otherwise I think we can close this issue with the described workaround as a solution.

lukasrad02 commented 9 months ago

Same here, let's see if the error returns. Otherwise I think we can close this issue with the described workaround as a solution.

I don't think so, since the workaround only helps to get rid of the error once it appeared. However, in my experience, the duplicate keys come back for other files after some time, so the issue should not be closed before the underlying cause has been fixed.

vidlb commented 9 months ago

True, we need to identify the root of this issue.
I guess something is wrong in the manner file ID / index is managed for both ncdata root files and groupfolders files.
If 2 conflicting index exists, the duplicate key may randomly appear because versions has constraint unique index for the file ref, but a version may already exists for another file - with ID from a different sequence.

Edit: trying to fix my terrible english

vidlb commented 9 months ago

So it didn't take long for the key error to appear again, probably because most of our folders are groupfolders.
This time because of a file moved in the recycle bin.

alphasixtyfive commented 9 months ago

Basically both Deleted files and File versions produce that error.

artonge commented 9 months ago

Hello, can one of you give me some reproduction steps? Not only running occ groupfolders:expire, but how to be in a state where running this command causes an error?

vidlb commented 9 months ago

Here it is @artonge

  1. Remove any file in a group folder (I tried one with no existing versions)
  2. Run occ groupfolders:expire => key error
  3. Error is gone if file is restored, but then versioning seems broken for that file

Edit : not any file, cannot reproduce it with another one randomly picked, but this may be due to the nature of the bug (see my comment above from 2 days ago)

vidlb commented 9 months ago

@artonge I was able to reproduce using a PNG : create a new version, delete the file, then run occ groupfolders:expire .
It seems the error occurs for file that have versions (or had, for the case above versions were removed two days ago when the expire command ran successfully)

DB logs :

2024-01-08 16:56:53.073 CET [54561] ERROR:  duplicate key value violates unique constraint "gf_versions_uniq_index"
2024-01-08 16:56:53.073 CET [54561] DETAIL:  Key (file_id, "timestamp")=(491166, 1700579665) already exists.
2024-01-08 16:56:53.073 CET [54561] STATEMENT:  INSERT INTO "oc_group_folders_versions" ("file_id", "timestamp", "size", "mimetype", "metadata") VALUES($1, $2, $3, $4, $5)

It's worth mentioning that the first time I saw this error, the problematic file was not in recycle bin

vidlb commented 9 months ago

I can confirm versioning is broken after restoring the file. May be linked to #2718 ?
Also just tried to create a new .md file and versioning does not work. No error in the logs reader (but I'm not in debug).
It seems to work only for files indexed prior to last update.

Even though it never had any (visible) versions, that .md file will provoke the key error during expire if deleted

vidlb commented 9 months ago

I don't understand how I could create a version via upload for the PNG : now it fails for another file created 2 months ago.
DB logs : 

2024-01-08 17:39:35.444 CET [57672] ERROR:  duplicate key value violates unique constraint "oc_filecache_extended_pkey"
2024-01-08 17:39:35.444 CET [57672] DETAIL:  Key (fileid)=(489291) already exists.
2024-01-08 17:39:35.444 CET [57672] STATEMENT:  INSERT INTO "oc_filecache_extended" ("fileid", "upload_time") VALUES($1, $2)
Marcus1Pierce commented 9 months ago
  1. select * from oc_filecache where fileid = ID;
MariaDB [nextcloud]> select * from oc_filecache where 'fileid' = 'ID';
Empty set (0,001 sec)

You should of coures replace ID with the fileid in question!

Here an example:

This is the messsage: SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry '8826002-1702996424' for key 'gf_versions_uniq_index'

In other words:

  • this is the file_id it is complaining about: 8826002
  • this is the timestamp in unixtime: 1702996424

--> If you are curious about unix time <-- So I checked fileid 8826002:

MariaDB [nextcloud]> SELECT path FROM oc_filecache WHERE fileid = 8826002;
+-----------------------------------------------------+
| path                                                |
+-----------------------------------------------------+
| __groupfolders/trash/9/Spring_83_Em.mp3.d1704291095 |
+-----------------------------------------------------+
1 row in set (0,002 sec)

This query provides the information that the file is located in the trashbin from groupfolder 9. So I emptied trashbin for groupfolder 9:

occ groupfolders:trashbin:cleanup 9
Are you sure you want to empty the trashbin of your group folder with id 9, this can not be undone (y/N).y

then:

sudo -u www-data php -f /var/www/nextcloud/occ groupfolders:expire

That procedure did fix it for me in all the cases where this message appeared

Much luck, ernolf

Thanks for your share. This comment fixed for me.

heisian commented 9 months ago

I'm unsure if this is the right place, but this is the error I see in my logs:

DbalException An exception occurred while executing a query: SQLSTATE[23505]:
Unique violation: 7 ERROR: duplicate key value violates unique constraint
"gf_versions_uniq_index" DETAIL: Key (file_id, "timestamp")=(208235, 1704399905) already exists.
Error while running background job (class: OCA\GroupFolders\BackgroundJob\ExpireGroupVersions, arguments: )

Any insight on how to address this (or if it's OK to ignore) would be great! I don't currently see any issues with accessing versioned files in my group folders.

vidlb commented 9 months ago

Any insight on how to address this (or if it's OK to ignore) would be great! I don't currently see any issues with accessing versioned files in my group folders.

You'll find out by reading the last 20 comments in this thread !
Spoiler alert : error will come back

It's kind of okay to ignore but keep in mind versions for groupfolders is currently broken

Dennis1993 commented 9 months ago

Any insight on how to address this (or if it's OK to ignore) would be great! I don't currently see any issues with accessing versioned files in my group folders.

You'll find that by reading the last 10 comments in this thread! Spoiler alert : error will come back

It's kind of okay to ignore but keep in mind versions for groupfolders is currently broken

Since 17. September 2023….. 🙄

vidlb commented 9 months ago

Since 17. September 2023….. 🙄

In fact it was different back then : new versions were created, but I was unable to restore (had to download one and overwrite)

heisian commented 9 months ago

Any insight on how to address this (or if it's OK to ignore) would be great! I don't currently see any issues with accessing versioned files in my group folders.

You'll find out by reading the last 20 comments in this thread ! Spoiler alert : error will come back

It's kind of okay to ignore but keep in mind versions for groupfolders is currently broken

@vidlb Admittedly I skimmed through the history, so I apologize, but thank you for giving me the summary, and also for creating this app!

As much as I could gather it seems like versioning is OK until you delete the file and restore it, at which point things break down? It seems I can download two different versions of a file (without performing any delete/restore operations) and I get the correct files.

For now I'm OK with versioning being broken as I'm taking ZFS snapshots hourly.

Cheers and thanks again for your work as Group Folders is IMO pretty vital to NextCloud and my switch away from other file sync platforms.

vidlb commented 9 months ago

@heisian obviously I'm not a nextcloud dev, I'd like to but I don't know PHP (I hate it), but yes we should thank them and avoid mean comments as there are some in this issue.
Group folders is indeed critical for NC to be complete as a file sharing solution.
I hope the problem is not too deep and that it will be resolved soon !

P.S. : +1 for ZFS ^^ it is awesome

heisian commented 9 months ago

@vidlb oops, I saw the Author tag but didn't realize it only applies to the Issue. Yes I think folks have a way of .. being demanding, especially when we are hot off of experiencing a frustrating bug or experience with the software.. hopefully the devs know they are much appreciated :)

pReya commented 9 months ago

I know this is off topic, but I really don't know where else to put this: Obviously it's not a good idea to be snappy about this or throw negativity towards the people trying to fix this. Thanks everyone who is working on this, recreating it or adding more info the issue, so it can hopefully be resolved soon ❤️ .

BUT there is a proper big company (Nextcloud GmbH) behind this project, which should manage and prioritize stuff and have an interest in keeping their product bug free. The fact, that one of the most used "official" apps for one of the core services of Nextcloud (share files in a group) has a major bug that's open for nearly four months is just no sign for good management. It feels like groupfolders is an afterthought next to some new AI or Talk features.

le-patenteux commented 9 months ago

I know this is off topic, but I really don't know where else to put this: Obviously it's not a good idea to be snappy about this or throw negativity towards the people trying to fix this. Thanks everyone who is working on this, recreating it or adding more info the issue, so it can hopefully be resolved soon ❤️ .

BUT there is a proper big company (Nextcloud GmbH) behind this project, which should manage and prioritize stuff and have an interest in keeping their product bug free. The fact, that one of the most used "official" apps for one of the core services of Nextcloud (share files in a group) has a major bug that's open for nearly four months is just no sign for good management. It feels like groupfolders is an afterthought next to some new AI or Talk features.

I agree with your take (and the fact that it is off-topic too!) Nextcloud should never have jumped to 28.x.x with major bugs in 27.x.x still not resolved (not talking about missing features, but function-breaking bugs!)

I don't understand the direction of this project TBH

EDIT: Not trying to diminish the hard work of the people building this... I just feel a general lack of direction for the entire project. But it is still an amazing project!

Chartman123 commented 9 months ago

Please stop off-topic comments. For general discussions like this one head over to the forums on https://help.nextcloud.com

Thank you very much!

fschrempf commented 9 months ago

@pReya @Normand-Nadon You're not alone with these thoughts. And this is not the first time things like this happen. The problems you describe are already a constant and well established pattern. You might want to read through the following threads for more information. Personally I would advise anyone to not use Nextcloud + Groupfolders anymore.

rakekniven commented 9 months ago

but I really don't know where else to put this

Put it in your cellar. You ranted at Mastodon, now you start here and kill motivation of volunteers like me. And please do not visit help.nextcloud.com to start politics. I recommend X ;-)

ostasevych commented 9 months ago

I can confirm this duplicate key error with NC 27.1.5

I was able to fix the problem using @ernolf solution: manually removed the only problematic file (from user folder then trash) - using occ so the db / index can be updated. Thanks !

I've found easier way: if you have to fix a lot of files in one folder, you may just rename that folder, expire and rename again. If a file in the trashbin has gone, and there is still the record in the filecache on it, just MySQL delete that record and expire again.