search

nextcloud / groupfolders

📁👩‍👩‍👧‍👦 Admin-configured folders shared by everyone in a group. https://github.com/nextcloud-releases/groupfolders
https://apps.nextcloud.com/apps/groupfolders
286 stars 87 forks source link

Group folders dramatically slows down my server. #2256

Open minamichail opened 1 year ago

minamichail commented 1 year ago

I have a large nextcloud setup (around 5TB) with most folders primarily group folders. Basically all the files and folders are shared across the company but depending on your group you get different permission per file. I have custom software that connects to nextcloud and using the api it sets the different permission per folders. After using that the setup for a few weeks nextcloud slowed down dramatically and we upgraded our server to a super server just to keep things moving. I did an sql analysis on the number of queries it makes just to get the root path and with group folders plugin enabled it makes around 12000 queries! vs. disabled it makes around 500. I also found one query that was extremely slow and took about 8 seconds to run, please see below. Are we doing something wrong in the setup or is group folders not made to handle so many files?

SELECT `file`.`fileid`, `storage`, `path`, `path_hash`, `file`.`parent`, `file`.`name`, `mimetype`, `mimepart`, `size`, 
`mtime`, `storage_mtime`, `encrypted`, `etag`, `permissions`, `checksum`, `unencrypted_size` 
FROM `oc_filecache` `file` 
WHERE (((`mimetype` <> 2) OR (`size` = 0)) AND (`mtime` > 1674421392)) AND (((`storage` = 2) 
AND ((`path` = 'files') OR (`path` LIKE 'files/%'))) OR ((`storage` = 68) 
AND ((`path_hash` = '570bcc05900b9eb600ba5e918520df78') OR (`path` LIKE '__groupfolders/461/%'))) OR ((`storage` = 68) 
AND ((`path_hash` = 'fbd7c214a3bd37098d045cf59bddc551') OR (`path` LIKE '__groupfolders/361/%'))) OR ((`storage` = 68) 
 AND ((`path_hash` = '959c98e248c3c3afefaeceaf803169d6') OR (`path` LIKE '__groupfolders/584/%'))) OR ((`storage` = 68)
 AND ((`path_hash` = '2b437574d7f2256ce0c8bffb1d3f7577') OR (`path` LIKE '__groupfolders/585/%'))) OR ((`storage` = 68) 
 AND ((`path_hash` = '0866597a092404891d9b01184f271c83') OR (`path` LIKE '__groupfolders/586/%'))) OR ((`storage` = 68) ... this keeps going for a while.
vithusel commented 1 year ago

Ive been having issues with postgres running out of connections. Checking the connections i can see close to 50 idle connections when the server is sat idle. I was hoping to do some dev work on this app but it seems its barely supported for such a crucial feature. I think its time i hopped off nextcloud and move to a NAS solution as i can't have clients servers going offline like this. I would try to fix the issue myself but im in no way skilled enough to do that.