performance considerations - many shares write of activites

[kinimodmeyer]

Steps to reproduce

1. create a group "test" with ~100 users (use a slower setup like hdd-database / ~16 users starting already to get recognisable for e.g. creating a comment in a deck-card)
2. share a entity like a "folder" or a "deck board" with this group
3. every save action in the frontend on this enties are now delayed and the frontend is not efficiently usable.

Actual behaviour

For me it seems that on every save action 100 entries needs to get calculated and written to the activity table. It seems that this is happening blocking on the frontend calls and not done afterwards/in the background.

Expected behaviour

Could this be done afterwards by let the frontend call finishing fast and store only one entry there which then get populated by a follow up non-blocking ghost process? could the maybe existing redis be used as a queue?

To keep that in perspective, our activity table has some million entries and it is blazing fast in showing/selecting the activities. We only have this issue for the writes and only if we have many activites on a specific operation (cause of many shares).

We are wondering if we are the first who have/had this issue. We are really open for tipps (without disable the app or make smaller shares)

[putt1ck]

We have installs with larger groups where I cannot see any performance issue with group folders shared with those groups, but concur that Deck does have a small delay where shared with a large group (in this case >250 members). The delay is immediately removed once the board is no longer shared with the group. A quick look at servers shows a small spike of load on the front end server (Apache) and big spike on the DB server (PostgreSQL). This install in question is v20.

[kinimodmeyer]

Thanks for your answer!

Could it be that you mean with "group folders" the folders generated by the group-folder app? We are not using the "group folder app". We are using the normal user folders which are shared with groups. I´m not sure if the activity app works a little bit different with group-folders app (i saw a option activity_use_cached_mountpoints regarding this, maybe this is not even set on your side?)

[juliusknorr]

So the way activities are currently stored we need to create one database row for each user who is able to see the activity, so from the deck app there is not much we can do there. I guess it also highly depends on the database performance here.

[kinimodmeyer]

i agree, we changed databases and its much better now. (only recognisable by more then 100 user) maybe the easiest speed improvements (besides running async) would be to group the inserts in one multi insert query (if that not already happening)

[pierreozoux]

I come here from the size perspective.

I have a reasonable instance, some 30 people, and the activity table, for a year takes more than 3GB, it is crazy for me. And yes, looking into it, it is writing one row for each user.

I'm not an expert, so maybe it is not relevant what I will say, but I'll try :) In the activity line, we have the resource id, and we already have the information of who can access which resource in other tables, so one row per activity seems possible to me, then even if it is blocking the frontend, it should be ok to write one row, and it would go easy on the database storage. (but maybe there are also better reasons to store one row for each user, I don't know)

My 2 cents :)