Global and per group disk quota [$40]

[hoerup]

It would be nice if I can set a disk quota per group (eg tot total amount of disk space used by group may not exceed xx GB) and a global MAX usage to prevent the disk form filling completely.

[pschichtel]

Also: group based default quota values.

Example: member's of the admin group all have 10GB space by default, normal users only have 5GB by default.

[lcx]

also would love a group quota.

[ghost]

I also would love to be able to set storage limits for groups as a whole.

For example: Company A is limited to 30GBs Company A has 4 normal users and one group admin

The group admin should be able to assign storage limit to each of the groups normal users up to 30 GBs

Thank you!

[eroueen]

The request for group quotas is quite popular in the forums; see for example [1] and [2].

Is it doable? Something for the next mile stone? Would be nice to hear from the dev folks :-)

Thanks,

• eroueen

[1] https://forum.owncloud.org/viewtopic.php?f=23&t=9321&p=23057 [2] http://forum.owncloud.org/viewtopic.php?f=23&t=10279

[cchris-org]

yes .. how can it be when a groupadmin can overwrite the limits which is set by admin ?
please fix this because it is near impossible to provide a customer owncloud with his own groupadmin and limited space.

thanks, chris

[Wadera]

No one is assigned... No milestone... ;(

Push up.

[tanghus]

If nobody volunteers to code it or is willing to pay for someone to develop it, it won't get done. Hard facts.

[ghost]

Been waiting to see this get fixed. I view this as a bug rather than a feature request because the logic of quotas assignment should follow this.

System admin assigns storage quota to group admin (1GB) > group admin cannot change quota the system admin gave him > group admin can only give out to group users the limit of what the group admin was assigned by system admin(1GB).

[jamesjwarren]

It would be extremely useful to have the option of giving a group a set amount of storage that any of the users of that group can use. When the cloud service is used for teams, this quota structure makes more sense.

[kristofseyen]

i'm waiting for that feature to, i don't really understand the logic of a group subadmin who can overrule the group quota for a user. In my opinion the subadmin can only change the quota for a user in his group between 0 and the quota set by the admin.

[sggalan]

Let's talk about Folder's quota, instead of Group's quota. This is because if a user belongs to several groups, how does the system manage the users maximum quota? which group(s) do an uploaded file belong to, and which one it's size is added to?

So creating something like a 'Group Folder' or 'Project Folder' with it's own quota, might help on accounting Group used space, and making it independent from each individual user's quota (or not, but individual stored filesize can be calculated).

[sggalan]

As soon as I wrote my previous comment, I found a nice workaround:

Let's have some Users collaborating on a Project. I want the Project have a quota of 50GB and every single user have 2GB, but Users' quota must be independent of Project's quota.

Then just create your 2GB Users' accounts, and a new user called 'Project' (for instance) and give it a 50GB quota. But no user needs to login with it unless you may want to have a Project Administrator. Well, you must login at least one time: create one folder into Project's account using its login, and share it with the desired Users. If any User uploads a file into the shared folder, it's size is substracted from Project's quota, but not from User's quota, which is exactly what I needed. :dart: (Dropbox, for instance, substracts from quotas of any user accessing the shared folder)

Tested under ownCloud 6.0. Looking forward to install the sync clients :D Good Work!!

[jancborchardt]

As this has been a long requested feature I’m marking it for triage. cc @karlitschek @MTRichards @craigpg

[chrsch]

I would also love to see this feature. The workaround from sggalan is nice but a workaround.

[Kedrigern]

Default quota for user's in group would be realy nice.

[luizluca]

I created a new proposal on #14600. In summary, I suggest to have a way to define, in a shared folder, on whose quota a new file will count. Also, any user with write access to a shared folder could define how much of its own quota can be used with this shared folder.

This might provide the basis for any group/folder quota solution.

[Trefex]

:+1:

[crysman]

+1 for implementing default group user quota and also global default user quota

[securityopa]

This is a really needed feature for such a great script.

[niptic]

This feature is a really important one, would be nice to create a group global quota, whereas I create a group admin, and every user created under that uses the same quota shared, so each user will know when the admin will reach the max cap and will therefore notify the group admin, and in turn notify the admin.

[JelleGe]

A feature that I really need

[binglinoioij]

Hello everyone! I hava a doubt, it is that i use this command -> curl -X GET http://user:bitnami@192.168.2.2/ocs/v1.php/cloud/users/Frank , i always get the result like that

/* <?xml version="1.0"?>

100 ok frank@example.org 13743895347 true

*/ a user's quota always 13743895347(12.8GB), but when i login the owncloud with admin user by browser, i get that user's quota is 35GB . And when i login with Frank i also get 35GB. If the some properties i have to set? And the uversion what i use is 8.x. Thank you!

[PVince81]

Group quota is not implemented yet, so there is no possible workaround to make it work with the current codebase.

@binglinoioij your problem is unrelated with this ticket, please raise a new one and use the issue template: https://raw.githubusercontent.com/owncloud/core/master/issue_template.md

[soydemadrid]

Hi how do you vote or also request this feature? I would also like to vote for a Group quota. So when a Group Admin makes a new user he/she can choose their quota but when he does it takes a chunk of his quota away and shares it out. OR for each user he/she creates it evenly divides the overall group quota between all the users.

[pako81]

@MTRichards also required here (SF 00004664). Do we have any expectation in terms of when this will be available? Thanks.

[MTRichards]

No. we don't currently have any plans to set group quotas. This also gets messy because people can be members of more than one group, and is quota additive, zero sum where max wins or what. No clear right answer for all use cases.

However, I do intend to eventually update the group admin role when we can, likely 9.2 or later though. This will enhance how we allow group admins to work with their users.

[lgg]

@MTRichards maybe add setting for this?

If user is more than in 1 group: 

• [x] use min
• [ ] use max

[MTRichards]

Good idea!

On Jun 6, 2016, at 6:48 PM, Fyodor notifications@github.com wrote:

@MTRichards maybe add setting for this?

If user is more than in 1 group: use min use max — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[chrsch]

@MTRichards is there an overview for the use cases? I could contribute by proposing a concept how to work with group quotas.

[MatthewReale]

If it's not implemented as a standard feature it would be nice if someone made an admin app for it.

[hodyroff]

@chrsch No overview at this time :( You are welcome to start a good one. The start of this FR is kind of mixed as only users can own files currently a group quota makes no real sense. Provisioning API and LDAP can set quotas very intelligently and even on the fly based on usage, so in some ways this could already be done from the outside ...

[ShamimIslam]

I would write this. There is only one problem. I have yet to find:

1. Class diagram
2. Class function documentation of the actual API calls
3. How the lookups for the queries are performed
4. How the data is segmented across the classes
5. How we are supposed to add things
6. Where the hooks are
7. How the dataflow takes place
8. How the routing is defined for new apps
9. How the routing is applied for existing apps.

The process as I've figured it out, is very simple. The oc_preferences table contains all of the "disabled" and "quota" bits. To override it, we need to update where the quota is calculated. That's the easy part. Understanding what to do.

The hard part is identifying where it should be done. There is no clear documentation that I have been able to locate that clearly identified datapath or process path or even module execution path where this behavior needs to be injected.

Anyone have any ideas?

P.S. I was able to set the quota for all users of a given group to value using a simple query - could use it to create an app to set the quota for all users (which would solve the problem). But it wouldn't make it manageable. But it might be more predictable - what do you do when two groups have a quota?

[Powie]

To restrict disk space usage for the whole OC installation is very important!!!!

+1

[hodyroff]

@Powie Please help us understand why. What is your use case? And why can this not be achieved with file system utilities?

[ShamimIslam]

Why does someone voting need to justify a vote? Is a vote in itself insufficient?

Having said that, have you TRIED to use the so-called system tools?

Please tell me how you update a whole group of pre-existing historical users to a group quota? And now if you have 3000 users, and of those, you need to update 1500 of those?

If those commands exist, we're golden. If not, you haven't maintained an OC instance long enough.nd up

Sorry to sound crass or upset but I find it annoying that someone requests an upvote and instead of EXPLAINING HOW the task can be accomplished, there is a knee-jerk questioning of why it is even required.

It's like going to the doctor, saying you would like to keep the pharmacy at the doctor open and being told that you don't need the doctor's pharmacy

• you can just use the one across town even though it will close before you get there.

Volunteer as much as you ask questions. Standing behind the wall is easy. Meet us in the keep instead so we can have a discussion. State your viewpoint with precision and indicate what steps you think are sufficient. Otherwise, it's all just in your head.

But an upvote does not require an explanation. Someone requesting an explanation of an upvote is the person that should be explaining their point of view in detail.

Thanks.

On Thu, Feb 2, 2017 at 12:21 PM, hodyroff notifications@github.com wrote:

@Powie https://github.com/Powie Please help us understand why. What is your use case? And why can this not be achieved with file system utilities?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/owncloud/core/issues/1347#issuecomment-277022449, or mute the thread https://github.com/notifications/unsubscribe-auth/AC6oTq6vOgHXSCub2_3MypePWQ9BKFpVks5rYhCygaJpZM4AY6Ic .

[Powie]

@hodyroff Example: The installation is inside a VM where the user has an amount of 500GB Space. Owncloud will see this 500GB space and will use it in max. But inside this container there are other space requirements too, for mysql, logs, etc. If nobody manually controls space consumption of OwnCloud, it will happen that oc will use anything of space inside the 500GB limit and this VM will run out of space and go down. Many users haven't the chance to control this via user quota, or partitioning, or anything similiar.... They use a "webspace" and have limited control. And also, if space is limited over quota for OC , OC will see the whole disk space and reports this as available. thats wrong and if quota is full oc will fail working.

[ShamimIslam]

At this time, quotas are assigned to users one at a time or by default at the time of creation. There is no mechanism to update all users to a new grop quota en mass when the group quota has been changed. The group is only used to assign the quota to the user at the time of creation. Someone thought that user quotas could only be enforced and/or valid at the use level.

On Fri, Feb 3, 2017 at 1:37 AM, Powie notifications@github.com wrote:

@hodyroff https://github.com/hodyroff Example: The installation is inside a VM where the user has an amount of 500GB Space. Owncloud will see this 500GB space and will use it in max. But inside this container there are other space requirements too, for mysql, logs, etc. If nobody manually controls space consumption of OwnCloud, it will happen that oc will use anything of space inside the 500GB limit and this VM will run out of space and go down. Many users haven't the chance to control this via user quota, or partitioning, or anything similiar.... They use a "webspace" and have limited control. And also, if space is limited over quota for OC , OC will see the whole disk space and reports this as available. thats wrong and if quota is full oc will fail working.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/owncloud/core/issues/1347#issuecomment-277175586, or mute the thread https://github.com/notifications/unsubscribe-auth/AC6oTgesCFnSQdYXdkusGpf3TEZ-O6_Bks5rYssmgaJpZM4AY6Ic .

[ghost]

@ShamimIslam You might have noticed that the question of @hodyroff was not redirected to you but to the previous poster @Powie asking for a global disk quota (thats why @Powie is in front of that question). No need to get aggressive like you're currently getting, especially as this is not about your per group quotas.

[ghost]

@Powie

And also, if space is limited over quota for OC , OC will see the whole disk space and reports this as available. thats wrong and if quota is full oc will fail working.

oC is using the native PHP method disk_free_space to get the available free space:

https://github.com/owncloud/core/blob/v9.1.4/lib/private/Files/Storage/Local.php#L294

If this method doesn't respect system quotas then this issue needs to be brought up to the PHP group / the PHP bugtracker.

[gmta]

@kdslkdsaldsal See issue #5352

The method disk_free_space reports free space on the partition. But even if it would include quotas, you would have many race conditions on your hands since multiple users can upload files at the same time.

[ghost]

But even if it would include quotas, you would have many race conditions on your hands since multiple users can upload files at the same time.

IMHO still better then nothing :-) This feature request now sticks here around since 4 years and no one stepped up to implement this in ownCloud.

[ShamimIslam]

In all honesty, I don't follow the justification required for a vote. It's always the same. Why does a vote require justification? I don't see others being required to justify votes. It's not about my request. It's about the knee-jerk "why should we, we think the product is complete" reaction. So I politely disagree.

Also, I am pointing out that the folks in charge patently ignore the needs of the community and stimply states "just use xyz feature - why do you need abc". Happens over and over. Perhaps in 2017, we can ask more intelligent questions and provide less condescending answers and questions?

E.g.

Dear user,

Please try feature xyz for your need abc and let us know if it works for you. If you need suggestions on how to use it, please lookin def location.

Could you please specify the use case for abc so we can see if it applies to our roadmap or if it should be included.

Cf.

What we had was;

User,

Have you tried xyz? Why do you need abc?

---

Poor communication of what you want and terse aggressive quips suggesting a lack of exploration or that the needs are invalid do nothing to promote community participation or good will.

Use a template like the one above and you'll never hear me complain that someone wants justification. Because the need has been validated and the desire to learn about the use case to see if it should be included has been communicated. Instead of meeting the bar for inclusion, it becomes a dialog.

On Fri, Feb 3, 2017 at 10:06 AM, kdslkdsaldsal notifications@github.com wrote:

@ShamimIslam https://github.com/ShamimIslam You might have noticed that this question was not redirected to you but to the previous poster asking for a global disk quota. No need to get aggressive like you're currently getting, especially as this is not about your per group quotas.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/owncloud/core/issues/1347#issuecomment-277270170, or mute the thread https://github.com/notifications/unsubscribe-auth/AC6oTv2FdgjQPxbu__Y2v-b8k4OGbAVpks5rY0J1gaJpZM4AY6Ic .

[ghost]

@ShamimIslam It seems that you're not capable of reading posts of other people and that you're still getting this somehow personally.

What we had was; User, Have you tried xyz? Why do you need abc?

No, what we have was:

Please help us understand why. What is your use case? And why can this not be achieved with file system utilities?

@hodyroff Asked why the user thinks that global disk quota is important and whats his use case for such a disk quota. The "Have you tried xyz? Why do you need abc?" is only a minor part of this question.

Exactly this question helps to understand people like @hodyroff to decide if a feature should be implemented by payed developers of his company if it doesn't get picked up by other developers not working for the company.

And finally my personal two cents:

If this is a such needed feature and it doesn't get picked up people are free to implement this on their own, pay the company to implement it or hire some one else to implement it. ownCloud is still Open Source...

It seems to me that it happens that often (especially within ownCloud) that people are taking it personally if a feature they want is not getting implemented. And that they also don't understand there is also only a limited amount of resources / developers available implementing new features.

[ShamimIslam]

It's not about that. It's about even IMPLEMENTING a feature yourself and other people finding it useful are not enough to make the solution available to others as a choice (which I've done before). I don't complain without offering something in return. The only thing I'm missing in implementing this feature myself is a complete class diagram. Any chance you know where this is? (I can't find it in the docs so a link would be appreciated.)

Just send me the link and my involvement in this thread is done.

[ghost]

I know that there is a documentation for developers available here:

https://doc.owncloud.org/server/latest/developer_manual/

If you need specific guidance for something within the ownCloud core you might want to jump into the IRC channel #owncloud-dev on Freenode. There you will find people which might be able to answer your questions.

[Powie]

Back to this feature....

For us, as a webhosting company, this missing option to OwnCloud is the issue why we disallow customers to install OwnCloud. Customer accounts, based on Plesk, have many ability's, also to limit ftp space etc. But, OnwCloud "don't cares" about that, and this has killed some machines in the past. The customer hasn't control over space consumption. If nobody can limit OwnCloud. its banned from our servers.... This is a pity

[ShamimIslam]

No class diagram. Just a lot of "here's what we think you should know." It does not help me find things. Is there an online API reference? I've looked through all of this before. Grepping across multiple files to trace call is very tedious. While I know WHAT to do, I can't find WHERE to do it.

On Mon, Feb 6, 2017 at 2:18 AM, kdslkdsaldsal notifications@github.com wrote:

I know that there is a documentation for developers available here:

https://doc.owncloud.org/server/latest/developer_manual/

If you need specific guidance for something within the ownCloud core you might want to jump into the IRC channel #owncloud-dev on Freenode. There you will find people which might be able to answer your questions.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/owncloud/core/issues/1347#issuecomment-277602548, or mute the thread https://github.com/notifications/unsubscribe-auth/AC6oTqpH9O4uN31GB650WG-gP9Cgoe32ks5rZskvgaJpZM4AY6Ic .

[ShamimIslam]

@Powie You can implement a global default quota in a group, and then new users get that quota. If you want to force a quota for already existing users, you have to set it in the preferences table. But currently there is no function to set this across the board. OC does care. The makers just haven't enabled the enterprise level needs. Just regular users. I have a similar situation. I manage a large team of users. Some are not allowed to edit anything. Used to be I could limit uploading content by setting a group (There was a restricted_access app but I can't upgrade the app which belongs to someone else and is unmaintained without proper documentation of the classes and function calls available - the entire OC tree is huge). Other users in my group should only be allowed to add X number of MB. But I have to set each and every user by hand. And I can't set group quotas by moving users around. If I can get hold of the right level of documentation, I will be implementing this myself. Unfortunately from what I know of the OC event system which is still being reworked, there is no consistent precondition/postcondition check in all cases that I can hook into to check group level quotas. So I'm still considering my options. For my group for the moment, I've resolved to just run SQL scripts but it's not useful if I have to hand over administration to someone else who is not so tech-savvy. I don't plan on administering this for ever. It has to be productionized at some point.

[ghost]

@ShamimIslam You might want to follow the advise in https://github.com/owncloud/core/issues/1347#issuecomment-277602548 if you need specific information on how to implement something:

If you need specific guidance for something within the ownCloud core you might want to jump into the IRC channel #owncloud-dev on Freenode. There you will find people which might be able to answer your questions.

[PVince81]

I few thoughts about the possible implementation details:

Need a way to retrieve used space for several users in a group.

This is likely not yet possible to do directly on database level because currently user and group lists are not always in the database depending on what user backend you use. If you use the default database backend, users are in oc_users and oc_groups. But for LDAP they are listed in oc_ldap_user_mapping and oc_ldap_group_mapping and other backends might have different tables or no tables at all (IMAP?). So if one wanted to implement a DB-level query it would not be possible and would require a PHP level join. The PHP level join would likely need to go through the group manager \OC::$server->getGroupManager() and ask for all users in a group, and then for each user, go to the "oc_filecache" and get the "size" of the "files" entry. (a possibly SQL query for this is here https://github.com/owncloud/core/issues/1344#issuecomment-245526902

So let's say when user "user1" who is in group "group1" wants to upload a file. The code from setupFS() would usually setup the filesystem for "user1". There is already a quota storage wrapper https://github.com/owncloud/core/blob/v9.1.3/lib/private/legacy/util.php#L201 getting applied. I suggest to write a new type of storage wrapper called GroupQuota and apply it in a similar manner. (note that apps can also register storage wrappers so it might be possible to write this as a separate app). Then the GroupQuota needs to fetch the usage of all users in the group "group1" to find how much space is left and then do proper limiting. That one usage query could already be expensive.

For now I feel that it's too early to be able to implement this efficiently as performance will likely be bad when querying the usage for all users in a group (unless someone has new ideas). There is another ticket in the works https://github.com/owncloud/core/issues/23558 to bring all the user/group information in central database tables regardless of user backend (LDAP, IMAP, whatever). Having these tables would make it possible to use actual table joins to calculate quota usage in one go.

Also some other challenges have to be thought through like what happens when a user is member of multiple groups. In this case, what quota applies where, etc. I don't remember the whole discussion but I think this needs to be figured out / designed properly before diving in.

Now to summarize what could be done with the current code:

1. Provide a new field in the users page to set a quota per group (this might need a good bunch of refactoring because currently groups are not editable). That page connects to https://github.com/owncloud/core/blob/v9.1.3/settings/Controller/GroupsController.php.
2. Provide new DB column somewhere, possibly a new table, to store the group quota. (remember, there is no central groups table yet until we have https://github.com/owncloud/core/issues/23558)
3. Implement a GroupQuota storage wrapper similar to https://github.com/owncloud/core/blob/v9.1.3/lib/private/legacy/util.php#L201 (copy+paste+adjust)
   • fetch used space for the group of the given user
     • use \OC::$server->getGroupManager()->getUserGroupIds($userId) to find the user's groups
     • for each user in the group, go to DB and do a query like https://github.com/owncloud/core/issues/1344#issuecomment-245526902 (could also be a `select ... where userid in (...) but beware of query limit with sqlite). Here need to figure out what to do if the user has multiple groups depending on the matching design decision.
     • store the used space of the group in a local variable (or cache it somewhere in a table ??)
   • make GroupQuota::free_space return the free space based on the previous value. This method is used indirectly in other places in the code that assess how much space is left like the Sabre QuotaPlugin: https://github.com/owncloud/core/blob/v9.1.3/apps/dav/lib/Connector/Sabre/QuotaPlugin.php#L107. So the quota restriction should already be in place.
   • investigate whether there are other code paths slipping through, just in case (retest all upload/save cases like webdav PUT and files_texteditor)
4. Register the GroupQuota storage wrapper like it was done in https://github.com/owncloud/core/blob/v9.1.3/lib/private/legacy/util.php#L201

This sounds like it could work assuming that users are only members of a single group. However need to also think of implications through the whole platform (trashbin, versions, etc). Some apps might rely on the free space to do some actions like cleaning up the trashbin when more than 50% space is reached. Need to make sure that this works correctly with the group quota.