search

owncloud / core

:cloud: ownCloud web server core (Files, DAV, etc.)
https://owncloud.com
GNU Affero General Public License v3.0
8.36k stars 2.06k forks source link

Sharing 2.0 #19331

Closed rullzer closed 8 years ago

rullzer commented 9 years ago

@schiesbn and myself spend some time at the conference coming up with a possible design for sharing 2.0. Here is basically an overview of my notes.

Note

As discussed in Berlin the sharing stuff will for now only be for files. Contact and calendar will use the old sharing code but will be moved to webdav.

Architecture

We want to move all real sharing to separate apps. Right now sharing is mostly in core and this is not really the right place. It makes it hard to have custom sharing implementations.

We propose a single sharemanager in core where apps can register themself. Furthermore in core we will provide a number of interface that apps have to implement in order to become a shareprovider.

Currently we have the following 3 categories of shares:

We will of course keep those 3 categories but each of them will be a separate app. This would mean we would get more core apps. Which will take some time but in the end we should end up with a more modular (and better maintainable) sharing setup.

We will then also need another new app that handles the OCS Sharing API. This clearly separates the API and the sharing implementation

No more sharing hierarchy

We propose to move to what we like to call a flat sharing model. This will prevent cyclic sharing or at least make sure it is no longer an issue. (see also https://github.com/owncloud/core/issues/9058)

This effectively means that shares are a combination of the following items

This means that the share owner always has the complete overview of who has access to a share. And that we can easily find out the max permissions for a sharee.

Interfaces

sharemanager

The share manager is just a place where shareproviders can register and where they can be queried. There will be some functions to get a specific share or to get a traversal of all shares in a path etc.

shareprovider

The share provider just implements sharing for a specific type. You can of course tell the provider to share a file. Or to get a share etc.

Each share provider has a unique id.

We plan to have two share providers:

  1. "internal share provider" which will be provided by the primary storage and handle (user, group and link sharing)
  2. "federated share provider".

In general share provider will register them self at the share manager. This way it is easy to develop and add new share provider. For the internal share provider (which could be seen as the main share provider) the share manager will ask the primary storage directly to provide it.

We will implement two share providers as apps, the "oc internals share provider" which does sharing basically as we are doing it today and the "oc federated share provider". Bot will register them self at the share manager if the apps are activated. Additionally the share manager will ask the primary storage if he implements his own share provider. If he returns a share provider it will be used for internal shares, if it returns false the "oc default internal share provider" will be used. That's also the reason why the "oc default share provider" needs to be implemented as a app and not as part of our default storage, because the share provider needs to be also available for other storage if they don't implement their own share provider.

share

Just an object representation of the share. Where you can get and set the attributes etc.

Each share withing a shareprovider has a unique id.

A system wide unique id could be constructed as "providerID#localShareID" or something similiar.

We decided to only pass around userids/groupids/federatedshare ids etc. This makes sure that only the places that have to care about existing users etc will handle that.

Example Code

I took the liberty of already starting with some interfaces:

ShareManager ShareProvider Share

Limitation

We can only handle 1 share provider per type.

During the conference we had the idea to do things on a per storage level. But this brings in a lot of problems. Assume there are 2 folders dirA and dirB. Now the storage where dirA lives supports link sharing but the storage of dirB does not. Now what happens if a file with link share is moved from dirA to dirB?

In general most people will just use the apps that we provide. And in the case that people want very specific sharing behaviour for certain storages they should write their own sharing apps.

Forcing this limitations keeps things simple and understandable and we do not believe this to be a limitation a lot of people will have problems with.

Migration/deprecation path

For third party apps like calendar and contact we will move the existing sharing code to the apps and will make them work. From this point on the app delevlopers have to maintain or replace the existing code.

For file sharing we will re-use the exiting table, update the values add needed columns and remove old columns we no longer need at the end. The idea is that this process will be triggered with a occ script after the main upgrade to 9.0 while the system is live (still need to evaluate if this is possible).

Open Question:

Since we kind of start over. There are a lot of things where we can improve and unify the current sharing design. Some things we came up with.

Expiration date on everything

Currently it is only possible to have expiration dates on link shares. There is no real technical reason for this and it is perfectly valid to have other shares expire as well.

Multiple link shares

Currently only 1 link share is allowed per file. Having multiple makes sense since I might want to share a file with multiple people but not for the same period of time. This could be combined with easily sharing via email so the individual link is automatically sent to the respective person. Individual links can then also be revoked easily based on email/name.

Open Issues

The idea is to implement it in parallel to the existing sharing with small pull request merging sharing 2.0 one by one. In this transition phase it will be possible to switch between old and new implementation for testing purpose. Only if sharing 2.0 is complete and works reasonable well (including all integration tests for the OCS API) we will move the old sharing to the calendar and contacts app like explained under "Migration" and switch permanently to the new sharing.

DB implentation and considerations

In order to faciliate the new sharing we need to modify the database. We want to do this as little as possible to avoid large and huge migration steps.

Eventually we will remove the columns that are only needed for calendar/contacts sharing since they move to webdav. But this can only be done once we kill off the old code (since it is heavily wired into it).

Currently we store the initiated of the share in uid_owner. So the table uid_owner maps to the sharer (sharedBy). We can get the owner of a file via the fileid in the table. But this does not scale. Thus we plan to introduce a new column that holds the owner of the file (shareOwner). This is then allows for quick lookups of share info for the owner since the owner of a file should always have the full overview.

Related issues:

rullzer commented 8 years ago

After thinking about this I'm still not entirely sure what we want. I understand that it is weird that shares keep on exisiting but I also agree with @schiesbn that it is very weird when one delete cascades, especially since it is visible now.

For reshares assume the following situation:

Folder
|-subFolder
  1. user0 shares Folder with user1
  2. user1 shares Folder with user2
  3. user2 shares subFolder with user3
  4. user3 shares subFolder with user4

Now we have a number of views. We have the view from the owner of the files (user0). From people that share files and receive files. Thus:

user0:

Folder (shared by you with user1, shared by user1 with user2)
|-subFolder (shared by user2 with user3, shared by user3 with user4)

user1:

Folder (shared with you by user0, shared by you with user2)
|-subFolder

user2:

subFolder (shared with you by user1, shared by you with user3)

user3:

subFolder (shared with you by user2, shared by you with user4)

user4:

subFolder (shared with you by user4)

As you can see user0 has the entrire view. Since it is their data. The other users have a much more limited view. This is since they do not need to know everything (it is not their data). And it will become very complex and costly to calculate that all for larger installations.

No assume user0 thinks I don't want to share my data anymore with user1. Then they have to construct in their head the entrire share tree to know what consecuenses their removal has.

I agree with @schiesbn that probably only removing the share to user1. Would make the most sense. Since user0 has the complete overview anyway. And they can delete all the other shares as well.

Deleting all the children was fine in the old situation since the owner of the files could only see their own shares. But now that they can see it all this makes it a bit more complex. I do think we should make sure that the behaviour is the same for reshares of folder and reshares of subFolder. So either we delete it all or we keep it all.

icewind1991 commented 8 years ago

Since user0 has the complete overview anyway

The problem is that with sharing sub-folders the re-share might be buried deep down into tree and easily missed

DeepDiver1975 commented 8 years ago

What about disallowing to reshare sub folders of a share? @MTRichards @schiesbn @rullzer

rullzer commented 8 years ago

@DeepDiver1975 that would significantly simplify things. And in general I agree that complex resharing structures are more of a theoretical need.

But there is the following situation (which I think is pretty common):

The reason I know this is happening is because my file does stuff like this all the time. We have a shared folder of photos. And she wants to share some of those with other people (by link).

This would then no longer be possible.

rullzer commented 8 years ago

On the other hand we already do this limitation to some extend in the existing code.

fodler
|- subfolder (shared with me by someone)

If I receive the share of subfolder. And move it to folder. Then I can no londer share folder because it contains a share shared with me. So we do already limit this kind of stuff.

MTRichards commented 8 years ago

We can't remove child shared folder resharing, but can't we just add the reshare to the owners sub-folder as part of the share refactor? Just like if you reshare the entire folder? Then if you reshare a sub folder, it is just added to the owners list of sharees, 3 folders down, or whatever the situation is.

rullzer commented 8 years ago

@MTRichards that is the plan. (See the situation I described in https://github.com/owncloud/core/issues/19331#issuecomment-158873018).

However, that does not solve what to do on deletion.

icewind1991 commented 8 years ago

I still feel deleting any reshares is the correct way

schiessle commented 8 years ago

The problem is that with sharing sub-folders the re-share might be buried deep down into tree and easily missed

Therefore you have the overview pages "shared with you", shared by you", etc.

The whole idea about getting rid of the re-share hierarchy was to make thinks easier. Otherwise we have the same mess again. What sounds relatively easy if we discuss this with three users (share + re-share) become quite complex if we take more levels of -re-shares into account and some user in between removes his share.

The whole idea of the flat sharing list was that you don't create a re-share in the sense of a tree but I give you the permission to create shares "in my name". Once you have done this there shouldn't be a difference whether I created the share by my own or you did it. The new share is a independent share. The share is there because I allowed other to create it, I as a owner can see and control it. That's it.

I could even imagine that it would be quite confusing for the owner if he removes one share and 10 other shares disappear magically. In the past this was different because the owner didn't saw the re-shares but now he sees them, so let him control it. I think in the most cases you want to remove exactly the share where you click on "unshare" if you see all shares and don't want to create a chain reaction deleting other shares as well.

The good thing about the flat share list is that we no longer have to guess and try to make the right decision for the owner. The owner sees all shares and has the full control. So let him execute his power and don't get in the way. At least this would be my approach.

PVince81 commented 8 years ago

from my pov we will need string because of all discussed reasons - but this would require v2 of the sharing api which I'd like to postpone for now (maybe a topic for 9.1).

Agreed. Also changing int to string on the interfaces later shouldn't be a big change (or introducing new interfaces). The biggest part is making sure the clients can still work with string ids.

DeepDiver1975 commented 8 years ago

Given the fact that a re-share is nothing more then a impersonated share of the owner - there is no need to handle the unshare operation any special.

A shares photos with B B shares photos/vacation with C For A C will be shown as shared with on photos/vacation If A removes B from photos there is no need to remove C from photos/vacation automagically

@MTRichards does this make sense from a PM pov?

rullzer commented 8 years ago

Added some text for the required db modifications (and why)

jancborchardt commented 8 years ago

@DeepDiver1975 yup. Actually if A removing B would automatically remove C as well that would be really confusing. After a few weeks/months no one remembers who reshared what with whom and unsharing from just one person should not cause that whole tree to collapse.

jospoortvliet commented 8 years ago

For what it's worth, I'd also vote with @DeepDiver1975 and @jancborchardt on this.

It is hard to figure out what people would expect and for sure some will have different needs and expectations anyway, but https://github.com/owncloud/core/issues/19331#issuecomment-158927029 by @schiesbn has the answer:

The whole idea of the flat sharing list was that you don't create a re-share in the sense of a tree but I give you the permission to create shares "in my name". Once you have done this there shouldn't be a difference whether I created the share by my own or you did it. The new share is a independent share. The share is there because I allowed other to create it, I as a owner can see and control it. That's it.

It might not always be what users want but it does lead to the most predictable behavior. And the worst it will ever do is create a bit 'extra work' for the original sharer to manually remove all shares done by others he/she doesn't want.

rullzer commented 8 years ago

We must also make sure that the webinterface is ported to the OCS Share API in order to ensure a consistent view. As the old ajax endpoints still use the old code paths.

moscicki commented 8 years ago

Is this going to be part of OC9? Sharing currently is a major source of page loading slowness in current cernbox, we would like to check if this improves in OC9.

CC: @karlitschek

DeepDiver1975 commented 8 years ago

Is this going to be part of OC9? Sharing currently is a major source of page loading slowness in current cernbox, we would like to check if this improves in OC9.

yes - definitely - parts have already been merged into master branch

davivel commented 8 years ago

@rullzer, I have a question

What's the need to have a READ bit in permissions of a Share? Is there any situation where a valid Share should have a value 0 for that bit?

I am writing some automatic tests for updating permissions in the Android app and I thought that updating a Share to remove the READ permission would fail, but that's not the case.

Thanks.

EDIT: asking here since I see in the list of tasks that "Update share" is not updated yet.

rullzer commented 8 years ago

@davivel well it is there for legacy reasons. We used to allow removing it (which resulted in broken shares). So right now it is just always added. Even when you create a share without read permissions they will be added automatically.

If you try to remove them it will thus allow the call. But if you then fetch the share again the read permissions should still be there.

davivel commented 8 years ago

Even when you create a share without read permissions they will be added automatically.

Nice to know :)

Thanks a lot, @rullzer

rullzer commented 8 years ago

@DeepDiver1975 I remember we talked about the bullet allow to pass in a token when creating a new share is that still required? I mean when update share allows you to migrate the share all should work right?

PVince81 commented 8 years ago

@rullzer @schiesbn please move all remaining non-9.0 items to a separate ticket so we can close this one (if applicable)

rullzer commented 8 years ago

Ok lets move all remaining internal stuff to https://github.com/owncloud/core/issues/22209 and then this can be closed.

lock[bot] commented 5 years ago

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.