Access control when creating Post objects

[stevestreza]

Posts should be able to be restricted in their visibility at the API. This means a Post is never delivered through the API to a Stream which is not allowed to see it. The inclusion of access control would have a profound impact on the platform and would enable a huge variety of apps and use cases that are not possible under existing social networks and message routing systems.

• With no access control, Posts should be public to all Streams, regardless of access token availability. With any level of access control, Posts must require an access token to be seen, and only are seen per the rules below.
• User-based access control (these are mutually exclusive)
  • Restricting a Post to specific users means that the Post only appears in Streams with access tokens that correspond to an allowed user
  • Restricting a Post from specific users means that the Post only appears in Streams with access tokens that do not correspond to a restricted user
• App-based access control (these are mutually exclusive)
  • Restricting a Post to specific apps means that the Post only appears in Streams with access tokens that correspond to an allowed app
  • Restricting a Post from specific apps means that the Post only appears in Streams with access tokens that do not correspond to a restricted app

This will enable lots of use cases that could really blow up the utility of the service beyond a Twitter-style system. Some examples include:

• Twitter-style Direct Messages or other one-to-one personal chat (restrict Posts to be visible to one user)
• Group chat applications (restrict Posts to be visible to a set of users)
• Facebook-style or Path-style list publishing (restrict Posts to be visible to specific people)
• Cloud-based application RPC/notifications (restrict Posts to be visible to a single application or set of applications, possibly for a single user)
• Friend-based gaming (restrict Posts to be visible to a single application AND a specific set of users)

[fields]

I wonder if it makes sense for app-to-app posts (non-human-readable) to be an entirely separate stream with different rules.

[JoshBlake]

@fields It would most likely appear as a separate stream in clients (not necessarily in UI except for developer debugging consoles) but I think in the API it can be implemented by adding the properties discussed above to the post and by filtering streams on those properties.

[JoshBlake]

@extremeboredom optional and required makes a lot of sense actually. in most cases I think it'd be a decision like "Do I want to just play chessapp once or twice and not bother my followers?" (minimum permissions) or "Do my group of friends play chessapp a lot and let chessapp brag to them?" (minimum + optional permissions) In any case, I don't think it would require permissions for chessapp or any app (even non authorized) to open a window with a suggested post text that I can edit and choose to post. Thinking similar to the twitter feature here.

[jschlesser]

Does the app need permissions? yes Do you trust the app? yes Are you smart enough to understand what the technical authorization jargon is?, yes Do you authorize the app and permissions?, yes app gets hacked are you still screwed, yes

None of the knowledge that you have or choices you make saves you from that, for my mom its even worse, she just wants the app because her friends have it. The rest is gibberish and extraclicks between her and the functionality she wants.

The question most people will ask is "Can I trust this app?".

I dont have an answer to how to make apps trustable but presenting people with a bunch of technical jargon they dont understand isnt going to make apps more or less hackable and will be ignored by most people.

Its more reasonable for my mom to see options inside the app to make decisions like: share photo with everybody [x] share photo with friends only [x] send photo to private jornal [x]

The app will need to do the right thing, if it doesnt, nobody will use that crapware and use something else.

Maybe there should be options for app developers for things like secure streams so that the designer of an app can secure sensitive info. That way sensitive apps could protect themselves from crapware. I guess strong encryption could be used in the client and the content would never be readable in the stream in the first place too though.

On Aug 14, 2012, at 10:52 PM, extremeboredom wrote:

You could allow for optional and required access, e.g. a chess game might require app to app posting to work, but optionally can use app to human posting if you want to post your scores. It does make things more complicated though.

My worry is that I'm going to give positing permissions out to a massive bunch of apps, and then one of the apps is compromised and suddenly your entire App.net stream is controllable by others. A lot of damage could be done in a short time like that. Maybe I'm just being paranoid though.

— Reply to this email directly or view it on GitHub.

[extremeboredom]

@jschlesser That's perfectly true, I could be very careful about what I authorise and still get hacked. The possibility though is not the same as wanting to provide a large number of potential entry points by allowing every app under the sun to post whatever to my stream.

I'm not advocating making it unintelligible to users, I'm looking for as simple as possible an answer. I'm just wary of the fact that App.net is not just like Twitter, it's also effectively the basis of multiple network 'protocols'. Trusting something to communicate with other instances of itself is not the same as trusting it to post human-readable content to my followers on a Twitter-like service.

[jschlesser]

I agree with all those statements. If you can make it transparent, intelligible and 1 step its probably worth while.

The more I think about it the more I think that the onus for protecting a sensitive stream of content should be given to the app creator, the burden of excluding other apps from seeing it shouldnt be up to the other apps or the user to go configure the other apps. think financial data or health data. responsibilities should be with the app creators in that instance not anybody else.

As for general stream posting, I think that its inevitable that my mom is going to sign up for an app that ends up spamming through her account so she can share virtual flowers or something with her friends. Its just inevitable. I see a few things to mitigate this.

1. A service/app that can act as a review and filter on app authorizations. Lets say app authorization is an event with pre and post hooks and I subscribe to a service that gets those hooks, it could check in a database of known bad or good (black or white list) apps and approve or deny.
2. Suppose my mom has that service but she wants to share virtual flowers anyway, she doesnt care that virtual flowers is placing 1800 flowers ads and mothers day ads in her stream. I want to be able to still see my moms stream without seeing the virtual flowers ads.

On Aug 14, 2012, at 11:55 PM, extremeboredom wrote:

@jschlesser That's perfectly true, I could be very careful about what I authorise and still get hacked. The possibility though is not the same as wanting to provide a large number of potential entry points by allowing every app under the sun to post whatever to my stream.

I'm not advocating making it unintelligible to users, I'm looking for as simple as possible an answer. I'm just wary of the fact that App.net is not just like Twitter, it's also effectively the basis of multiple network 'protocols'. Trusting something to communicate with other instances of itself is not the same as trusting it to post human-readable content to my followers on a Twitter-like service.

— Reply to this email directly or view it on GitHub.

[duerig]

Here is my own effort to create a prototype tagging system that we can use to try out various ideas. It just uses post/get variables for arguments so we can just use curl or any http library to interact with it.

Lets start with a simple social graph. I'm going to use letters for userids here even though you will want to use app.net userids which are numbers for the real deal. Bob follows Alice and Alice follows Carol. We add these relationships using the follow command.

curl -F 'userid=a' -F 'followers=["b"]' -F 'followed=["c"]' sentagstream.appspot.com/follow

Now Carol comes across a hilarious lolcat so she tags it as "+funny" with her client. It makes the following call to indicate that Carol thought the post was funny.

curl -F 'postid=0' -F 'post=lolcat' -F 'user=c' -F 'key=funny' -F 'value=+' sentagstream.appspot.com/tag

When Alice's client next grab's Alice's stream, it sees Carol's new tag on it. Depending on the filter used, it uses that tag to decide whether to show Alice the message.

curl -F 'userid=a' sentagstream.appspot.com/stream [{"post": "lolcat", "postid": "0", "tags": [{"value": "+", "user": "c", "key": "funny"}]}]

Note that Bob's client does not yet see this in his stream at all because it hasn't been tagged by someone he follows.

curl -F 'userid=b' sentagstream.appspot.com/stream []

Lets say that it does show Alice the message, but she is offended by the lolcat's grammar so she tags it as -funny. Her client adds the tag for her.

curl -F 'postid=0' -F 'post=lolcat' -F 'user=a' -F 'key=funny' -F 'value=-' sentagstream.appspot.com/tag

Now when Bob's client grabs his stream, it will see Alice's tag but not Carol's tag because he only follows Alice. His client will see that it is marked not-funny so will not show the lolcat to him (depending on his client's filter).

curl -F 'userid=b' sentagstream.appspot.com/stream [{"post": "lolcat", "postid": "0", "tags": [{"value": "-", "user": "a", "key": "funny"}]}]

If you want a global view of the tags on this post, you can get it by postid and see all the tags that everyone has attached to it.

curl -F 'postid=0' sentagstream.appspot.com/get [{"post": "lolcat", "postid": "0", "tags": [{"value": "+", "user": "c", "key": "funny"}, {"value": "-", "user": "a", "key": "funny"}]}]

Now you can upload parts of the actual app.net social graph and start using actual users/posts and tagging them.

Available at: sentagstream.appspot.com No authorization, will likely not scale to heavy usage. This is really only good for prototyping. It can tide us over until a more real implementation is completed by the app.net crew.

All parameters can be passed as get or post variables. It has the following entry points:

/follow

'userid': An app.net userid (ex: "234") to add followers/followed to. 'followers': A JSON-encoded list of strings each of which is an app.net userid (ex: "234"). These will be marked as followers of user. 'followed': A JSON-encoded list of strings each of which is an app.net userid (ex: "234"). The user will be marked as followers of these.

Returns: Nothing

/tag

'postid': An app.net postid (ex: "5320") of the post to tag. 'post': A JSON-encoded app.net post object. This will be stored in the database if this is the first time this post has been tagged (only tagged posts are stored here). 'user': An app.net userid (ex: "234") which is the user which tagged the object. 'key': A string (should be [a-zA-Z]+ but not enforced) which is an opaque tag name. 'value': A string (arbitrary) associated with the key. A simple scheme might just use "+" or "-". More complicated ones might use numbers, ids, etc.

Returns: Nothing

/stream

'userid': An app.net userid (ex: "234")

Returns: A json-encoded structure of posts tagged by the users that are followed by that user.

[ {"post": <json-encoded post object as a string>,
   "postid": <app.net postid>,
   "tags": [{"user": <app.net userid>, "key": <key>, "value": <value>}
            ...]
  }
...]

/get

'postid': An app.net postid (ex: "5320")

Returns: A json-encoded structure of posts tagged by any users.

[ {"post": <json-encoded post object as a string>,
   "postid": <app.net postid>,
   "tags": [{"user": <app.net userid>, "key": <key>, "value": <value>}
            ...]
  }
...]

[orianmarx]

@duerig you should probably move your tagging post to another thread? The difference between filters and permissions is definitely nuanced but I think this falls into the filters category.

[orianmarx]

We definitely need a Group object of some kind in the API. This will be critical for a fully functional permissions system. I'm still working my way through the API but I don't see how follower lists can scale without being treated as group. A User object should have potentially many associated Groups, which are all lists of other users and groups. For every network running on app.net that I join, I should have following and follower groups created. I can't imagine my App.net account having one global follower lists that is then filtered down by service, rather than the reverse, where my global follower list is built up from all my follower lists on all the different services I use.

[orianmarx]

I just came across ToS;DR - a new effort to give visual report cards to startups' Terms of Service. http://tos-dr.info/

I really like this. This could be a model for representing app permissions in a more user friendly way as @jschlesser and other have brought up here.

[mjstrasser]

This discussion has wandered a lot around filters and apps and bots and stuff but @amazingsyco’s proposal can add great power to app.net that is not available in other social networks AFAIK.

As I read it, he is proposing that every post to app.net can optionally specifiy any or all of:

• users or groups that may access the post (whitelist)
• users or groups that may not access the post (blacklist)
• apps (groups of apps?) that may access the post (whitelist)
• apps (groups of apps?) that may not access the post (blacklist)

Is mutual exclusivity needed within user and app permissions? Can a whitelist and blacklist combination work?

These are attributes of the post that app.net will enforce so subscribing users/apps are unaware of posts to which they are not entitled. Sure, the contents may be extracted and reposted by another party without respecting the intentions of the original creator but that kind of ‘sharing’ cannot be avoided and falls outside the control of app.net.

This kind of fundamental feature would apply to any payload: not simply short text messages. (I am assuming that app.net will be capable of transporting a range of items outside the scope of Twitter.)

—Michael Strasser (a.k.a. @pharsicle)

[duncanwilcox]

You need to beat this with the simple stick. There's never going to be a simple, understandable user interface for this. Ever.

[schwa]

"You need to beat this with the simple stick. There's never going to be a simple, understandable user interface for this. Ever."

I completely agree. This is looking to become a clusterfuck of a specification that no developer will be able to present to a user in a sane, easily understood manner.

[markwilcox]

@duncanwilcox @schwa This is attempting to define a generic mechanism for the API that can be used to implement lots of different use cases - unfortunately the discussion has mixed up lots of related concepts in the API.

However the point I want to make regarding an understandable user interface for this is that there will not be one UI that exposes all of this directly. For example, you can use user access control lists to implement direct messaging. You could have a UI to create groups of users in a similar way to the creation of mailing lists and then send messages to a group just as you would to a mailing list. Alternatively, there could be a group admin feature for an app and then all of the users in that group automatically just send messages to one another without any addressing UI at all.

Access control for apps is not something that should ever be presented in a UI (in use cases I've thought of so far anyway) it's a way to keep machine-to-machine communications private to the apps that understand them.

The mistake is not in having an abstract API that's flexible but in imagining that you should try to implement it all within a UI. :)

[markwilcox]

@mjstrasser Your summary is good. I believe that groups of apps can be used to implement all of the "protocol" related stuff discussed above. Groups of both users and apps will also need some access control schemes for who can add and remove members from the groups.

Whitelist + blacklist at the same time doesn't make sense logically though, regardless of the subject matter - if someone should be blacklisted simply do not include them on the whitelist.

[duncanwilcox]

@markwilcox so don't make it a generic behemoth then. Give the use cases understandable labels, say "private message", "app-to-app comm" and so on, so that everybody in the chain (specs, backend, frontends) has no doubt about what the thing does and how to build it. It's either that or you end with the non-supported use case and the generic UI.

[orianmarx]

@duncanwilcox I'm all for using concrete examples as ways to talk about how to make this work. That said I don't believe this has to be extremely complex in the end. I think the UNIX permissions model is probably the way to go, and I agree with @markwilcox that there is a big difference between how this gets implemented in the API and how it will be presented to users in a UI. The UI's will reflect whatever the complexity of the associated service is. If it's a completely public service then there will be basically no permissions UI.

[fields]

The simple stick involves reasonable defaults so that no end user has to make these choices and they'll get something that still makes sense and does what they expect it to do. Yes, apps will have to do some of that, but the API can encourage that through logical groupings of settings (i.e.: use cases).

[jschlesser]

Yes yes yes, beat some sense into it with the simple stick. As a person in favor of simplicity here are the things that I would like to have.

1. Super easy authorization of apps, even easier than it is today if possible. Think of a way to make it happen without directing the user to app.net to click a button.
2. A description of what the app does and uses its permissions for.
3. The place where I can deauth can also show the technical permissions but other than "post on your behalf" and "read your stream" I think they get esoteric.
4. Things I need to know before authorizing "do i know what this does", "do I trust it".

Assume that some apps from people you follow are going to be spammy and post crap into their streams for instance my mom will likely sign up for some crap with her friends that posts ads in her stream. FIgure this out by allowing me to filter out that crap on my side by muting crap auto sent on my moms behalf from a specific app.

[fields]

The problem with doing authorization in the app is that you have no way to trust that it's not sniffing your password. In order for oauth to be secure, the auth step has to be in the browser, where you can verify that it's actually app.net (and only app.net) that you're putting your password into.

[duncanwilcox]

The problem with doing authorization in a browser is it looks nothing like the app and why should the user trust that?

The app is going to request full permission on the account anyway, and if it really wanted it could do a man in the middle in the local webkit and sniff the password anyway.

Bottom line is oauth sucks and there's no need to subject users to a crappy user experience.

[duncanwilcox]

@jschlesser what will an app that only thinks rw- makes do with --x? Does --x even make sense? Will the confused developer have to test all combinations of r-- -w- and --x? Or just give up and build a generic UI for the user to figure out?

Why can't it just be "public message", "private message" and the like all the way down to the app.net core?

[jschlesser]

@duncanwilcox i dont understand completely. maybe the best thing to do is to draw up a hypothetical ux and see if your avg. non tech friend can make sense of it. If yes, then im for it. I use my mom as a good test, she is an artist, if she gets it, its usually good for everybody.

[jschlesser]

@duncanwilcox I also agree with crappy ux re oauth

[markwilcox]

OK, the discussion on this issue has wandered far from the original proposal, including tagging, filtering and user permissions for authorising an app, rather than the original scope of determining which posts which clients have permission to see via access control lists.

Following the spirit of my https://github.com/appdotnet/api-spec/issues/33#issuecomment-7649787 @orianmarx comment here https://github.com/appdotnet/api-spec/issues/33#issuecomment-7749107 and the one by @mjstrasser here https://github.com/appdotnet/api-spec/issues/33#issuecomment-7754520 - I've created a new issue #90 where the protocols concept is subsumed into groups of apps.

I find having a simple set of core building blocks in the API much more "simple" than making an large number of separate APIs for very similar use cases, particularly when we haven't invented all the use cases yet. The latter is easy for developers initially but not necessarily simple for the system as it grows.

[fields]

I put together another diagram to try to unify all of this stuff. Certainly there are many details to be worked out and all of this may not be feasible, but I think this captures most of what I'd like to see. This is more of a data abstraction than a use case flow. As depicted, I do think protocol groups, app selection, and tags are all parts of the same whole, and I think it works this way.

http://i.imgur.com/GUiri.jpg

Comments?

[fields]

Two other notes: 1) I left out general stream filtering on other metadata, which is certainly desirable, and 2) I realize I didn't explain the signing/app restriction stuff very well. I see that as allowing an application to restrict its posts to only other users of that same application, by some mechanism that allows presentation of a specific token or some kind of signing at post time combined with presenting the same token or signed request when fetching the stream or post. That would permit private usage like games, layered on top of user permissions.

[CocoaNative]

Something not yet mentioned is the concept of posts viewable only by people who have app.net accounts.

Currently all posts are "public", viewable by everyone and indexed by Google, et al. Each app or user might want to limit such exposure, or restrict access to users who have agreed to app.net's TOS (or the app's license agreement), etc.

As for another point: once lists of users become objects in the API (I assume they must, #90), then each of those could be specified for access control, which would create private streams or public side-streams (depending on each list's attributes). This could be useful for project teams and other workgroups.

[markwilcox]

After thinking about this for a while, seeing the annotations spec defined and discussed and observing discussion of a machine/human readable flag, I'm wondering if this isn't all more complex than it needs to be because we're assuming single input and output streams of posts per user. That leads to all of the users posted content from potentially many different types of apps or separate networks being multiplexed in a single stream. Also, all the content from everyone a user follows is multiplexed and then needs complex filtering for each app to get the relevant bits. Multiplexing is typically good for sharing a scarce/expensive resource like radio spectrum or cables in the ground - a real-time feed of short messages is an artificial construct with potentially a very low storage overhead on the server. Why not create more of them and use them separate content out at creation time to simplify filtering at retrieval time?

What if users could create multiple separate streams (or feeds if you prefer) of posts and also multiple channels for listening to individual streams from other users? (I assume you'd have to limit this per user, maybe more for devs to enable them to create multiple bots and test feeds for different apps within the one account rather than buying separate accounts for bots - the latter would seem to discourage experimentation).

Access control could then act at the stream level rather than the individual post level. I suspect simplifying to public streams that anyone can see (with app or group of apps access restriction) and private streams that are restricted to a group (as in #90) of users which is limited in number (<100?) would still cover all the practical use cases. Probably with separate endpoints for posting to and fetching from the private versions vs the public ones.

Other things would probably make more sense at the stream level than the individual post level - e.g. copyright licenses, or keys for encrypted comms as in #149.

Parts of this have been discussed in #120 & #123. I see separately followable channels are superior tagging for lots of use cases. For example I might be really interested in dev chatter about iOS but not financial analysts or fanboy posts on the same topic - at the same time I'm very likely not interested in the other random hobbies of some excellent developers. The one-size fits all social network feels like a very broken starting point we're currently living with but a single social identity that can be used across multiple subnetworks makes a lot more sense.

By questioning the 1 input stream per user and 1 (filtered) output mux per user assumption you can also look at related questions like: 1) Should all channels (output mux's) be private to the user? (public curated channels?) 2) Should all input streams only be writable by only one user? (maybe multi-author public streams are something you'd pay for separately, although a single private stream written by all the participants in a conversation would be a neat way to implement private 1-1 or group chats)

I could go on but as a final thought before seeking feedback - wouldn't separating various strands of purely machine readable posts and distinct sub-networks of content out into separate streams rather than filtering from one merged stream make it much easier to create a tiered pricing model and enable separate premium apps on top of the network later? Someone that only wants access to a Twitter-like service with a handful of separate topics they post on can pay a much lower fee than someone that wants to play with all the shiny new toys and services that developers are creating.

[ravisorg]

@markwilcox I like this idea, but a couple of quick comments:

Limiting groups it to a set number of users (especially a low number like 100) would kill very cool ideas like @maks' paid newspaper subscriptions

How would this allow direct messaging? Would you create a new stream every time you wanted to talk privately with a user?

[markwilcox]

@ravisorg Implement a paid newspaper subscription or anything requiring really large numbers of users by having a public stream that is access restricted to a set of apps (in this case official readers).

Private streams would hopefully be more lightweight than public ones, since you only have a small set of users for each. I think one stream per user you chat with privately would generally be OK? Possibly you have some kind of time-based archival process there such that the stream is no longer "live" and usable if you haven't used it in a long time - depends if that has any impact on server performance issues I guess.

[berg]

We just released the Messaging API today. I'm going to close this, and ask folks to create separate issues for specific changes to that API, if necessary.

Here are the docs: http://developers.app.net/docs/basics/messaging/