leave data access API concerns out, SocialAPI spec should address them

elf-pavlik commented 9 years ago

Relevant issue in SocialAPI editor's draft https://github.com/w3c-social/SocialAPI/issues/1 As mentioned there I will raise issue in official WG tracker and reference in it those two github issues.

As of now AS2.0, besides modeling social data seems to address concerns of read access to this data. I see it as responsibility of API and SocialAPI spec a more appropriate place to address it!

Paging of Collections

While as:Collection (ordered or not) itself makes sense as part of Social Data model, how to access it as smaller fragments doesn't seem as part of data model, but responsibility of API.

I propose moving Activity Streams 2.0 Core: 3.6.1 Collection Paging to Social API draft. Also all relevant terms we can reuse from IANA link relations, which soon will have normative URI prefix and better integrate with JSON-LD.

Current AS2.0 Core draft also doesn't consider write aspect of an API and doesn't guide implementer through cases where collection changes while we retrieve it's pages (which will affect ordered collections in significant way). LDP Paging provides certain guidance and SocialAPI draft should take it into consideration

Link

as:Link seems to try address discovery and access to various representation of given resource, which uses different media types. as:mediaType has only as:Link in its domain. SocialAPI spec should address issue of discovery and negotiation of particular media types of given resource. In range of concerns for Social Data to address, media types seem clearly out of scope. Person will not like some particular encoding of an image, video or audio. In most if not all social interactions, media types have no relevance. They become relevant during accessing data and in my opinion stay within responsibilities of API spec.

azaroth42 commented 9 years ago

:-1: to removing paging from the model. The method by which pages are generated and retrieved is orthogonal to the description of those pages. For example, a paged, ordered collection of objects could be embedded within a web page, included in a zip file, or used as part of another API beyond the Social API.

Secondly, there still needs to be a model to be serialized for the representation from the Social API. If the intent is to keep the social API focused on protocol level interactions, and not the exact nature of the content of the representations, then paging and collections would still need to be defined somewhere.

LDP Paging does not allow for ordering within a single page, only between pages. There are many use cases when intra-page ordering is required, and it would be problematic to use the sorting feature from Paging. Examples include when the order is dependent on the user's preferences, when the sort order is valuable business IP (consider the exact value of page rank), or when the order is determined by the query, and not a property of the resource.

[Side note: There is a normative URI prefix for IANA relations defined in RFC 4287 (https://tools.ietf.org/html/rfc4287#page-22). It's http://www.iana.org/assignments/relation/]

jasnell commented 9 years ago

-1. The Paging of Collections that is in AS2 currently deals primarily with the representation of the paging details and has nothing to do with the actual API for those pages. It's the equivalent to the distinction between use of anchor tags to represent links vs. the definition of the URI structure used within those links. In other words, the Core AS2 draft does not dictate whether or how the Collections are actually paged, only how to represent the data if an implementation chooses to do so.

Second: the WG decided quite some time ago that the AS 2.0 vocabulary should not directly depend on external vocabularies for it's core functions -- especially vocabularies that do not yet exist. I know there is work to define a standard prefix for IANA link relations but we should not hold up any part of the AS2 functionality waiting for that work to finish. The current paging property definitions in AS2 are consistent with those registered link relations already (next, prev, current, first, and last are all registered link relations). The fact that there might be a standard prefix for these things at some point in the future has no bearing on what we do with these today.

For those who may be concerned about not using a common URI prefix for these things now, there are very well defined mechanisms in RDF and OWL that can be used to easily declare equivalence so that http://www.w3.org/ns/activitystreams#next is equivalent to whatever the "official" IANA prefix ends up being.

The current AS2.0 core does not NEED to consider the write aspect of an API that does not currently exist, nor does it need to guide implementers through all of the aspects of collection management. There are very well defined best practices that have been learned over the years of implementing Atom and RSS and other things that apply equally well to AS2.0. If the API that eventually comes out of this WG chooses to go with LDP Paging, there is absolutely nothing in AS 2.0 core that would interfere with that approach.

As for as:Link. No, it does not address discovery or even access to representations of the resource. As I have explained many times before, it's intent is solely to provide the same functionality as an atom:Link or an HTML anchor tag. That is, it's an indirect reference to a resource that allows metadata about the reference to be specified. That's a completely different problem that discovery and negotiation at the protocol level.

The AS 2.0 core spec is very clear about what the as:Link is for and how it works in establishing an indirect qualified relationship.

elf-pavlik commented 9 years ago

Does it mean that adding new item to and instance of as:OrderedCollection will not change how items get divided into various as:OrderedCollectionPage which reference it with partOf?

as:items defines as:Collection as its domain, not as:CollectionPage or as:OrderedCollectionPage. Examples in core spec use as:items directly on pages, does it mean that we have as:Collections with no items but only pages, which in turn 'have items'?

I really have impression that it tires to address paging in API access scenario and not model meaningful social data. If we want nested collections, why not to use

as:Collection - hasPart -> as:Collection

?

elf-pavlik commented 9 years ago

To look at it from another perspective. If pages defined by AS2.0 have identity and meaning independent from API. To add new items:

1) In REST API, client should POST directly to URI of Page (NOT Collection linked using partOf) 2) In RPC API, Add/Create activities would use URI of a Page as value for target (NOT Collection linked using partOf)

In case of above, just nesting as:Collection in as:Collection seems to make sense, why introducing concept of a Page?

azaroth42 commented 9 years ago

I still disagree. The model for paging should not be concerned with the api for manipulating the contents of the pages. There could be 20 different APIs each of which generate pages using the same model. The two are completely orthogonal.

elf-pavlik commented 9 years ago

I question if those 20 APIs want to manipulate content of 'a page' or content of 'a collection', which has pages? As I read current spec and example, seeing pages as not API specific - if collection has pages, than it does NOT have items. If we don't intend to keep items related to Collection but instead to Page, why we don't just use Collections as items of 'parent' Collection? Nothing prevents Collection to have as:items relation with many other collections, so if having item relation to a page, does not mean having item relation with paged collection why we need this page concept?

Can we maybe discuss it, based on concrete, real world example? Foo-bar example used in spec seems very unhelpful!

azaroth42 commented 9 years ago

Who knows whether they want to change the content of a page or of a collection? That's an API detail, not a model detail. The model allows either (as far as I understand), which thus allows more use cases than picking only one of those two scenarios.

Some real use cases:

A static collection of annotations that have been broken up into pages, where each page consists of the annotations about a chapter of an ebook. The collection (e.g. set of pages) is sold by a vendor as an add-on to the ebook, as the annotations are from the author of the book. [Think DVD extras for ebooks]
A managed collection of annotations where each page is the set of best ones, determined by some voting mechanism, about a particular topic. Thus the number of pages is the number of topics, and the number of annotations per page is fixed. [Think slashdot etc]
A service that exposes millions of annotations at a single endpoint, and allows filtering by some query/search mechanism. Even still, there are too many annotations to have in a single response, so the result set is broken up into pages. [Think any search service, without a REST interface]
A REST service that has millions of annotations, but allows harvesting them based on last modified time such that a search service such as the above can be created by some other organization. [Think any harvest/synchronization service on top of a REST interface]

elf-pavlik commented 9 years ago

A managed collection of annotations where each page is the set of best ones, determined by some voting mechanism, about a particular topic. Thus the number of pages is the number of topics, and the number of annotations per page is fixed. [Think slashdot etc]

I understand that in this case the collection has no annotations as items, each of the 'pages' has annotations as items. Why this can't work as as:Collection (or as:OrderedCollection) which has as items many instances of as:OrderedCollection - one per topic, each of which has fixed number of annotations related through as:items. Why do we need 'Page' construct here if each of topics has independent ordered list of annotations? It looks like you just try to group together those independent lists. Do those topics have previous/next relation to each other?

A service that exposes millions of annotations at a single endpoint, and allows filtering by some query/search mechanism. Even still, there are too many annotations to have in a single response, so the result set is broken up into pages. [Think any search service, without a REST interface]

This one clearly mentions endpoint and response which needs 'breaking up into pages'. Which I see as API responsibility. As I understand as:Collection would have as:item relation with all the annotations from each 'page' which we get in response. Not each page having their set of annotations independent from the collection.

Do you happen to have those two examples in JSON-LD? This would provide clear reference of what AS2.0 paging tries to address and hopefully would replace foo-bar example which we currently find in a spec.

jasnell commented 9 years ago

I think you're missing the point entirely elf. The AS2 model does not care about how the content is actually split up, or why, or whether they are independent sets of items or not. It only says here is how to represent those divisions if you need to do so.

You also are straying from your originally stated concern about whether paging in AS2 is an API detail. Robert and I have both illustrated that it is not. How the paging model works is a separate concern that's already been discussed several times now. On Oct 23, 2015 12:37 PM, "elf Pavlik" notifications@github.com wrote:

A managed collection of annotations where each page is the set of best ones, determined by some voting mechanism, about a particular topic. Thus the number of pages is the number of topics, and the number of annotations per page is fixed. [Think slashdot etc]

I understand that in this case the collection has no annotations as items, each of he 'pages' has annotations as items. Why this can't work as as:Collection (or as:OrderedCollection) which has as items many instances of as:OrderedCollection - one per topic, each of which has fixed number of annotations related through as:items. Why do we need 'Page' construct here if each of topics has independent ordered list of annotations? It looks like you just try to group together those independent lists.

A service that exposes millions of annotations at a single endpoint, and allows filtering by some query/search mechanism. Even still, there are too many annotations to have in a single response, so the result set is broken up into pages. [Think any search service, without a REST interface]

This one clearly mentions endpoint and response which needs 'breaking up into pages'. Which I see as API responsibility. As I understand as:Collection would have as:item relation with all the annotations from each 'page' which we get in response. Not each page having their set of annotations independent from the collection.

Do you happen to have those two examples in JSON-LD? This would provide clear reference of what AS2.0 paging tries to address and hopefully would replace foo-bar example which we currently find in a spec.

— Reply to this email directly or view it on GitHub https://github.com/jasnell/w3c-socialwg-activitystreams/issues/221#issuecomment-150671457 .

elf-pavlik commented 9 years ago

In my last comment, I explained that while one of the examples doesn't look like API concern, it also doesn't seem to need construct of a Page. At the same time other example which seems to need Pages, looks to me like API concern, similar to one currently discussed in Hydra API mailing list and what we find in LDP Paging Note

@jasnell do you have it already implemented and deployed somewhere? We could clarify it further based on a real world example of usage.

How the paging model works is a separate concern that's already been discussed several times now.

Could you please also possibly share link to the conversation where other WG members expressed support for current model? I do understand and recognize that @azaroth42 supports it.

azaroth42 commented 9 years ago

As for example data: http://w3c.github.io/web-annotation/protocol/wd/paging.html This has yet to be merged with the current protocol draft, but did get the approval of the Anno WG. As part of that merge, I ran into exactly this problem -- I "needed" to introduce a lot of modeling discussion into a protocol document in order to describe the representation that was expected. As the use cases above show, not all of the use cases require REST or even HTTP for the paging model to be useful, and so we're looking to add as a new section in TR/annotation-model/ instead, by reference to AS2.0

Comments are very welcome on the approach --> https://github.com/w3c/web-annotation/issues/50 :smile_cat:

jasnell commented 9 years ago

No, I'm not going to go hunting for links. You have access to the same mail archives, meeting minutes and issue trackers that I have access to.

One example implementation of collection paging can be found here: http://asgh.mybluemix.net/

You can also find an implementation of the paging model in my javascript/node.js implementation: http://www.github.com/jasnell/activitystrea.ms

elf-pavlik commented 9 years ago

Thank you @jasnell, I will take a look at your implementations! Thank you @azaroth42, I will also take a look at your Annotations examples!

For now you may possibly find interesting to see how I see some of the differences between vocab describing Data Model with terms like as:Collection and vocab describing API with terms like ldp:Container or hydra:Collection. I tried to use all three of them in this example: https://github.com/w3c-social/SocialAPI/issues/2

jasnell commented 9 years ago

Looks like the bluemix server is undergoing some maintenance this weekend so that asgh link is not currently functioning.

elf-pavlik commented 9 years ago

@azaroth42 looking at http://w3c.github.io/web-annotation/protocol/wd/paging.html#multiple-ordered-responses-with-annotations

Can we first write the pure data (no API details - so no terms from ldp: namespace), and then look separately at how one might want to access it over HTTP ? For example:

{
  "@context": [
    "http://www.w3.org/ns/oa",
    { "anno": "http://example.org/annotations/anno" }
  ],
  "@id": "http://example.org/annotations/?uris=1",
  "@type": "as:OrderedCollection",
  "totalItems": 42023,
  "label": "An Ordered List of Web Annotations",
  "orderedItems": [ "anno:1", "item:2", ... , "anno:42023" ]
}

As I understand all 42023 annotations have logical relationship as items of the particular list denoted by http://example.org/annotations/?uris=1 and no distinct instances of some "foo:AnnotationsPage" which would have meaning for the actual data.

Alternative model, which might work better with paged API access ( following @lanthaler suggestion)

{
  "@context": [
    "http://www.w3.org/ns/oa",
    { "anno": "http://example.org/annotations/anno",
      "olo": "http://purl.org/ontology/olo/core#" }
  ],
  "@id": "http://example.org/annotations/?uris=1",
  "@type": "olo:OrderedList",
  "olo:length": 42023,
  "label": "An Ordered List of Web Annotations",
  "olo:slot": [
    { "olo:index": 1, "olo:item": "anno:1" },
    { "olo:index": 2, "olo:item": "anno:2" },
     ... ,
    { "olo:index": 42023, "olo:item": "anno:42023" }

  ]
}

Or

{
  "@context": [
    "http://www.w3.org/ns/oa",
    { "anno": "http://example.org/annotations/anno",
      "schema": "http://schema.org/" }
  ],
  "@id": "http://example.org/annotations/?uris=1",
  "@type": "schema:ItemList",
  "schema:numberOfItems": 42023,
  "label": "An Ordered List of Web Annotations",
  "schema:itemListElement": [
    { "schema:position": 1, "schema:item": "anno:1" },
    { "schema:position": 2, "schema:item": "anno:2" },
     ... ,
    { "schema:position": 42023, "schema:item": "anno:42023" }

  ]
}

Once we clearly understand the actual data, we can look at possible ways of paging it with API specific terms and endpoints (eg. partial views) which can use some additional URIs which have no relevance for the data we model but only for API we use to access this data.

azaroth42 commented 9 years ago

Elf, please can you move the annotation specific discussions to w3c/web-annotation#50? Thanks :)

elf-pavlik commented 9 years ago

@azaroth42 I don't see it as something specific to Annotations, I just haven't seen any non foo-bar example of AS2.0 paging requirement other than annotation one which you provided. I also do find it an interesting case which can help us clarify responsibilities of Data Model and responsibilities of API for read/write access to the actual data.

jasnell commented 9 years ago

I'm inclined to close this issue. The topics of paging and as:Link have been dealt with and discussed several times already on the mailing list and the spec as written reflects the consensus that was reached. There's been no new information presented.

akuckartz commented 9 years ago

For the record: I think that this issue will be discussed again in the near future. This WG is not the only one trying to specify paging.

w3c / activitystreams

leave data access API concerns out, SocialAPI spec should address them #221

Paging of Collections

Link