Allow >1 role per resource

[azaroth42]

TextualBody and SpecificResource say that there may be 0-1 roles, however there are use cases where multiple roles would be valuable:

• Tagging + clarification. For example tagging + describing, to distinguish from just describing where the body is full text.
• Broad + specific. For example editing + ext:typoReplacing, to have a general category and a more specific, machine actionable role that would only be understood by limited implementations
• Conciseness. If the same resource really plays two roles, it would be strange to have it as two bodies rather than just giving it two roles. For example, the text of the body may both describe and identify the target.

Proposal: Change 3.2.2.2 and 4.1 to both be MAY have 1 or more roles

[BigBlueHat]

:+1: this brings roles closer to motivations--which I think makes the usage clearer...if I can have > 1 motivation for an annotation, why can't I have > 1 motivation for a body or target?

Also (editorial nit), we should like to the motivation list from the roles section: http://w3c.github.io/web-annotation/model/wd/#h-creation-reason

[iherman]

Works for me.

On 6 Nov 2015, at 20:40, Rob Sanderson notifications@github.com wrote:

TextualBody and SpecificResource say that there may be 0-1 roles, however there are use cases where multiple roles would be valuable:

Tagging + clarification. For example tagging + describing, to distinguish from just describing where the body is full text. Broad + specific. For example editing + ext:typoReplacing, to have a general category and a more specific, machine actionable role that would only be understood by limited implementations Conciseness. If the same resource really plays two roles, it would be strange to have it as two bodies rather than just giving it two roles. For example, the text of the body may both describe and identify the target. Proposal: Change 3.2.2.2 and 4.1 to both be MAY have 1 or more roles

—

[iherman]

Closed by virtue of telco discussion: http://www.w3.org/2015/11/18-annotation-irc#T16-15-49

[iherman]

Reopening per Doug's request on the call http://www.w3.org/2015/11/18-annotation-irc#T16-22-16

[BigBlueHat]

@shepazu could you restate your reasons for wanting role to be a singular, textual body vs. an array of values? Thanks!

[shepazu]

While I'm sympathetic to the rationale, this seems like it would make the model more complex, less efficient for processing, introduce possible conflicting behaviors, and decrease interoperability.

1) Structurally, this is explicitly more complex. Complexity's not necessarily bad, but it's a tradeoff that doesn't seem to be worth it (at least without more concrete and pressing use cases).

2) If the UA is treating different roles/motives with different presentations or functionality (e.g. the copy-edit use case), it's already a burden to have to dig into each body to get the role; forcing them to treat each role as an unstructured array in which different values can be in any order, and to scan for each type of role value in each body is going to be inherently hard (e.g. computationally expensive) to process.

3) If there are multiple role values, where different UAs may process and present them each uniquely, there is a strong chance that some combinations of roles may conflict with each other. For example, if a client presents commenting and editing and `tagging' in 3 distinct ways, how should it present a single body that had all 3 role values?

4) Sometimes, the best thing a spec can do for interoperability is to limit options, leaving only a single clear way to do it right. If we increase the options on what role value to add to single the intent for any particular body, there is likely to be a profusion of different combinations, meaning different UAs will have to deal with non-specified combinations that each generator arbitrarily assigns. There should be a single minimally-ambiguous role value for each type of body, such that interoperability emerges between different annotation systems.

We are still in the early phases of this annotation model, and there is very little real-world experience for UI presentation for these different role values; to the extent that they are backed by real-world examples (e.g. in Hypothesis, Twitter, Facebook, MS Word, etc.), many of these role values already have distinct presentations. I expect that over time, patterns for presenting many of these role values would emerge, improving the user experience. But if we let UAs mix different role values, there won't be an opportunity to find how each is best presented (or just processed) on its own.

If real-world experience doesn't bear out this distinct processing, there could be an opportunity to allow multiple role values in some future version of the spec, but I suggest we defer this to a later version, even if we don't close it outright.

If we find common combinations of role values that would elicit specific processing and presentation behavior, then we can mint new role values that address both sets of needs simultaneously.

[azaroth42]

1) Concrete use cases are given in the issue, and were brought up by @paolociccarese as an implementation requirement. So I disagree with your first point. Unless you intend to refute those provided use cases?

2) Agree that processing an array is more expensive than switching on a string. However I think that switching on many strings (and note that they're not /actually/ strings, they're URIs that identify instances of the oa:Motivation class) is harder than processing an array of enumerable options. All three use cases in the issue would require a combinatorial explosion, which is worse than combining as needed in an array.

3) Don't do that then. We can't predict what everyone will need, and we should not try to. If someone creates an annotation with a body that has two roles that the UA doesn't know what to do with, then it won't display properly, and it'll get fixed. No one likes broken windows when it comes to their own content. Secondly, and more relevantly, the number of annotations that are writen by hand without a UI is going to be vanishingly small. Creation UAs won't permit conflicting combinations, and hence there won't be a problem for Display UAs. Likely those UAs will be fill both roles (but not always).

4) We haven't done that so far, otherwise we would not have string literal bodies and we would not have hasRole on TextualBody, we'd require a SpecificResource. So yes, I agree with your point ... and if we can agree on that as a design principle and follow through with it, then I think the model would be much more coherent and elegant. But at the moment, that's not the case.

I also disagree with:

We are still in the early phases of this annotation model...

We have had a lot of experience with it, and there are many hundreds of millions of annotations out there using OA. Just yesterday we got 500,000 of them from a partner on a small project. The use in IIIF at the Internet Archive alone is significantly greater than 10 million. In the IIIF space as a whole there's greater than 100 million annotations. Combined with other implementations and there's probably over a billion annotations in the OA model. It has been worked on consistently for more than 5 years, and has global adoption.

[shepazu]

@azaroth42 When I said we don't have a lot of UI/UX experience in annotations, I didn't mean that there isn't a lot of content generated by existing UAs; clearly there is. I specifically meant that there's not a lot of content that's been designed with the new role-on-body model we just published, and we don't have a lot of experience with that specific factor.

[shepazu]

@azaroth42

1) Concrete use cases are given in the issue, and were brought up by @paolociccarese as an implementation requirement. So I disagree with your first point. Unless you intend to refute those provided use cases?

By concrete use cases, do you mean "tagging + describing", "editing + ext:typoReplacing", and "describe + identify"? None of those seemed compelling enough to overcome the downsides of added complexity, additional processing requirements, and risk of non-interop. In some cases, you could simply add tags to get the additional nuance you're looking for, especially in the case of the motivation extension ("ext:typoReplacing").

[BigBlueHat]

Tagging is not an extension mechanism...it's tagging. :smiley: Plus, the magic string "ext:typoReplacing" in a body with the role "tagging" doesn't really prep for interop anything...especially not copy-editing.

Additionally, we already have Appendix D for extending motivations--which is the only proper value of oa:hasRole--so that piece is already done regardless of having one or more than one role expressed.

Perhaps putting some code (of various kinds) on the two scenarios (role as string vs. role as string or array of strings) could shed some light on the actual cost of this proposal to JSON developers. Graph consuming folks won't be effected by this, afaik...at least not negatively. :smile:

[shepazu]

@BigBlueHat Okay, I don't necessarily agree that tags can't be used for a similar purpose to motivations (since tags are sometimes used that way in various media), but that's orthogonal to my point, which is that I don't feel the added complexity is justified by the use cases, at least not as they are stated in the issue thus far.

[akuckartz]

Allow >1 role per resource

:+1:

[iherman]

As a user I would think that restricting the number of roles to zero or one is too much of a restriction. In any similar situations (tagging, usage of ARIA role attribute; in some sense assigning a class to a term in RDF) the fact of being able to add several such labels seems natural for me.

As an (imaginary) implementer I of course see the additional complication. However... in my experience, in everyday usage of Javascript (and, as an extension, in models using JSON) this type of type overloading is fairly widespread. (We have faced similar issues in the case of the CSV Metadata vocabulary.) Javascript programmers are pretty much used by such checks and branches.

We should give higher priority to the user expectations; within reasonable bounds implementers will follows. Ie, I would prefer to allow possible more than one role per resource.

[shepazu]

@iherman Of course I agree with your reasoning to use the Priority of Constituencies (users over authors over implementors over specifiers over theoretical purity), but I differ with your conclusion.

As @azaroth42 points out, users aren't going to hand-code annotations [1]. Unlike tagging, where users can select from a pre-set vocabulary or even write in free-form tags, users won't be explicitly selecting the annotation motivation, they'll be selecting it from UI options, which are most likely to be modal (e.g. you either select the highlight tool, or the comment tool, or whatever other tool; for variations (like editing), you might type into the input field for some aspect, or leave it blank. Based on which tool the user selects, the UA picks the most appropriate motivation. (If the motivation choice isn't clear, then that's a problem with which set of motivations we've define, which is why I raised #113 ).

So, the question is, what behavior best meets user needs, each body or target having multiple motivations or a single one? You suggest that users will always select "more options"; I think it's more likely that users want simple choices (or better yet, to not have to think at all about the options at this level), to make it easier to create an annotation, and also will want their annotations to work the same across different annotation UAs. Both of these considerations could argue for a single value, which is simpler and more likely to achieve interoperablity.

Obviously, we could construct other arguments around "which is better for user experience"… but we don't have real data. I suggest that once we introduce multiple values for role, we are not going to have the option to step back out of that once we have real data from a variety of UAs; whereas, if we constrain it to a single value, we could revisit that decision in v2, if some evidence shows that we really do need it.

I like @BigBlueHat's suggestion that we write up examples of both models, and try to test the performance characteristics.

[1]: "the number of annotations that are writen by hand without a UI is going to be vanishingly small". (I'm not sure why he though I was asserting that people would hand-code annotations… I was referring to different annotation generators, not people, making different choices about combinations.)

[iherman]

On 21 Nov 2015, at 23:36, Doug Schepers notifications@github.com wrote:

@iherman https://github.com/iherman Of course I agree with your reasoning to use the Priority of Constituencies (users over authors over implementors over specifiers over theoretical purity),

O.k. This means that we should not argue about the complexities of the UA-s (because their increased complexity is minimal, ie, that is not really what counts). The only thing that really matters is what user may or may not prefer; UA-s will adapt. but I differ with your conclusion.

That is fair. :-) As @azaroth42 https://github.com/azaroth42 points out, users aren't going to hand-code annotations [1]. Unlike tagging, where users can select from a pre-set vocabulary or even write in free-form tags, users won't be explicitly selecting the annotation motivation, they'll be selecting it from UI options, which are most likely to be modal (e.g. you either select the highlight tool, or the comment tool, or whatever other tool; for variations (like editing), you might type into the input field for some aspect, or leave it blank. Based on which tool the user selects, the UA picks the most appropriate motivation. (If the motivation choice isn't clear, then that's a problem with which set of motivations we've define, which is why I raised #113 https://github.com/w3c/web-annotation/issues/113 ).

So, the question is, what behavior best meets user needs, each body or target having multiple motivations or a single one? You suggest that users will always select "more options"; I think it's more likely that users want simple choices (or better yet, to not have to think at all about the options at this level), to make it easier to create an annotation, and also will want their annotations to work the same across different annotation UAs. Both of these considerations could argue for a single value, which is simpler and more likely to achieve interoperablity.

Obviously, we could construct other arguments around "which is better for user experience"… but we don't have real data. I suggest that once we introduce multiple values for role, we are not going to have the option to step back out of that once we have real data from a variety of UAs; whereas, if we constrain it to a single value, we could revisit that decision in v2, if some evidence shows that we really do need it.

The way I could summarize what you say is that.. we do not really know. We are all trying to make conjectures on what we believe the users would really use (and, thereby, how UA-s will adapt to those needs) and your conjecture is different than mine (more exactly of those who were in favour of allowing for more than one role, including me).

Where can we go from here? I see two possible alternatives:

1. Restrict the functionality to zero or one, and try to seek feedbacks later, with a possible extension in version 2 of the model later (although, at this moment, we do not know whether such a version 2 would ever exist so I am not sure it is wise to plan for it this way already now).
2. Allow for the more 'liberal' approach, i.e., allowing any number of roles, and mark this features explicitly as an 'At Risk' feature in our document, which means that we are asking for the community (implementers and potential users) to comment on this issue. (This is quite a standard way of operating, we can keep it At Risk up to Proposed Rec if we wish.)

I am in favour of the second approach. I presume you will favour the first one...

I like @BigBlueHat https://github.com/BigBlueHat's suggestion that we write up examples of both models, and try to test the performance characteristics.

Right. We could go down that road, too, but, as I said, I do not really believe the issue is on performance, it is, instead, on user expectation and usage. Which makes it very difficult to measure and to make really informed decision in a short amount of time.

Why 'short'? The current WG plan is to provide a kind of a 'frozen' version of the model sometimes very early next year. The current version of the model has been considered as being, 'essentially', final except for the multiplicity issue, and that is necessary in view of our further plan to be CR-ready around April next year, and in view of the much more complex open issues in the Protocol and the FindText documents. Compared to those, and compared to the WG-s remaining timeline, I do not consider this issue is so utterly essential, i.e., I do not believe that we should put a major amount of work in testing user expectations (which I am not even sure how we would do). That is also why I am in favour of the second approach above, which would allow us to call out for the community for feedback but would also allow us to move on in a timely manner.

[1]: "the number of annotations that are writen by hand without a UI is going to be vanishingly small". (I'm not sure why he though I was asserting that people would hand-code annotations… I was referring to different annotation generators, not people, making different choices about combinations.)

[BigBlueHat]

Here's the current list of specified Motivation (and therefor Role) values:

• bookmarking
• classifying
• commenting
• describing
• editing
• highlighting
• identifying
• linking
• moderating
• questioning
• replying
• reviewing
• tagging

Here's an example of using just these to do a common thing using multi-roles on bodies:

{
  "type": "Annotation",
  "motivation": "bookmarking",
  "target": "http://w3.org/",
  "body": [
    {
      "role": ["tagging", "identifying"],
      "text": "W3C"
    },
    {
      "role": ["tagging", "classifying"],
      "text": "homepage"
    },
    {
      "role": ["commenting", "describing"],
      "text": "The home page of the World Wide Web Consortium"
    }
    {
      "role": ["commenting", "reviewing"],
      "text": "Fabulous! Would spec here again!!1!"
    }
  ]
}

These could all be made via a form not dissimilar to Hypothes.is form, but with the ability to add multiple inputs where I could also "explain" (by picking a motivation) what this additional field might be about / for.

If we constrained the role field to a single value, then I've no way of understanding what I'm getting in the value of text nor what it's intended use to others might be (reviewing, describing, etc.).

If we constrain it's use to a single value, then the expectation becomes that these values are actually "types", but that's clearly not what's being provided here--nor should it be.

The main reason we have Motivation at all (vs. types, "hard" RDF classes, etc). is that it's hard to know when something's "displacing"--and stating that "for the world" isn't going to make us any friends--given that I can obviously be describing (or reviewing or editing etc) a resource with a tag, a comment, a highlight, an image, etc.

Hopefully the above example JSON points to a(nother) valid use case and scenario that could be implemented (with relative ease) consumed with some confidence, and make meaningful value of multiple roles--where the inverse (having only 1 role) would be confusing, less valuable, and ultimately damaging to the use of a thing called Web Annotation.

[azaroth42]

I (likely unsurprisingly) prefer the liberal option. No implementation is going to support 100% of the features. If an implementation will only process one role, then that's very unlikely to be a significant problem (IMO). Compare to, for example, not processing HTTPRequestState, where you would end up with the wrong representation. Or not processing the only Selector associated with a SpecificResource, and thus not know where to anchor the selection to.

I agree with @iherman about time being important and focusing on making progress. We have a lot of much bigger decisions to make than this, on all of our deliverables.

[azaroth42]

I take back the issue as a technical one, it is merely editorial.

3.2.2.2 says:

There should be exactly 1 role for a Textual Body,

But does not say that there MUST NOT be more than 1. There SHOULD be one, but there may be 0 or more than 1.

Similarly, 4.1 says:

There may be exactly 1 role associated with the SpecificResource,

But also does not say that there MUST NOT be more than 1. We also don't say that there MAY be 0, as like above, it's unnecessary.

I think the action is that we need to clean up the cardinality requirements across the board.

[jjett]

+1 for cleaning up the cardinality requirements.

I'd also like to add an observation that I thought one of the use cases that "roles" (or purposes) and motivations are intended to service is retrieval of annotations by "kind", i.e., there're like fuzzy classes. E.g., one can search for all "editing" annotations or annotation bodies and get them back from a search feature.

IMO, it's a information retrieval (IR) system design best practice to allow for more than one "kind" to be asserted as it allows relevant (and probably even pertinent) search results to returned for multiple kinds of queries. So I'd be -1 for either motivation or "role" / purpose (whatever we finally decide to call it) to be 0 or 1 properties as it'll be a serious drag on search functionality.

[fhirsch]

'exactly' suggests 1 and only 1. We should change it to say "should be one or more roles"

[azaroth42]

Done: http://w3c.github.io/web-annotation/model/wd2/#motivation-and-purpose and http://w3c.github.io/web-annotation/model/wd2/#purpose-for-external-web-resources