Redesigning "Relationship Annotation" Mode

[rsimon]

This is a discussion ticket for gathering requirements for a new relationship annotation mode for RecogitoJS. Add ideas below!

Problems with the current implementation

• Hairball problem. The visualization looks messy when there are many relations. It's no longer possible to properly see what's connected to what.
• Overlapping connetions can not always be selected. Also because, currently, the label (in the middle of the link) is the only selectable element.
• Design inadequate for very short links. The label might be longer than the line.
• Design inadequate for very long links. If one node of the connection is off the screen, the line essentially leads no-where. It's almost useless for navigation.
• Design inadequate for small line spacing. The old Recogito platform had a fixed line-spacing that was large enough for the links to fit "in between" the lines. RecogitoJS is embeddable, line spacing might be different - this breaks the design.

Additional needs & ideas for the future

• Should work for RecogitoJS and Annotorious (including scenarios that mix both: i.e. to connect text- and image annotations)
• Should work across multiple RecogitoJS/Annotorious instances (i.e. connections can link between different images)
• It should be possible to create a relation without having to "switch on" relationship mode explicitely, e.g. by having a handle appear on the annotation on hover
• To improve navigation, there should be a mechanims that hides (or makes partially transparent?) all relations except a currently "selected" one. (Maybe as a hover effect, etc.)

Implementation + design challenges

• Must work on touch. This needs to be considered for aspects such as the handle-on-hover (there is no hover on touch)
• Must work for complex shapes. That not only includes text annotations that cross line breaks. But also polygon/circle/etc. image annotations. What would be good handle placement spots? How and where do we attach arrows?
• Image annotation: assuming we display a handle on hover as an affordance, how do we communicate clearly to the user that this is for linking vs. the normal selection behavior (which is: click the shape to edit)

[rsimon]

Notes on Affordances

Big question: how to design "handles" that indicate that their purpose is to connect other annotations

• This might be easier for text annotations. Simple possibility: circles on top (and bottom) of the the selected rectangle. (Or on each rectangle, if the annotation crosses line breaks.)
• For image annotations, uses will have the expectation that shapes can be resized. Therefore, the linking affordance should IMO not be at the edge of the shape. A circle at the center might work better to avoid confusion.

[reckart]

Note that such affordances don't have to be permanently visible - i.e. they should probably appear on mouse hover.

[reckart]

Same for the visibility of the relations. It may not be necessary to permanently show the relations. It would be good to have an indicator that there is some relation originating or targetting a span, but the actual relation might only become visible once the user hovers the mouse over the respective span - maybe make this optional.