BrechtDeMan / WebAudioEvaluationTool

A tool based on the HTML5 Web Audio API to perform perceptual audio evaluation tests locally or on remote machines over the web.
https://code.soundsoftware.ac.uk/projects/webaudioevaluationtool
GNU General Public License v3.0
115 stars 38 forks source link

Two Dimensional Plane for scoring #140

Open djmoffat opened 8 years ago

djmoffat commented 8 years ago

Instead of having an APE type interface, can we make a 2d Plane, where participants can place any sample on the plane - eg. Arousal Valence

nickjillings commented 8 years ago

Like it. Very easy to do really (he says without thinking what happened the last 10 times he said that). Effectively the same standard as APE except mandatory of 2 interface nodes per page, which is sensible.

BrechtDeMan commented 8 years ago

This would be great - I requested this early on but apparently this was never tracked.

(It was part of an early WAC draft though but commented out, I assume because we couldn't implement it in time: \item APE style 2D \cite{ape}: Multiple stimuli on a 2D plane for inter-sample rating (e.g. Valence Arousal).)

nickjillings commented 8 years ago

I remember it being put in, but yes was removed from the earlier releases because of other higher priority bugs/enhancements. Probably got lost in the move from soundsoftware to here.

djmoffat commented 8 years ago

Instead of having two individual interface nodes, it would be great if it could be a single interface node that had two planes- a box where you can put each sample anywhere in the box - so that the two dimensions are clearly interconnected - does that make sense?

nickjillings commented 8 years ago

Yes, the two interface nodes however generate the two axis, giving both axis their title, reference name in the results and the scales. So both probably have to be in there for it to work properly. Otherwise it'll be a break from the standard (not against re-writting the standard but we need to then create something better).

Having two interface nodes to generate the single 2d plane is right. In terms of generation, should the elements be outside the plane and be dragged on? Or should they be randomnly distributed on the plane already? Something to consider. Equally saying both should be supported is OK, but we need a way to define this.

In terms of results file generation, will create a structure identical to a 2-axis APE file so that's OK.

BrechtDeMan commented 8 years ago

In terms of generation, should the elements be outside the plane and be dragged on? Or should they be randomnly distributed on the plane already? Something to consider.

Indeed - in which case this also applies to APE style interfaces. Would be nice to have the option to drag in somehow.

Interface nodes: double seems sensible (this is regarding the XML specification and I don't think it would have any effect on the visuals and user experience). Would be good if it was checked that a '2D plane interface' always has two nodes defined.

nickjillings commented 8 years ago

Yes, the 2D plane would always need two interface nodes to generate a plane. To avoid confusion as well, it should also only generate one plane per page (so if I put in 4 interface nodes, it will disregard the other 2, with appropriate warning messages). The test creator should also detect this and show a warning (or even an error) informing the user that there must be 2.

BrechtDeMan commented 8 years ago

Thus prohibiting multiple rating panes...

Alternatively every pair could create a pane, just like APE can have multiple sliders? Probably not worth too much effort though, and we can take it one challenge at a time.

nickjillings commented 8 years ago

I figure that could be confusing though, and if the planes are unrelated they may as well be on separate pages anyway?

BrechtDeMan commented 8 years ago

Well with the multi-axis APE the idea is that different axes are for different attributes, with the stimuli lighting up on both scales when playing. For the 2D version, two panes would mean four attributes, allowing you to rate sounds in terms of e.g. brightness, loudness, roughness, and spatial quality all at the same time.

I agree it is esoteric, confusing and probably not great practice. I would certainly never use it. I'd say don't do it unless it is a sensible extension of the standard, making the 'rules' equal for all/most interface types.

nickjillings commented 8 years ago

It's not a hard thing to change down the line either way, but would make implementation easier. Especially for dragging onto the plane. If there is one plane, that's ok, but with two how do we initialise the second plane? Or are there two dragging motions?

I think for linked axis above 2 then using APE makes more sense, as it will be more compact than a 2d plane, so possibility of examining all the axis in one frame, rather than the potential for some scrolling which may make people consider one pair of terms at one time.

Also the 2d plane may make people consider those two terms together rather than all four terms (some randomisation here may help to ensure that planes are generated differently).

BrechtDeMan commented 8 years ago

Agreed, it has those issues too. Let's not do it until a use case comes up, or until we have time to sit down and address these things.