Results from Parallel Coordinates Implementation and User Testing

[anobel]

I'm a research fellow in urologic oncology at the University of Washington, and our group has been using the htmlwidgets version of the parallel coordinates library (https://github.com/timelyportfolio/parcoords) by @timelyportfolio to create a diagram of a validated risk prediction tool for patients with metastatic renal cell carcinoma. Each line represents a specific patient with metastatic renal cell cancer, the axes represent their clinical data, and the right most axis shows survival times. Using R, htmlwidgets, Shiny, and the parcoords package, we're able to display calculated median survival times based on the selections the user makes.

A working prototype is at http://faculty.washington.edu/odisho.

We also performed usability testing with 400 non-medical users, and I wanted to share the results here. In discussing with @timelyportfolio, it seems some of the changes we would like to implement in response to user comments can only be made in the javascript source code, not on the R/htmlwidgets side.

We went through and collated over 500 comments, and some of the most common responses were:

1. It would be helpful if there was an option to input values (or upper/lower limits) into a text box, as opposed to clicking and dragging.
2. Users wanted to see what they were selecting, ideally in real time. So as I'm selecting calcium levels between 4 and 8.5, I can see when I'm hovering about the 4, and as I drag a counter shows me how far I'm going until I hit 8.5. Users felt this would allow for more precise navigation. (this is what I meant by the "visualize slider cutoff" comment in the spreadsheet). this is tangentially related to issue #270.
3. For fields where the choices are not continuous (treatment, nephrectomy y/n) check boxes, or some way to click to select an option, would make the tool more user-friendly. Having check boxes for treatments would also be an easy way to select multiple treatments that are not next to each other.
4. Users expressed frustration with difficulty clearing their selections. This was surprising since it seems pretty easy to just click off of the boxed area to clear it, but there were enough comments on it that I thought I should highlight it nevertheless. Other suggestions along this vein were to have an option to clear all entries, as opposed to the option to reload the whole page, in order to cut down on loading time.

I have more detailed results and all of the specific comments, if anyone is interested. I think solutions to 2 and 3 would be highest priority. Also, apologies for the long post and not having each of these as separate issues, wanted to keep the context/explanation in one place.

[BroHammie]

1 and 2: What do you think about http://bl.ocks.org/mcwillso/b27021c28762c1dea2da 3: You could use the 'Multi Brush' mode, but this might cause issues with the solution to 1 and 2. 4: In the brushing example there is a 'Reset Brushes' button that clears the brushes vs what you have that reloads the page. d3.select('#btnReset').on('click', function() {parcoords.brushReset();})

[anobel]

In the example you linked to, I think showing the ranges of the current selection at the top/bottom of the axes is elegant but could be confusing. what do you think about a tooltip? Either way, it would be nice to have values rounded. Not sure how it would work in situations with categorical variables (3.5 cylinders doesn't make much sense)

In regards to entering numbers directly by clicking on numbers at top/bottom of axis, it seems non-intuitive for lay-users. It may not be possible to do it cleanly in the diagram and maybe have the diagram take input from separate entry boxes?

3: I tried multi-brush, but I think the issue is users were confused when it came to categorical variables. They didn't understand why they had to select/drag a range around a point when it made the most sense for them to just click that point. For example, when selecting cylinders, why not just be able to click/select 6, instead of brushing a range from 5.5-6.5 or whatever that encompasses 6.

[matanox]

@mcwillso would you possibly be able to point me at the code changes that you performed in your fork to create this spike at http://bl.ocks.org/mcwillso/b27021c28762c1dea2da? I somehow got lost on the commit log... it's interesting to see safe ways of tweaking the core component.

[BroHammie]

@matanster https://github.com/mcwillso/parallel-coordinates/commits/Column-Fields is a good starting point.

[matanox]

@mcwillso thanks, I will look at that branch https://github.com/mcwillso/parallel-coordinates/commits/Column-Fields

[tbadams45]

@anobel slightly off-topic to this issue, but have you released the code for this shiny app you're working on? I would be curious to see how you calculate the median values in Shiny. I'm currently working on another Shiny app where I'm hoping to link additional views to the results of a parcoords plot, and it seems like that's what you've been able to accomplish.

[timelyportfolio]

@tbadams45 this https://github.com/timelyportfolio/parcoords/blob/master/inst/examples/shiny/app.R might help. I'll try to work up an example to specifically calculate median.

The crosstalk branch is highly experimental and subject to change, but here is an example with it https://github.com/timelyportfolio/parcoords/blob/feature/crosstalk/inst/examples/examples_crosstalk.R

[timelyportfolio]

@tbadams45, probably better to move discussion to this just-reopened https://github.com/timelyportfolio/parcoords/issues/15