search

julianna-langston / chart2music

Turns charts into music so the blind can hear data
MIT License
29 stars 7 forks source link

Do you have plans for supporting other chart types? #39

Closed jooyoungseo closed 1 year ago

jooyoungseo commented 2 years ago

I was wondering if you had any plans for supporting the following chart types:

I know, we cannot cover everything at once, but it would be great if we could prioritize some chart types. FYI, our team has been working on SVG-based accessible charts for bar plot, boxplot, heat map, and scatter plot. Will share our progress as well going forward. We are also trying to keep the same json metadata format in our own module to be compatible with C2M in the long run.

julianna-ciq commented 2 years ago

Bar charts

We already support bar charts with multiple categories. Here is a stacked bar chart example. Here is a side-by-side bar chart example. The categories are effectively in separate layers, and you press Page Up or Page Down to navigate between each category.

Note that, for stacked bar charts, C2M never communicates the total of all of the bars. I can look into that.

Histograms

Sure, we can add support for that pretty easily.

Box plots

We can support box plots, but it'll be a few weeks before we can get to it.

Scatter plot and heatmap

We don't currently have plans to work on that. We need to do a more extensive literature review to see if anyone's done research into the best approach for that dense of data.

Geospatial maps

We'd consider that to be outside the bounds of what Chart2Music can reasonably cover. Additionally, this patent fairly aggressively covers sonification of geospatial data, so I'd be concerned about possible patent infringement.

jooyoungseo commented 2 years ago

Thanks for your response. Scatterplot is a quite challenging when it comes to representing point spreadness although it's easily possible to sonify its smoothed best-fit line. I have been trying to use background noise to address that. Regarding heat maps, do you think it's possible to use one more spatial audio panning sound (up and down) just like left and right stereo mode to represent the grid? I know it might require 3D audio sound output device for users to fully benefit from its 4-way-direction audio channel though.

julianna-ciq commented 2 years ago

We have released Chart2Music v1.4.0, which includes support for histograms and preliminary support for box plots.

Here is a CodePen demo with a histogram. Here is a CodePen demo with a box plot.

For box plots, we are considering this feature experimental, and may change the interactions in the future as we refine the user experience. Also, we only support the 5 statistics; we don't sonify any of the outliers.

If you want to interact with the box plot, use the left/right arrow keys to move between boxes. Use the up/down arrow keys to select different statistics.

jooyoungseo commented 2 years ago

@julianna-ciq -- This is great! I understand that the boxplot feature is still experimental, but it's pretty neat. Is there any special reason for you not to include outliers in the up/down arrow statistic cycle?

julianna-ciq commented 2 years ago

There were a lot of unanswered questions about the best way to approach it.

Because there could be several outlier values for a box plot. I feel like a good UX for that would be using the up/down arrow keys to go between each individual item, but I wasn't confident about the best way to architect that just yet. Also, do I want all outliers to appear in the same place, or should lower/upper outliers be separated out? Sometimes outliers are labelled; how should those labels be handled? What if I just had a dialog that listed individual outliers for each box, instead of trying to sonify them? If I play the full box, should I include the outliers because their relevant, or exclude the outliers because it may confuse the user about where the core of the box actually exists?

These are some of the design questions I'd like to experiment with before I try to ship a more fleshed-out box plot sonification.

Please let me know if you have any thoughts or reactions to any of these questions. I was hoping to put together some samples of different ways to sonify a box plot, and then let people try them out. Unfortunately, I haven't had time to build any of these PoCs yet.

julianna-ciq commented 1 year ago

Today, we released version 1.6.1, which includes support for heatmaps and correlation matrices. The type we use for both of them is "matrix". You can see examples of both here:

Both examples include visual syncing.

When designing the interaction model for a matrix/heatmap, we wanted the end user's cognitive burden to be as low as possible. So, we took interactions for bar and line charts, and we applied them to a matrix. When you interact with a matrix, you can use the left/right arrow keys to navigate between columns, and you can use Page Up / Page Down to navigate between rows.

julianna-ciq commented 1 year ago

We currently support 5 of the 7 chart types you asked about. The other 2, scatter plots and geospatial maps, are out of scope for us at this time.

For scatter plots, we've experimented with a couple of approaches to scatter plots, but we're not happy with the design or implementation details for any of them. For now, scatter plots will have to be on the backburner.

For geospatial maps, we consider those to be far outside the scope of Chart2Music, and we have no plans to support them at this time.