On clicking a number line image, the region label is displayed and this is confusing for learners

[KolliAnitha]

Describe the bug On entering a mixed fraction as answer it is displayed in text and numbers after submitting

To Reproduce Steps to reproduce the behavior:

1. Go to Topic 'Fractions'
2. Click on 'Mixed numbers and the number line 1'
3. Goto exploration page Which point on this number line represents 2 1/4
4. Select the point and submit

Expected behavior After submitting the answer, it should be displayed either in numbers or in text format as it is displayed in other exploration pages as below.

Actual Result: After submitting the answer, it is displayed in the form of text and numbers

Environment Device name: One plus Nord2 5G Android version : Android 11 App version : 0.7-alpha-d1b2827517

[BenHenning]

This is a limitation in the image selection interaction, and the ideal way to solve it would be to introduce a dedicated number line interaction (which we've talked about before).

Out of curiosity, @KolliAnitha how does this behave on web?

[KolliAnitha]

@BenHenning, below is the behavior on the web.

[BenHenning]

Thanks @KolliAnitha!

So perhaps the fix here is to use a similar formatting as web.

@seanlip do you have any suggestions on the product side for how this should be addressed?

[seanlip]

I think, for a start, let's at least follow the formatting on Web, with the parentheses and quotation marks, so we have a bit of consistency.

Longer term ... this is a hard problem. I think what I lean most towards is to have a snippet of the part of the image that was clicked, with a circle around the clicked point. /cc @dianakc for thoughts as well since this should probably be consistent on both Web and Android.

[BenHenning]

For searching context, this issue was found in 0.7-alpha (MR4).

[adhiamboperes]

I think, for a start, let's at least follow the formatting on Web, with the parentheses and quotation marks, so we have a bit of consistency.

Longer term ... this is a hard problem. I think what I lean most towards is to have a snippet of the part of the image that was clicked, with a circle around the clicked point. /cc @Dianakc for thoughts as well since this should probably be consistent on both Web and Android.

I do not think that formatting the text like this to be similar to web actually solves the consistency issue -> There is a high probability that the android users don't usually use the web app.

The various number line image interaction questions in this chapter have the following outcomes:

https://user-images.githubusercontent.com/59600948/203553537-dd4061f1-4f9d-4dd0-a936-48bdc73f9658.mp4

Given some of these question display clicks on correct as the response, can we simply say “Clicks on Correct” as a way of introducing uniformity?

• Pro - simple solution, introduces consistency
• Con - does not provide reinforcing feedback like actually specifying what was clicked.

I do understand that the answer texts are derived from the region labels. Is this a difficult thing to change? cc @seanlip, @BenHenning.

[seanlip]

Hi @adhiamboperes, responses inline:

I do not think that formatting the text like this to be similar to web actually solves the consistency issue -> There is a high probability that the android users don't usually use the web app.

Acknowledged, but it at least means we (on the developer team) have a consistent mental model of what exactly is shown across both Android and Web.

I think the problem with using “Clicks on Correct” everywhere is that not all regions are "correct". (Sometimes we would give specific feedback for particular wrong answers.)

I do understand that the answer texts are derived from the region labels. Is this a difficult thing to change?

I think the correct fix would be to use the alt text, but this would require adding an alt text field to each region on Web (and then making use of that on Android). That is probably the best solution here, but it will require some work on Web before this can happen. In the interim, the solution I suggested (about snapshotting the part of the image that was clicked on) might be a reasonable approach, too -- but if that's too hard I would just suggest going for something that's consistent with Web and we can do the proper fix later once image regions have alt texts on Web.

[adhiamboperes]

In the interim, the solution I suggested (about snapshotting the part of the image that was clicked on) might be a reasonable approach, too -- but if that's too hard I would just suggest going for something that's consistent with Web and we can do the proper fix later once image regions have alt texts on Web.

I cannot say at the moment how much work it could be to do it. I will do a little bit of investigation and provide some feedback.

Additionally, wouldn't the fallback solution -

going for something that's consistent with Web

Take as much engineering effort as replacing the text with what we should actually be showing/want to show?

[BenHenning]

FWIW we do assume that a content description will eventually be added for regions, but this work hasn't yet been completed on the web side. This is meant to produce a human-readable alternative to the label for use in a11y cases, as the label is more creator-facing and often gives away the answer.

It may actually be better to consider doing away with the image region selection interaction entirely, @seanlip. While we've put work into trying to make it accessible, anything that requires visual consumption is inherently not an accessible interaction.