Dynamic refreshable tactile graphics for use in digital source files

[GeorgeKerscher]

Now that graphics can be displayed on the latest generation of tactile displays, the delivery of tactile images with digital source files should be supported.

Problem statement: Extended descriptions (long descriptions) of graphics are now being distributed as part of the digital source files. These are primarily text, tables, or MathML. It should be possible to also provide a tactile graphic that could be used by a person with one of the new tactile displays used for presenting graphics. However, there is no standard for the delivery of these graphic files.

Today there are two recommended techniques for including extended descriptions: Use the "details" element below an image to enclose the extended description, or place a link below an image to the extended description somewhere in the digital version.

For example, using the details element, the extended description is placed in side the details. The screen reader announces on the image, "has details" and the details element can be expanded to present the extended description. The same technique could be used to enclose a tactile image. We believe this could be done today with an SVG, but we do not know if an SVG would be suitable for tactile graphics. Most certainly guidelines would be needed to support whatever tactile graphic file format is identified or developed for these devices.

[avneeshsingh]

@GeorgeKerscher is the use case mainly focused on identifying and promoting or may be facilitating development of a suitable tactile graphics standard?

[GeorgeKerscher]

Yes, IMO a standard for tactile graphics targeted at the new generation multi line devices would be used by organizations providing a range of formats for learners. The ImageShare approach.

[wfree-aph]

Meeting: Thinking that this could be an independent specification. Also suggested that the information necessary could be included in the metadata of the img tag. What is the minimal viable graphic that we can include in the file format and that's tactile graphics made by an expert- also mentioned monochrome, not saleable, dot-by-dot. Could also have multiple file formats. Also mentioned to leave it to the reading system and not limit the file types and include metadata and labels in the file type. Also need for fallback position.

Needs to be split into multiple use cases: showing alt text, different file types, and specification for handling image files- and potentially more.

[andrewflatres1]

SVG will not be a good file format option giving the resolutions will be high. For multi-line and tactile displays that can show both braille spacing and tactile graphics, a low resoltion would be best suited. Jpeg for example would be ideal for displaying images but will struggle to capture any braille literacy. PRN file format which has been used in the past only caters for double density, which will cause jumbo braille on refreshable braille displays, so if PRN is an option we need to include single density. PDF on the otherhand is the most popular format in education. It should be up to the reading device to covert this. I agree that we should not limit the file format, as it is important to have these options giving the reading device.

Note, that we should not limit the alt text description for when single line is only seen. For tactile and multiline displays, it will be required that the user can switch between the actual image and text description which will allow any onboard TTS to grab. It would also be requird for deaf and blind.

[mrhunsaker]

I agree with @andrewflatres1 regarding svg being a challenging format. However, I feel we need to discuss what vector-based format would work. From my work with OCR, I have conerns with bmp, png, and jpeg formats as they often result in images that have noisy backgrounds that I fear would cause erroneous braille signal in a graphic (though png can work well given a background alpha channel so the background is not spotty and providing inappropriate braille input)

Optimally, one would be able to "zoom in" on a graphic; I think of the example the DotPad and TactilePro are showing of a typical eukaryotic cell. I also think we may look at the DotPad and whatever formatting they have in place and whether they are relying heavily on on-device AI or if they are using vectorized images to provide diagrams/graphics that can be resized.

[nicolegaines]

Given that the digital tactile graphics piece is complex and will require additional consideration at a later time, I would suggest we include some type of placeholder in the eBRF specification that reflects the assumption that graphics will be vetted to ensure that they meet a minimum standard of viability. The issue I am anticipating here is confusion regarding the ability of braille displays (or even embossers) to adequately handle image formats imported wholesale into the eBRF format from a source document, without any further review or editing. I think it would be helpful to state at the outset that the expectation given available technologies at this time is that human intervention will be required in most cases in order to create meaningful tactile graphics from source images found in visual media.