Markit and Talkit: A Low-Barrier Toolkit to Augment 3D Printed Models with Audio Annotations

As three-dimensional printers become more available, 3D printed models can serve as important learning materials, especially for blind people who perceive the models tactilely. Such models can be much more powerful when augmented with audio annotations that describe the model and their elements. We present Markit and Talkit, a low-barrier toolkit for creating and interacting with 3D models with audio annotations. Makers (e.g., hobbyists, teachers, and friends of blind people) can use Markit to mark model elements and associate then with text annotations. A blind user can then print the augmented model, launch the Talkit application, and access the annotations by touching the model and following Talkit's verbal cues. Talkit uses an RGB camera and a microphone to sense users' inputs so it can run on a variety of devices. We evaluated Markit with eight sighted "makers" and Talkit with eight blind people. On average, non-experts added two annotations to a model in 275 seconds (SD=70) with Markit. Meanwhile, with Talkit, blind people found a specified annotation on a model in an average of 7 seconds (SD=8).
視覚障害者がインタラクションできる３Dモデル（本研究では地球儀で触った地域のデータを読み上げてくれるもの）を簡単に製作できるツールとワークフローを提案

3Dプリントするモデルにアノテーションをするツール（Markit）とCV技術を使って触れている場所を特定し，アノテーションされたデータを音声で読み上げるアプリ（Toolkit）の二種類のツールを提案
事前に視覚障害者が３Dモデルとどのようにインタラクションするのが適切か調査した
調査結果を元にMarkitをデザイン

触ったところにアノテーションされたデータを読み上げるものは世に出ているが，作るのが大変だし専門知識が必要だったので，誰でも簡単に作れて，使えるツールを提案した点

SeitaKayukawa / Blind_Accessibility_Papers