Support multilingual discussion using topic based dictionary service

[SteveALee]

DART would like to see the ability for 2 people looking at media on a shared device to have a discussion when they do not share a language. The loose thoughts are this would allow one persons text input to be converted to another language and displayed or spoken with TTS

This feature would access the Mtral EU project online service that provides Digital Grammars. DART would develop a new lexical resources for new grammer appropriate for Brian usage

There is a MOLTO example for travel conversations.

@OpenDirective/brian-collaborators

[SteveALee]

Digital Grammars are providing the dictionary.

• We need to know how to technically access the grammar service from Brian.
• We also need to defined the user experience we want.

Both can be answered on this issue discussion

[sancarder]

Steve, I made a sketch on the interface for this with Balsamiq (which is a great tool!), which I will send to you. I'm curious about the speech synthesis function. How will it work? Where is it coming from (the device, the server?), and which languages could we support do you think?

[SteveALee]

Thanks @sancarder !

Please see https://github.com/OpenDirective/brian/issues/24 for TTS

Once we have collected all features that impact browser choice we can make a decision

[SteveALee]

I made a sketch on the interface for this with Balsamiq (which is a great tool!), which I will send to you.

@sancarder you can drag files into the GitHub comment editor so we could try that first.

[sancarder]

Ok, great, @SteveALee, I'll try that once I've finished them up a bit :)

[SteveALee]

The following suggestion by @sancarder is copied from #27

This is the part with multi language functionality and symbol support. Starts with a grid with topics, e.g. "Outside", "In the kitchen", "Traveling". Each button goes to a layout with a visual scene of this topic. The visual scene has hot spots that the user can click. When clicking a hot spot, text in one language goes up to a message window, and the text is spoken out in another language. Placed around the VS are buttons with symbols and text that could be used to express even more words (e.g. "like", "want", "more", or specific words for the VS). This should also be possible to hide/show to tailor the needs of the user. Materials in this module is static and can't be edited by the user.

What do you think about this? Could it be done? I attach a few wireframes I have made of the DART module and the Start page, so you get a view of it. Visual Scenes - carer.pdf Visual Scenes - main.pdf Visual Scenes - user.pdf

[SteveALee]

Yes this is achievable, but requires some new features. Need to discuss and select milestone

• hotspots - with full accessibility
• More complex layout for extra buttons eg 'want'

[SteveALee]

@sancarder Are there any standard formats for adding hotspots to images? What about editors?

[sancarder]

@SteveALee - I have never done this programmatically, just used it in existing softwares, so I'm not sure about the technicial specifications here. Could you clarify what you are asking about editors?

[SteveALee]

Could you clarify what you are asking about editors?

Well we need to define a hotshot. How have you done it previously. My expectation is a hot spot editor tool will let you interactively select and alter hotshots for an image. It would be good if we could use an existing tool. Otherwise we have to create it

[sancarder]

Couldn't you just simply put smaller images with links above the larger image, and make them invisible? That way, they would react to a click, but not be seen. I think that is a way to do it in some softwares, but it might be an ugly solution, and perhaps there are more built-in functions for this somewhere.

[SteveALee]

Ah but you still need to position those hotshots some how :)

[SteveALee]

see also #48