search

andrewn / neue-radio

Neue Radio: Prototype connected object using web technologies
10 stars 6 forks source link

Speak emoji descriptions #64

Closed andrewn closed 6 years ago

andrewn commented 6 years ago

This replaces emoji in text sent to our speech service with the emoji's description. The expanded text is then sent on to the speech server to be spoken.

The emoji-describe module is used to do the replacement. Is uses the name of the emoji from the unicode standard.

For example: "Good luck 🤞"

Is expanded to: "Good luck hand with index and middle fingers crossed"

It will also expand skin tone variations based on the same language that Apple uses:

For example: "Good luck 👏🏽"

Is expanded to: "Good luck clapping hands sign with medium skin tone"

Disabling emoji replacement

The emoji expansion can be disabled by sending a message with a payload containing { replaceEmoji: false }.

pixelblend commented 6 years ago

This is great! Is there a way to express a different tonality when speaking the emojis?

e.g. in macOS say, you can add emphasis to certain words

andrewn commented 6 years ago

That's a really nice idea.

speechd supports SSML MODE, which is a markup language that supports things like <emphasis> but it does say "here is no guarantee that the SSML markup will be respected, so the application shouldn’t rely on them."

Being able to encode markdown-like emphasis in utterances would be very nice for example "Good *luck*, 🤞" and have the * be translated into an emphasis. Then we could also use this to add tonality to the emoji expansion.

I've created issue #65 for this as I think it's a bigger task.

andrewn commented 6 years ago

@pixelblend Are you happy with what's here? think adding emphasis is another task?