innovationOUtside / nbev3devsim

Ev3DevSim ipywidget in Jupyter notebooks
Apache License 2.0
5 stars 0 forks source link

Clear speech synthesis buffer #8

Open psychemedia opened 4 years ago

psychemedia commented 4 years ago

Support for speech synthesis in simulator programmes is provided by the following:

import playsound
playsound.say("Hello world...")

If we cue up to many speech items, we need to kill them somehow.

If we can get how of the speech synthesiser, we can clear its utterance queue/buffer: https://developer.mozilla.org/en-US/docs/Web/API/SpeechSynthesis/cancel

The cancel() method of the SpeechSynthesis interface removes all utterances from the utterance queue. If an utterance is currently being spoken, speaking will stop immediately

speechSynthesisInstance.cancel();

The question is - can we access it to cancel it, eg via a "Shut up" button in the simulator, or pressing the simulator stop button?

Original code for my Skulpy/playsound module:

//playsound.js
function say (obj) {
    speechSynthesis.speak(new SpeechSynthesisUtterance(obj.$jsstr()))
  }

var $builtinmodule = function(name)
{
    var mod = {};

    mod.say = new Sk.builtin.func(function(obj){
        say(obj);
        return new Sk.builtin.none;
    })

    return mod;
}

//import playsound
//playsound.say('hello world')
psychemedia commented 4 years ago

It might make sense to consider all the audio stuff in a single package?

So eg some more general Javascript audio routines as part of playsound?

var context = new AudioContext()
var o = null
var g = null
document.addEventListener('DOMContentLoaded', function() {
    $(".js_play_sound").on("click", function(e) {
        e.preventDefault()
        var $target = $(e.target)
        eval($target.data("source"))
    })
    $(".js_stop_sound").on("click", function(e) {
        e.preventDefault()
        o.stop()
    })
}, false)

function example1() {
    o = context.createOscillator()
    o.type = "sine"
    o.connect(context.destination)
    o.start()
}

function example2() {
    o = context.createOscillator()
    g = context.createGain()
    o.connect(g)
    g.connect(context.destination)
    o.start(0)
}

function example2Stop(decreaseTime) {
    g.gain.exponentialRampToValueAtTime(0.00001, context.currentTime + decreaseTime)
}

function example3(type, x) {
    o = context.createOscillator()
    g = context.createGain()
    o.connect(g)
    o.type = type
    g.connect(context.destination)
    o.start(0)
    g.gain.exponentialRampToValueAtTime(0.00001, context.currentTime + x)
}

function example4(frequency, type) {
    o = context.createOscillator()
    g = context.createGain()
    o.type = type
    o.connect(g)
    o.frequency.value = frequency
    g.connect(context.destination)
    o.start(0)
    g.gain.exponentialRampToValueAtTime(0.00001, context.currentTime + 1)
}

And then in a notebook: Javascript(tone+"example3('square', 1.5)") or Javascript(tone+"example4('50', 'sawtooth')"), for example.

psychemedia commented 4 years ago

Could we create a speechSynthesis() object on the simulator and then use that?

eg var speaker = window.speechSynthesis() and then from var utterance = new SpeechSynthesisUtterance(obj.$jsstr()) call speaker(utterance).

This should then give us a queue we can cancel against and also an object we can select the voice of?

psychemedia commented 1 year ago

Support for various audio features were introduced into the original ev3devsim project via this commit: https://github.com/QuirkyCort/ev3dev-sim/commit/7783aef5f0b3b0c42ca8fd46dea97e436dbc7a63