API for recording audio

[Glavin001]

It would be nice to add support for using Witd instead of manually recording audio. If there is interest Id be happy to submit a pull request.

[Glavin001]

I will attempt to implement this feature soon. How about for the API:

var wit = require('node-wit');
var fs = require('fs');
var ACCESS_TOKEN = "IQ77NWUPUMNBYEUEKRTWU3VDR5YSLHTA";

console.log("Using Witd to send audio to Wit.AI");

var witdHost = "http://localhost:9877";
// Make request to <witdHost>/start?autoend=true&access_token=<YOUR_ACCESS_TOKEN>"
wit.captureSpeechIntentFromWitd(ACCESS_TOKEN, witdHost, function (err, res) {
    console.log("Response from Wit for audio stream: ");
    if (err) console.log("Error: ", err);
    console.log(JSON.stringify(res, null, " "));
});

[Glavin001]

Bump Is this something of interest for node-wit? Or am I the only one who would like to be able to use witd from Node.js? If there is interest I can look into submitting a PR this weekend, potentially.

[tielur]

@Glavin001: I'm not extremely familiar with witd but is this something that could be implemented by allowing an option for the wit host to be configurable?

[Glavin001]

is this something that could be implemented by allowing an option for the wit host to be configurable

I see it as a substitute for the existing API method, captureSpeechIntent, which takes in an audio stream from file. I'd like to add method, such as captureSpeechIntentFromWitd, which takes in the witHost and other parameters.

I'm not extremely familiar with witd

I am not either, just read about it and have been planning on developing a project (Donna-ai) using Wit.ai and Node.js that I expect to run on Raspberry Pi and other devices.

For reference, here's a link to Witd: https://github.com/wit-ai/witd

My reasoning for Witd support

At the current time with node-wit we are required to install sox and manually run:

sox -d -b 16 -c 1 -r 16k sample.wav

to record an audio file and then send it to Wit with

var stream = fs.createReadStream('sample.wav');
wit.captureSpeechIntent(ACCESS_TOKEN, stream, "audio/wav", function (err, res) {
    console.log("Response from Wit for audio stream: ");
    if (err) console.log("Error: ", err);
    console.log(JSON.stringify(res, null, " "));
});

I would like to utilize witd's ability to both record and then send that audio recording to Wit for processing and then return the intent result. See https://github.com/wit-ai/witd#send-requests Instead of manually running the above command with sox or writing a node child process that executed it, I would prefer to let the already existing witd daemon handle this.

From Witd about:

We provide clients that do exactly that for iOS and Android. witd is an attempt to do the same for more devices, e.g. Raspberry Pi, BeagleBone, etc.

This means that we could easily write a Node app using node-wit that would leverage witd to record audio, transcribe from speech audio to text, and return the intent results, and skip out usage of sox completely and also have great device support (Raspberry Pi, etc).

It would be as simple as:

1. Start witd daemon
2. Configure node-wit to communicate with the witd daemon
3. Capture speech intent from witd at any time

[martinraison]

Hi @Glavin001,

Thanks for your interest in node-wit. The solution you described would work, however please be aware that witd is pretty stale at the moment, meaning that the compilation will fail due to breaking changes in the Rust language. If you're only goal is to run it on a Raspberry Pi however, then you might be fine using the provided witd for armv6 binary.

We may update witd at some point, most likely after Rust v1.0 is released, so that we know it won't break as much in the future. In this case using witd could become more useful. Of course in the meantime we still welcome any pull requests for witdt :)

Another (but more advanced) idea would be to link node-wit directly to libwit (which is the library that witd and our python/ruby sdks depend on), using something such as node-ffi. libwit provides the audio recording functionality and makes the API calls to wit.ai, while witd only wraps it in a small http server that accepts recording start/stop requests. Using libwit directly removes the need to manually start witd along the node-wit-based app and send it HTTP requests.

[Glavin001]

witd is pretty stale at the moment, meaning that the compilation will fail due to breaking changes in the Rust language.

It is unfortunate to hear that witd is becoming stale, however integrating with libwit directly sounds like a significantly better idea!

Another (but more advanced) idea would be to link node-wit directly to libwit (which is the library that witd and our python/ruby sdks depend on), using something such as node-ffi. libwit provides the audio recording functionality and makes the API calls to wit.ai, while witd only wraps it in a small http server that accepts recording start/stop requests. Using libwit directly removes the need to manually start witd along the node-wit-based app and send it HTTP requests.

This would be excellent! I agree that you second approach using node-ffi and libwit would be prefered. I notice that libwit is also written in Rust. Is libwit well maintained? I see the last commit was 7 days ago, so that's not bad. However I do see the warning message:

Compiling libwit yourself is highly experimental and will likely not work.

Regardless, this is definitely the way to go especially since that is what witd and your python/ruby sdks already depend on.

Do you have any suggestions for expected usage / configuration? I'll take a closer look at how it is implemented in the python/ruby sdks sometime.

Thank you!

[martinraison]

libwit and witd used to be a single Rust codebase, but we later split it in order to reuse the libwit part for our other SDKs. Compilation is still currently broken though (for the same reasons as witd), however as you can see on the releases page, we have prebuilt binaries for more platforms than witd. We hope to update the libwit code (along with witd) as Rust is becoming more stable. However our priority remains the regular HTTP API (which is what all our SDKs are using).

[Glavin001]

I just found https://github.com/gillesdemey/node-record-lpcm16 which uses sox to record audio in streams that is supported by Wit.ai! I hope to use Wit.ai for an upcoming project, MyoMedia, and I will submit a Pull Request if node-record-lpcm16 works! This would mean developers could simply call captureSpeechIntentFromMic (or other method name) to both record audio from microphone and stream to Wit.ai!

wit.captureSpeechIntentFromMic(ACCESS_TOKEN, function (err, res) {
    console.log("Response from Wit for audio stream recorded from Microphone: ");
    if (err) console.log("Error: ", err);
    console.log(JSON.stringify(res, null, " "));
});

[Glavin001]

@martinraison I have developed a Pull Request (see #8 ) that is working great for me. I hope you or someone else could review it and merge. Thank you!