Request: Example Flow? - Githubissues

guillad commented 4 years ago

Hi @yhwang , Thanks for this long-awaited node. Is there an example flow you can share? My node-red crashed due to unsuccessful flow. Highly appreciated...

yhwang commented 4 years ago

[
    {
        "disabled": false,
        "id": "e6ed8a49.34ead",
        "info": "",
        "label": "Flow 1",
        "type": "tab"
    },
    {
        "ca": "",
        "caname": "",
        "cert": "",
        "certname": "",
        "id": "17135f9f.de2a4",
        "key": "",
        "keyname": "",
        "name": "",
        "servername": "",
        "type": "tls-config",
        "verifyservercert": false,
        "z": ""
    },
    {
        "id": "d93ad8ac.86c4f8",
        "modelURL": "https://ds2-tfjs.mybluemix.net/model/model.json",
        "name": "DeepSpeech2",
        "type": "tf-model",
        "wires": [
            [
                "76afc3ba.74ab24"
            ]
        ],
        "x": 640,
        "y": 280,
        "z": "e6ed8a49.34ead"
    },
    {
        "id": "e3854d50.043fa",
        "name": "",
        "type": "microphone",
        "wires": [
            [
                "8611dca7.967968"
            ]
        ],
        "x": 110,
        "y": 100,
        "z": "e6ed8a49.34ead"
    },
    {
        "id": "8611dca7.967968",
        "name": "STFT",
        "strideSize": "10",
        "type": "wav-stft",
        "windowSize": "20",
        "wires": [
            [
                "6353faee.b6475c"
            ]
        ],
        "x": 210,
        "y": 180,
        "z": "e6ed8a49.34ead"
    },
    {
        "active": true,
        "complete": "false",
        "console": false,
        "id": "76afc3ba.74ab24",
        "name": "",
        "tosidebar": true,
        "tostatus": false,
        "type": "debug",
        "wires": [],
        "x": 670,
        "y": 440,
        "z": "e6ed8a49.34ead"
    },
    {
        "func": "return { payload: { features: msg.payload } };",
        "id": "6353faee.b6475c",
        "name": "PrepareNamedTensorMap",
        "noerr": 0,
        "outputs": 1,
        "type": "function",
        "wires": [
            [
                "d93ad8ac.86c4f8"
            ]
        ],
        "x": 460,
        "y": 180,
        "z": "e6ed8a49.34ead"
    }
]

In this flow, I use:

node-red-contrib-browser-utils: the microphone node
node-red-contrib-wav-stft: pre-process the WAV with STFT
node-red-contrib-tf-model: I used DeepSpeech2 model I trained earlier. I still need a BeamSearch node to decode the CTC result from the model. but you can get the ideal of how to use the tf-model node

yhwang commented 4 years ago

You can also try to create an npm project with the following content in its package.json:

{
  "name": "test-node-red",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "start": "node-red flows.json",
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "author": "",
  "license": "ISC",
  "dependencies": {
    "node-red": "^1.0.3",
    "@tensorflow/tfjs-node": "^1.4.0",
    "node-red-contrib-browser-utils": "0.0.9",
    "node-red-contrib-tf-model": "0.1.1",
    "node-red-contrib-wav-stft": "0.1.0"
  }
}

You just run npm install and then put the flows.json that I provided above into the project folder. Then you can run npm run start to start the node-red with that flow.

guillad commented 4 years ago

Thanks @yhwang for the prompt response:-) Now it's clear how to use this node in a flow context. Thanks for that.

BTW, i tried to run the example above, with and without downsampling the audio, and I get: "TypeError: engine_1.ENGINE.makeTensor is not a function"

yhwang commented 4 years ago

@guillad what version of @tensorflow/tfjs-node are you using? and do you see any error while installing the package?

yhwang / node-red-contrib-tf-model

Request: Example Flow? #1