Test: speech extension with new SDK and updated models

bpasero commented 1 month ago

Refs: https://github.com/microsoft/vscode/issues/230660

[x] macOS @joyceerhl
[x] linux @joshspicer
[x] windows @bhavyaus

Complexity: 2

The speech extension was updated to a new major SDK version and new versions of the speech-to-text and text-to-speech models.

Setup Install the pre-release (!) version of "VS Code Speech" extension: ms-vscode.vscode-speech in version v0.11.2024091917 (pre-release).

Note: the pre-release version will only work with english as language!

Testing

test that speech-to-text is still working as before, allowing you to speak for example into the chat input and get that transcribed as input
test that text-to-speech is still working as before, allowing you to read aloud a chat response

joshspicer commented 1 month ago

Working well on Ubuntu 24.04!

(Including for completeness, not a bug on this TPI)

I also tested it on my personal machine (running NixOS). It didn't work there (can't find libstdc++).

I suspect either my OS is not loading the shared libraries in the expected spot, or a library in this extension is not searching in the right spot. The extension didn't provide an error, just greyed out the microphone after I clicked it. It also dumped this error in the extension host logs:

2024-10-22 14:52:05.940 [error] Activating extension ms-vscode.vscode-speech failed due to an error:
2024-10-22 14:52:05.940 [error] Error: libstdc++.so.6: cannot open shared object file: No such file or directory
    at process.func [as dlopen] (node:electron/js2c/node_init:2:2559)
    at Object.Module._extensions..node (node:internal/modules/cjs/loader:1602:18)
    at Object.func [as .node] (node:electron/js2c/node_init:2:2559)
    at Module.load (node:internal/modules/cjs/loader:1295:32)
    at Module._load (node:internal/modules/cjs/loader:1111:12)
    at Function.c._load (node:electron/js2c/node_init:2:16955)
    at Function.e._load (file:///nix/store/8qn2bwi0gpb4nisqkass8y1cf7cxwmaa-vscode-insiders-latest/lib/vscode/resources/app/out/vs/workbench/api/node/extensionHostProcess.js:174:10879)
    at Function.t._load (file:///nix/store/8qn2bwi0gpb4nisqkass8y1cf7cxwmaa-vscode-insiders-latest/lib/vscode/resources/app/out/vs/workbench/api/node/extensionHostProcess.js:166:22433)
    at Function.i._load (file:///nix/store/8qn2bwi0gpb4nisqkass8y1cf7cxwmaa-vscode-insiders-latest/lib/vscode/resources/app/out/vs/workbench/api/node/extensionHostProcess.js:158:4651)
    at Module.require (node:internal/modules/cjs/loader:1318:19)
    at Module.require (/home/josh/.vscode-insiders/extensions/github.copilot-chat-0.22.2024102202/dist/extension.js:195:34657)
    at require (node:internal/modules/helpers:179:18)
    at bindings (/home/josh/.vscode-insiders/extensions/ms-vscode.vscode-speech-0.11.2024091917-linux-x64/node_modules/bindings/bindings.js:112:48)
    at Object.<anonymous> (/home/josh/.vscode-insiders/extensions/ms-vscode.vscode-speech-0.11.2024091917-linux-x64/node_modules/@vscode/node-speech/index.js:8:40)
    at Module._compile (node:internal/modules/cjs/loader:1484:14)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1564:10)
    at Module.load (node:internal/modules/cjs/loader:1295:32)
    at Module._load (node:internal/modules/cjs/loader:1111:12)
    at Function.c._load (node:electron/js2c/node_init:2:16955)
    at Function.e._load (file:///nix/store/8qn2bwi0gpb4nisqkass8y1cf7cxwmaa-vscode-insiders-latest/lib/vscode/resources/app/out/vs/workbench/api/node/extensionHostProcess.js:174:10879)
    at Function.t._load (file:///nix/store/8qn2bwi0gpb4nisqkass8y1cf7cxwmaa-vscode-insiders-latest/lib/vscode/resources/app/out/vs/workbench/api/node/extensionHostProcess.js:166:22433)
    at Function.i._load (file:///nix/store/8qn2bwi0gpb4nisqkass8y1cf7cxwmaa-vscode-insiders-latest/lib/vscode/resources/app/out/vs/workbench/api/node/extensionHostProcess.js:158:4651)
    at Module.require (node:internal/modules/cjs/loader:1318:19)
    at Module.require (/home/josh/.vscode-insiders/extensions/github.copilot-chat-0.22.2024102202/dist/extension.js:195:34657)
    at require (node:internal/modules/helpers:179:18)
    at Object.<anonymous> (/home/josh/.vscode-insiders/extensions/ms-vscode.vscode-speech-0.11.2024091917-linux-x64/dist/extension.js:1:138)
    at Module._compile (node:internal/modules/cjs/loader:1484:14)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1564:10)
    at Module.load (node:internal/modules/cjs/loader:1295:32)
    at Module._load (node:internal/modules/cjs/loader:1111:12)
    at Function.c._load (node:electron/js2c/node_init:2:16955)
    at Function.e._load (file:///nix/store/8qn2bwi0gpb4nisqkass8y1cf7cxwmaa-vscode-insiders-latest/lib/vscode/resources/app/out/vs/workbench/api/node/extensionHostProcess.js:174:10879)
    at Function.t._load (file:///nix/store/8qn2bwi0gpb4nisqkass8y1cf7cxwmaa-vscode-insiders-latest/lib/vscode/resources/app/out/vs/workbench/api/node/extensionHostProcess.js:166:22433)
    at Function.i._load (file:///nix/store/8qn2bwi0gpb4nisqkass8y1cf7cxwmaa-vscode-insiders-latest/lib/vscode/resources/app/out/vs/workbench/api/node/extensionHostProcess.js:158:4651)
    at Module.require (node:internal/modules/cjs/loader:1318:19)
    at Module.require (/home/josh/.vscode-insiders/extensions/github.copilot-chat-0.22.2024102202/dist/extension.js:195:34657)
    at require (node:internal/modules/helpers:179:18)
    at __.xb (file:///nix/store/8qn2bwi0gpb4nisqkass8y1cf7cxwmaa-vscode-insiders-latest/lib/vscode/resources/app/out/vs/workbench/api/node/extensionHostProcess.js:166:23753)
    at async Promise.all (index 0)

bpasero commented 1 month ago

Yeah, the supported Linux distributions are listed on https://learn.microsoft.com/en-us/azure/ai-services/speech-service/quickstarts/setup-platform?tabs=linux%2Cubuntu%2Cdotnetcli%2Cdotnet%2Cjre%2Cmaven%2Cnodejs%2Cmac%2Cpypi&pivots=programming-language-cpp

microsoft / vscode

Test: speech extension with new SDK and updated models #231815