V-Sekai / godot-whisper

An GDExtension addon for the Godot Engine that enables realtime audio transcription, supports OpenCL for most platforms, Metal for Apple devices, and runs on a separate thread.
MIT License
48 stars 5 forks source link

Could not find base class "SpeechToText" #76

Closed snorreks closed 2 weeks ago

snorreks commented 2 weeks ago

The SpeechToText class is not found in gd script or when trying to create a new node.

Plugin version: 1.1.0 (release tag from download says 1.0.5, but plugin.cfg says 1.1.0) Godot Engine v4.2.2 OS: Linux (NixOS)

Steps to reproduce:

  1. Downloaded from https://github.com/V-Sekai/godot-whisper/releases/tag/v1.0.5
  2. Copied addons folder to my project
  3. Added the plugin in the editor. I could download models through the interface, but SpeechToText is not working

I also tried to update the imports in godot_whisper.gdextension and include it in project.godot but that did not work either.

project.godot

config_version=5

[application]

config/features=PackedStringArray("4.2", "GL Compatibility")

[editor_plugins]

enabled=PackedStringArray("res://addons/godot_whisper/plugin.cfg")

[native_extensions]

paths=PackedStringArray("res://addons/godot_whisper/godot_whisper.gdextension")

[rendering]

renderer/rendering_method="gl_compatibility"
renderer/rendering_method.mobile="gl_compatibility"
textures/vram_compression/import_etc2_astc=true

...
  
addons/godot_whisper/godot_whisper.gdextension

[configuration]

entry_symbol = "godot_whisper_library_init"
compatibility_minimum = 4.1

[libraries]

macos.debug = "res://addons/godot_whisper/bin/libgodot_whisper.macos.template_release.universal.framework"
macos.release = "res://addons/godot_whisper/bin/libgodot_whisper.macos.template_release.universal.framework"
macos.debug.double = "res://addons/godot_whisper/bin/libgodot_whisper.macos.template_release.double.universal.framework"
macos.release.double = "res://addons/godot_whisper/bin/libgodot_whisper.macos.template_release.double.universal.framework"

ios.debug = "res://addons/godot_whisper/bin/libgodot_whisper.ios.template_release.arm64.dylib"
ios.release = "res://addons/godot_whisper/bin/libgodot_whisper.ios.template_release.arm64.dylib"
ios.debug.double = "res://addons/godot_whisper/bin/libgodot_whisper.ios.template_release.double.arm64.dylib"
ios.release.double = "res://addons/godot_whisper/bin/libgodot_whisper.ios.template_release.double.arm64.dylib"

windows.debug.x86_32 = "res://addons/godot_whisper/bin/libgodot_whisper.windows.template_release.x86_32.dll"
windows.release.x86_32 = "res://addons/godot_whisper/bin/libgodot_whisper.windows.template_release.x86_32.dll"
windows.debug.x86_64 = "res://addons/godot_whisper/bin/libgodot_whisper.windows.template_release.x86_64.dll"
windows.release.x86_64 = "res://addons/godot_whisper/bin/libgodot_whisper.windows.template_release.x86_64.dll"
windows.debug.double.x86_32 = "res://addons/godot_whisper/bin/libgodot_whisper.windows.template_release.double.x86_32.dll"
windows.release.double.x86_32 = "res://addons/godot_whisper/bin/libgodot_whisper.windows.template_release.double.x86_32.dll"
windows.debug.double.x86_64 = "res://addons/godot_whisper/bin/libgodot_whisper.windows.template_release.double.x86_64.dll"
windows.release.double.x86_64 = "res://addons/godot_whisper/bin/libgodot_whisper.windows.template_release.double.x86_64.dll"

linux.debug.x86_64 = "res://addons/godot_whisper/bin/libgodot_whisper.linux.template_release.x86_64.so"
linux.release.x86_64 = "res://addons/godot_whisper/bin/libgodot_whisper.linux.template_release.x86_64.so"
linux.debug.double.x86_64 = "res://addons/godot_whisper/bin/libgodot_whisper.linux.template_release.double.x86_64.so"
linux.release.double.x86_64 = "res://addons/godot_whisper/bin/libgodot_whisper.linux.template_release.double.x86_64.so"

android.debug.x86_64 = "res://addons/godot_whisper/bin/libgodot_whisper.android.template_release.x86_64.so"
android.release.x86_64 = "res://addons/godot_whisper/bin/libgodot_whisper.android.template_release.x86_64.so"
android.debug.x86_32 = "res://addons/godot_whisper/bin/libgodot_whisper.android.template_release.x86_32.so"
android.release.x86_32 = "res://addons/godot_whisper/bin/libgodot_whisper.android.template_release.x86_32.so"
android.debug.arm64 = "res://addons/godot_whisper/bin/libgodot_whisper.android.template_release.arm64.so"
android.release.arm64 = "res://addons/godot_whisper/bin/libgodot_whisper.android.template_release.arm64.so"
android.debug.arm32 = "res://addons/godot_whisper/bin/libgodot_whisper.android.template_release.arm32.so"
android.release.arm32 = "res://addons/godot_whisper/bin/libgodot_whisper.android.template_release.arm32.so"

android.debug.double.x86_64 = "res://addons/godot_whisper/bin/libgodot_whisper.android.template_release.double.x86_64.so"
android.release.double.x86_64 = "res://addons/godot_whisper/bin/libgodot_whisper.android.template_release.double.x86_64.so"
android.debug.double.x86_32 = "res://addons/godot_whisper/bin/libgodot_whisper.android.template_release.double.x86_32.so"
android.release.double.x86_32 = "res://addons/godot_whisper/bin/libgodot_whisper.android.template_release.double.x86_32.so"
android.debug.double.arm64 = "res://addons/godot_whisper/bin/libgodot_whisper.android.template_release.double.arm64.so"
android.release.double.arm64 = "res://addons/godot_whisper/bin/libgodot_whisper.android.template_release.double.arm64.so"
android.debug.double.arm32 = "res://addons/godot_whisper/bin/libgodot_whisper.android.template_release.double.arm32.so"
android.release.double.arm32 = "res://addons/godot_whisper/bin/libgodot_whisper.android.template_release.double.arm32.so"

web.debug.wasm32 = "res://addons/godot_whisper/bin/libgodot_whisper.web.template_release.wasm32.wasm"
web.release.wasm32 = "res://addons/godot_whisper/bin/libgodot_whisper.web.template_release.wasm32.wasm"
web.debug.double.wasm32 = "res://addons/godot_whisper/bin/libgodot_whisper.web.template_release.double.wasm32.wasm"
web.release.double.wasm32 = "res://addons/godot_whisper/bin/libgodot_whisper.web.template_release.double.wasm32.wasm"

  
Folder structure

addons/godot_whisper on  master [!?] via C v13.3.0-gcc 
❯ lt
 .
├──  audio_stream_to_text.gd
├──  bin
│   ├──  ggml-metal.metal
│   ├──  libgodot_whisper.android.template_release.arm32.so
│   ├──  libgodot_whisper.android.template_release.arm64.so
│   ├──  libgodot_whisper.android.template_release.double.arm32.so
│   ├──  libgodot_whisper.android.template_release.double.arm64.so
│   ├──  libgodot_whisper.android.template_release.double.x86_32.so
│   ├──  libgodot_whisper.android.template_release.double.x86_64.so
│   ├──  libgodot_whisper.android.template_release.x86_32.so
│   ├──  libgodot_whisper.android.template_release.x86_64.so
│   ├──  libgodot_whisper.ios.template_release.arm64.framework
│   │   ├──  Info.plist
│   │   └──  libgodot_whisper.ios.template_release.arm64
│   ├──  libgodot_whisper.ios.template_release.double.arm64.framework
│   │   ├──  Info.plist
│   │   └──  libgodot_whisper.ios.template_release.double.arm64
│   ├──  libgodot_whisper.linux.template_release.double.x86_64.so
│   ├──  libgodot_whisper.linux.template_release.x86_64.so
│   ├──  libgodot_whisper.macos.template_release.double.universal.framework
│   │   ├──  _CodeSignature
│   │   │   └──  CodeResources
│   │   ├──  libgodot_whisper.macos.template_release.double.universal
│   │   └──  Resources
│   │       └──  Info.plist
│   ├──  libgodot_whisper.macos.template_release.universal.framework
│   │   ├──  _CodeSignature
│   │   │   └──  CodeResources
│   │   ├──  libgodot_whisper.macos.template_release.universal
│   │   └──  Resources
│   │       └──  Info.plist
│   ├──  libgodot_whisper.web.template_release.double.wasm32.wasm
│   ├──  libgodot_whisper.web.template_release.wasm32.wasm
│   ├──  libgodot_whisper.windows.template_release.double.x86_32.dll
│   ├──  libgodot_whisper.windows.template_release.double.x86_64.dll
│   ├──  libgodot_whisper.windows.template_release.x86_32.dll
│   └──  libgodot_whisper.windows.template_release.x86_64.dll
├──  capture_stream_to_text.gd
├──  ggml-metal.metal
├──  godot_whisper.gdextension
├──  label_transcribe.gd
├──  model_downloader.gd
├──  models
│   └──  gglm-tiny.en.bin
├──  plugin.cfg
├──  speech_to_text.cpp
├──  speech_to_text.h
├──  whisper_dock.gd
└──  whisper_dock.tscn
  

Minimal reproducible repo: https://github.com/snorreks/godot-whisper-bug_SpeechToText_base_class

fire commented 2 weeks ago

Investigating. I will be slow.

fire commented 2 weeks ago

I suspect somehow the shared library isn't being loaded onto your specific linux. "nix" is not a specific Linux distro I am aware of.

snorreks commented 2 weeks ago

The distro is called NixOS, I have a windows laptop as well, i will clone and setup godot there and see if it works there

snorreks commented 2 weeks ago

It is the same problem on Windows 11 Pro

Edit: I got it working on windows now.

fire commented 2 weeks ago

Can you join our discord? I'm in a middle of a funk, so there are a few folks here who can help.

snorreks commented 2 weeks ago

After more debugging I found out that it is a Linux only issue related to #73