alphacep / vosk-unity-asr

Automatic Speech Recognition in Unity using Vosk library
65 stars 16 forks source link

Plugin crashes on first run #1

Closed thnewlands closed 2 years ago

thnewlands commented 2 years ago

When opening Vosk Unity ASR for the first time the application crashes. I've found this in the Windows editor and the Android runtime on Oculus Quest.

Crashlogs for editor on Windows report "ERROR: SymGetSymFromAddr64, GetLastError: 'Attempt to access invalid address.' (Address: 00007FFF908ACEF3)"

nshmyrev commented 2 years ago

Hi. Sounds strange. What are the messages in the log please?

nshmyrev commented 2 years ago

It should say something about model loading.

thnewlands commented 2 years ago

Here's the end of one of my Unity logs. I'm also running FMOD which could be interfering.

Start importing Assets/Plugins/FMOD/Cache/Editor/FMODStudioCache.asset using Guid(60098df5ab68449f3b224c564c94ee60) Importer(-1,00000000000000000000000000000000)  -> (artifact id: '14cca54af5aedd80f7d4a43c1bc85f82') in 0.254310 seconds 
Refreshing native plugins compatible for Editor in 2.13 ms, found 40 plugins.
Preloading 1 native plugins for Editor in 0.08 ms.
RefreshInfo: StopAssetImportingV2(NoUpdateAssetOptions) scanfilter: 
RefreshProfiler: Total: 488.602ms
    InvokeBeforeRefreshCallbacks: 0.001ms
    ApplyChangesToAssetFolders: 0.077ms
    Scan: 2.622ms
    OnSourceAssetsModified: 0.000ms
    InitializeImportedAssetsSnapshot: 10.559ms
    GetAllGuidsForCategorization: 0.635ms
    CategorizeAssets: 3.547ms
    ImportAndPostprocessOutOfDateAssets: 452.657ms (5.120ms without children)
        ImportManagerImport: 260.989ms (4.636ms without children)
            ImportInProcess: 256.300ms
            UpdateCategorizedAssets: 0.053ms
        PostProcessAllAssets: 169.490ms
        ReloadImportedAssets: 0.001ms
        EnsureUptoDateAssetsAreRegisteredWithGuidPM: 0.894ms
        InitializingProgressBar: 0.004ms
        PostProcessAllAssetNotificationsAddChangedAssets: 0.700ms
        OnDemandSchedulerStart: 0.664ms
        RestoreLoadedAssetsState: 8.125ms
    UpdateImportedAssetsSnapshot: 6.668ms
    ReloadSourceAssets: 1.639ms
    UnloadImportedAssets: 0.277ms
    Hotreload: 10.291ms
    FixTempGuids: 0.008ms
    GatherAllCurrentPrimaryArtifactRevisions: 0.221ms
    UnloadStreamsBegin: 0.080ms
    LoadedImportedAssetsSnapshotReleaseGCHandles: 2.161ms
    GetLoadedSourceAssetsSnapshot: 14.146ms
    PersistCurrentRevisions: 0.376ms
    UnloadStreamsEnd: 0.080ms
    Untracked: -10.773ms
FMOD: Cache updated.
UnityEngine.StackTraceUtility:ExtractStackTrace ()
UnityEngine.DebugLogHandler:LogFormat (UnityEngine.LogType,UnityEngine.Object,string,object[])
UnityEngine.Logger:Log (UnityEngine.LogType,object)
UnityEngine.Debug:Log (object)
FMODUnity.EventManager:UpdateCache () (at Assets/Plugins/FMOD/src/Editor/EventManager.cs:318)
FMODUnity.EventManager:RefreshBanks () (at Assets/Plugins/FMOD/src/Editor/EventManager.cs:30)
FMODUnity.EventManager:Startup () (at Assets/Plugins/FMOD/src/Editor/EventManager.cs:486)
UnityEditor.EditorApplication:Internal_CallDelayFunctions ()

(Filename: Assets/Plugins/FMOD/src/Editor/EventManager.cs Line: 318)

[XR] Creating Textures
XRTextureManager::RequestDestroyTexture
XRTextureManager::RequestDestroyTexture
XRTextureManager::RequestCreateTexture
XRTextureManager::RequestCreateTexture
XRTextureManager::SetupRenderTextureFromXRRequest (id: 15 col: 0000000000000000 d: 0000000000000000)
RenderTexture::Create (id: 15)
XRTextureManager::SetupRenderTextureFromXRRequest (id: 16 col: 0000000000000000 d: 0000000000000000)
RenderTexture::Create (id: 16)
INFO: cmn_live.c(120): Update from < 40.00 10.00 10.00  0.00  0.00  0.00  0.00  0.00  0.00  0.00  0.00  0.00  0.00 >
INFO: cmn_live.c(138): Update to   < 45.90  4.29 -7.84  7.36 -20.24  2.69 -7.94 -6.41 -4.15 -5.78  4.90 -1.78 -10.59 >
INFO: fsg_search.c(867): 113 frames, 1559 HMMs (13/fr), 4087 senones (36/fr), 421 history entries (3/fr)

INFO: fsg_search.c(877): fsg 5.23 CPU 4.592 xRT
INFO: fsg_search.c(879): fsg 2.60 wall 2.284 xRT
Stacktrace:

  at <unknown> <0xffffffff>
  at (wrapper managed-to-native) DesktopSpeechRecognizer.readMicBuffer (intptr) [0x00009] in <01a1838b1b134325b4d3a89d6db37dba>:0
  at DesktopSpeechRecognizer/<coUpdateWithDelay>d__28.MoveNext () [0x00038] in <01a1838b1b134325b4d3a89d6db37dba>:0
  at UnityEngine.SetupCoroutine.InvokeMoveNext (System.Collections.IEnumerator,intptr) [0x00027] in <41d34619c06946aebeba0ca1ee6d765b>:0
  at (wrapper runtime-invoke) <Module>.runtime_invoke_void_object_intptr (object,intptr,intptr,intptr) [0x00027] in <41d34619c06946aebeba0ca1ee6d765b>:0

=================================================================
Got a SIGSEGV while executing native code. This usually indicates
a fatal error in the mono runtime or one of the native libraries 
used by your application.
=================================================================

Received signal SIGSEGV
Stack trace:
RtlLookupFunctionEntry returned NULL function. Aborting stack walk.
<Missing stacktrace information>
nshmyrev commented 2 years ago

Hi. From the log it seems you are trying to run pockesphinx, not Unity Vosk.

thnewlands commented 2 years ago

Oh oops! Thank you for looking at this my bad! Must have left a conflicting voice recognition package in the mix.

thnewlands commented 2 years ago

Sorry looks like I just sent you the wrong bug. I'm still getting this issue consistently on the Quest. The error reads:

2022/06/03 13:33:11.754 14307 14328 Error VoskAPI Model():model.cc:122) Folder '/storage/emulated/0/Android/data/com.vendor.app/files' does not contain model files. Make sure you specified the model path properly in Model constructor. If you are not sure about relative path, use absolute path specification.

Seems like an issue with streaming assets?

Again this is only an issue the first time the zip is loaded into the app -- subsequent runs don't have this issue.

nshmyrev commented 2 years ago

Seems like an issue with streaming assets?

Yes, we need to wait for unpack process to finish I suppose

thnewlands commented 2 years ago

More context:


2022/06/03 13:49:21.301 15542 15801 Debug AAudio AAudioStream_requestStart(s#7) returned 0 ---------
2022/06/03 13:49:21.309 15542 15542 Verbose MediaRouter Selecting route: RouteInfo{ name=Phone, description=null, status=null, category=RouteCategory{ name=System types=ROUTE_TYPE_LIVE_AUDIO ROUTE_TYPE_LIVE_VIDEO  groupable=false }, supportedTypes=ROUTE_TYPE_LIVE_AUDIO ROUTE_TYPE_LIVE_VIDEO , presentationDisplay=null }
2022/06/03 13:49:21.311 15542 15807 Debug AudioStreamLegacy onAudioDeviceUpdate() devId 2 => 2
2022/06/03 13:49:22.068 15542 15562 Info Unity Reading Zip file
2022/06/03 13:49:22.068 15542 15562 Info Unity System.Action`1:Invoke(T)
2022/06/03 13:49:22.068 15542 15562 Info Unity <Decompress>d__26:MoveNext()
2022/06/03 13:49:22.068 15542 15562 Info Unity UnityEngine.SetupCoroutine:InvokeMoveNext(IEnumerator, IntPtr)
2022/06/03 13:49:22.068 15542 15562 Info Unity 
2022/06/03 13:49:22.182 15542 15847 Info Telemetry App memory usage: PSS=373MB DalvikPSS=0 MB PrivateDirty=346MB PrivateClean=21MB
2022/06/03 13:49:22.187 15542 15847 Info VrApi FPS=72/72,Prd=45ms,Tear=0,Early=72,Stale=0,VSnc=1,Lat=1,Fov=0,CPU4/GPU=2/2,2304/414MHz,OC=FF,TA=0/0/0,SP=N/N/N,Mem=1295MHz,Free=1445MB,PLS=0,Temp=30.0C/0.0C,TW=1.63ms,App=1.70ms,GD=0.00ms,CPU&GPU=3.48ms,LCnt=1,GPU%=0.27,CPU%=0.07(W0.09),DSF=1.00
2022/06/03 13:49:23.182 15542 15847 Info VrApi FPS=72/72,Prd=45ms,Tear=0,Early=72,Stale=0,VSnc=1,Lat=1,Fov=0,CPU4/GPU=2/2,2304/414MHz,OC=FF,TA=0/0/0,SP=N/N/N,Mem=1554MHz,Free=1445MB,PLS=0,Temp=30.0C/0.0C,TW=1.63ms,App=1.70ms,GD=0.00ms,CPU&GPU=3.48ms,LCnt=1,GPU%=0.27,CPU%=0.07(W0.09),DSF=1.00
2022/06/03 13:49:23.885 15542 15562 Info Unity Decompressing complete!
2022/06/03 13:49:23.885 15542 15562 Info Unity System.Action`1:Invoke(T)
2022/06/03 13:49:23.885 15542 15562 Info Unity <Decompress>d__26:MoveNext()
2022/06/03 13:49:23.885 15542 15562 Info Unity UnityEngine.SetupCoroutine:InvokeMoveNext(IEnumerator, IntPtr)
2022/06/03 13:49:23.885 15542 15562 Info Unity 
2022/06/03 13:49:24.182 15542 15847 Info VrApi FPS=1/72,Prd=14ms,Tear=0,Early=5,Stale=67,VSnc=1,Lat=1,Fov=0,CPU4/GPU=4/2,2304/414MHz,OC=FF,TA=0/0/0,SP=N/N/N,Mem=1554MHz,Free=1445MB,PLS=0,Temp=30.0C/0.0C,TW=2.38ms,App=10.23ms,GD=0.00ms,CPU&GPU=7.91ms,LCnt=1,GPU%=0.43,CPU%=0.32(W0.52),DSF=1.00
2022/06/03 13:49:24.637 15542 15562 Info Unity Loading Model from: /storage/emulated/0/Android/data/com.vendor.app/files
2022/06/03 13:49:24.637 15542 15562 Info Unity System.Action`1:Invoke(T)
2022/06/03 13:49:24.637 15542 15562 Info Unity <DoStartVoskStt>d__24:MoveNext()
2022/06/03 13:49:24.637 15542 15562 Info Unity UnityEngine.SetupCoroutine:InvokeMoveNext(IEnumerator, IntPtr)
2022/06/03 13:49:24.637 15542 15562 Info Unity 
2022/06/03 13:49:24.686 15542 15562 Error VoskAPI Model():model.cc:122) Folder '/storage/emulated/0/Android/data/com.vendor.app/files' does not contain model files. Make sure you specified the model path properly in Model constructor. If you are not sure about relative path, use absolute path specification.
thnewlands commented 2 years ago

I think that's the issue too. Although Decompress is already awaited in the coroutine for Vosk's startup.

thnewlands commented 2 years ago

It might make sense to leave the option for users to provide the models in a decompressed state so that they don't have to unpack on first run.

thnewlands commented 2 years ago

I suppose that's as easy as just unpacking it in editor inside streaming assets though right?

thnewlands commented 2 years ago

Update: Unpacking into streaming assets didn't solve my problem since it's expected in the persistentdatapath.

thnewlands commented 2 years ago

Update2: Moving to streaming data doesn't work on android bc it requires a web request to access it. I think the best solution is probably still unpacking at runtime.

thnewlands commented 2 years ago

Okay I think I've got it figured out -- the issue is that ionic's zip utility's ExtractProgressEventArgs.ExtractLocation outputs a blank string. The solution that works for me is to just make the path again with:

_decompressedModelPath = Path.Combine(Application.persistentDataPath, Path.GetFileNameWithoutExtension(ModelPath));

nshmyrev commented 2 years ago

Should be fixed now!