search

watson-developer-cloud / unity-sdk

:video_game: Unity SDK to use the IBM Watson services.
Apache License 2.0
569 stars 206 forks source link

Error: could not detect endianness - ExampleStreaming.cs on MacBook Pro #630

Closed pvanallen closed 4 years ago

pvanallen commented 4 years ago

I get the below error when running the ExampleStreaming.cs example program in Unity. This occurs only on my MacBook Pro (MacBook Pro (15-inch, 2017), MacOS 10.15.3 (19D76). It does NOT happen on my iMac running the exact same code.

I've tried several different microphones and check to make sure Unity has permission to use the mic.

====== Error: could not detect endianness after looking at the tail 0 non-zero byte string in a data stream of 22050 bytes. Is the bytestream really PCM data?

pvanallen commented 4 years ago

I also tried this example on VM Ware Fusion running emulating Windows 10 and current Windows Unity, and it worked fine, no errors. I'm checking with my students, so far the problem consistently shows on MacBook Pros in MacOS.

mamoonraja commented 4 years ago

Hi @pvanallen , I was able to run the example in Mac OS 10.14.5. Can you provide more info around the error by sharing a screenshot of the Unity window?

mamoonraja commented 4 years ago

We can try two solutions here:

Since, I am not seeing the issue I can not verify this solution. If it works we will update the example to use wav.

pvanallen commented 4 years ago

HI @mamoonraja - Thanks for the update. Just a reminder that for testing, the original error happens only on a MacBook Pro, not desktops.

I made the suggested change, but now see the following error in both desktop and laptop.

[03/31/2020 17:23:13][SpeechToText.OnListenMessage()][ERROR] Error: unable to transcode data stream audio/wav -> audio/x-float-array UnityEngine.Debug:LogError(Object) IBM.Cloud.SDK.Debug.DebugReactor:ProcessLog(LogRecord) (at Assets/Scripts/unity-sdk-core-1.2.0/Debug/DebugReactor.cs:60) IBM.Cloud.SDK.LogSystem:ProcessLog(LogRecord) (at Assets/Scripts/unity-sdk-core-1.2.0/Logging/Logger.cs:207) IBM.Cloud.SDK.Log:Error(String, String, Object[]) (at Assets/Scripts/unity-sdk-core-1.2.0/Logging/Logger.cs:280) IBM.Watson.SpeechToText.V1.SpeechToTextService:OnListenMessage(Message) (at Assets/Scripts/unity-sdk-4.5.0/Scripts/Services/SpeechToText/V1/SpeechToTextServiceExtension.cs:651) IBM.Cloud.SDK.Connection.d__49:MoveNext() (at Assets/Scripts/unity-sdk-core-1.2.0/Connection/WSConnector.cs:430) IBM.Cloud.SDK.Utilities.Routine:MoveNext() (at Assets/Scripts/unity-sdk-core-1.2.0/Utilities/Runnable.cs:131) UnityEngine.SetupCoroutine:InvokeMoveNext(IEnumerator, IntPtr) (at /Users/builduser/buildslave/unity/build/Runtime/Export/Scripting/Coroutines.cs:17)

mamoonraja commented 4 years ago

Thanks for testing, we need changes on the core side too to make that change. What's your MacOs version in iMac?

pvanallen commented 4 years ago

iMac MacOS: 10.15.3

mamoonraja commented 4 years ago

I think it is an underlying hardware issue with l16 format, I am not able to re-create in MacBook pro, but I am on a different OS version. One thing you can do is change sampling size in https://github.com/watson-developer-cloud/unity-sdk/blob/master/Examples/ExampleStreaming.cs#L58 to 48000, not sure if it will work. If this doesn't work I will look into adding support for wav file in example, but not sure when we will be able to get to that.

pvanallen commented 4 years ago

Thanks for taking a look... I tried 48000, but same error, with no change to l16.

[03/31/2020 17:57:37][SpeechToText.OnListenMessage()][ERROR] Error: could not detect endianness after looking at the tail 0 non-zero byte string in a data stream of 48000 bytes. Is the bytestream really PCM data? UnityEngine.Debug:LogError(Object) IBM.Cloud.SDK.Debug.DebugReactor:ProcessLog(LogRecord) (at Assets/Scripts/unity-sdk-core-1.2.0/Debug/DebugReactor.cs:60) IBM.Cloud.SDK.LogSystem:ProcessLog(LogRecord) (at Assets/Scripts/unity-sdk-core-1.2.0/Logging/Logger.cs:207) IBM.Cloud.SDK.Log:Error(String, String, Object[]) (at Assets/Scripts/unity-sdk-core-1.2.0/Logging/Logger.cs:280) IBM.Watson.SpeechToText.V1.SpeechToTextService:OnListenMessage(Message) (at Assets/Scripts/unity-sdk-4.5.0/Scripts/Services/SpeechToText/V1/SpeechToTextServiceExtension.cs:651) IBM.Cloud.SDK.Connection.d__49:MoveNext() (at Assets/Scripts/unity-sdk-core-1.2.0/Connection/WSConnector.cs:430) IBM.Cloud.SDK.Utilities.Routine:MoveNext() (at Assets/Scripts/unity-sdk-core-1.2.0/Utilities/Runnable.cs:131) UnityEngine.SetupCoroutine:InvokeMoveNext(IEnumerator, IntPtr) (at /Users/builduser/buildslave/unity/build/Runtime/Export/Scripting/Coroutines.cs:17)

mediumTaj commented 4 years ago

I've only seen this error when the mic wasn't getting audio - can you ensure your mic is working and pulling audio into Unity?

pvanallen commented 4 years ago

@mediumTaj Thanks for the note - do you have a test in Unity you suggest I run? The mic works fine in other applications.

mediumTaj commented 4 years ago

Can you join the Watson Developer Community slack workspace? We can talk on the #unity-sdk channel.

http://wdc-slack-inviter.mybluemix.net/

mediumTaj commented 4 years ago

Looking into this - it looks like there is an issue with OSX Catalina that may be disallowing recording. Will look into this.

mediumTaj commented 4 years ago

found a thread describing unity not asking for permissions for mic and camera in catalina. might be worth trying the workaround

https://issuetracker.unity3d.com/issues/osx-unity-fails-to-ask-camera-permission-and-crashes-on-privacy-violation?page=1#comments

mediumTaj commented 4 years ago

From a direct message:

This method from above worked: 1) Disable SIP: Go into recovery mode (hold CMD+R when you restart your Mac) Don’t be afraid, we are not doing anything crazy. 2) After that open Terminal (it should be accessible from one of the Menus at the Top) Type in “csrutil disable” and hit enter. Then reboot your Mac normally. 3) Open Terminal and type in: “sqlite3 ~/Library/Application\ Support/com.apple.TCC/TCC.db” and hit enter 4) For microphone access, type in: “INSERT INTO access VALUES(‘kTCCServiceMicrophone’,‘com.unity3d.unityhub’,0,1,1,NULL,NULL,NULL,‘UNUSED’,NULL,0,1541440109);” and hit enter For camera, type in: “INSERT INTO access VALUES(‘kTCCServiceCamera’,‘com.unity3d.unityhub’,0,1,1,NULL,NULL,NULL,‘UNUSED’,NULL,0,1541440109);” and hit enter 6) check in your SystemPreferences/Security&Privacy > unity hub should now show up in both mic and cam 7) reboot in recovery mode again (CMD+R), open terminal again, type in: “csrutil enable” and hit enter, to enable SIP again 8) reboot normally and enjoy!

From this issue