search

wit-ai / wit-unreal

The Wit.ai SDK for Unreal.
MIT License
2 stars 2 forks source link

UE5.0.3 TTS not working & no event callbacks #57

Closed GameBayoumy closed 1 year ago

GameBayoumy commented 1 year ago

Trying to call Speak function, but failed to make connection with the API it seems. When converting TTS, no event callbacks are being called (OnSynthesize)

I've created a Data Asset with Client token. Used a preset from /VoiceSDK/Presets Calling Speak a second time, the console will log "Warning: ConvertTextToSpeechWithSettings: cannot convert text because a request is already in progress"

Currently using: voicesdk-all-in-one-marketplace-ue5_0-47_0_2 UE5.0.3 (Blank template)

ssPFBcX

wangpingsx commented 1 year ago

Hi @GameBayoumy, How are you? Hope you are having a good week. I am so happy to know that you are using our plugin. I will help you with this issue.

  1. It will be great if you can share more logs.
  2. Could you please try the demo map: /voicesdk-unreal/Content/Demo/Levels/VoiceSDKShapesDemo.umap if you setup a config with the right client token, you should be able to hear the TTS response("hello TTS") after game started. I just tested; it works as expected.
  3. Regarding the second time call. You will need to add enough delay before the 2nd call. In later version we will return the audio's duration, so you can make the 2nd call after the 1st audio fully played
  4. the callback -- "OnSynthesizeResponse" works as expected on my side, I cannot see it on your blueprint.

Summary: Please let's step back a little bit and just make 1 TTS call only. Let's get it to work first, please try the demo map -- VoiceSDKShapesDemo.umap, if it doesn't work for you, please share me more logs.

GameBayoumy commented 1 year ago

Hi @wangpingsx, I've been doing great thank you!

For now I try to get started with wit inside UE but still no success. My steps are as follows:

  1. Create UE blank template
  2. Drag voicesdk-unreal inside Plugins folder of project
  3. Generate & compiled solution (Development Editor config)
  4. Loading VoiceSDKShapesDemo.umap
  5. Creating a WitAppConfigurationAsset and setting the Client Access Token from Wit App
  6. Changing the BP_TtsSpeaker -> TTS/Configuration to DataAsset with Client Access Token
  7. Even tried to Fetch Voices (just in case)
  8. Play the map in level editor viewport

No audio, it does log ConvertTextToSpeechWithSettings: converting text (Hello TTS Demo) with voice (Charlie)

Rest of the logs including LogWit Verbose: LogDebuggerCommands: Repeating last play command: Selected Viewport LogPlayLevel: PlayLevel: No blueprints needed recompiling PIE: New page: PIE session: VoiceSDKShapesDemo (8 Dec 2022, 21:32:45) LogPlayLevel: Creating play world package: /VoiceSDK/Demo/Levels/UEDPIE_0_VoiceSDKShapesDemo LogPlayLevel: PIE: StaticDuplicateObject took: (0.003942s) LogPlayLevel: PIE: Created PIE world by copying editor world from /VoiceSDK/Demo/Levels/VoiceSDKShapesDemo.VoiceSDKShapesDemo to /VoiceSDK/Demo/Levels/UEDPIE_0_VoiceSDKShapesDemo.VoiceSDKShapesDemo (0.004166s) LogSubsystemCollection: Failed to initialize subsystem dependency (WorldPartitionSubsystem) LogChaos: FPhysicsSolverBase::AsyncDt:-1.000000 LogAIModule: Creating AISystem for world VoiceSDKShapesDemo LogPlayLevel: PIE: World Init took: (0.000848s) LogUObjectHash: Compacting FUObjectHashTables data took 0.33ms LogAudio: Display: Creating Audio Device: Id: 3, Scope: Unique, Realtime: True LogAudioMixer: Display: Audio Mixer Platform Settings: LogAudioMixer: Display: Sample Rate: 48000 LogAudioMixer: Display: Callback Buffer Frame Size Requested: 1024 LogAudioMixer: Display: Callback Buffer Frame Size To Use: 1024 LogAudioMixer: Display: Number of buffers to queue: 2 LogAudioMixer: Display: Max Channels (voices): 0 LogAudioMixer: Display: Number of Async Source Workers: 0 LogAudio: Display: AudioDevice MaxSources: 32 LogAudio: Display: Audio Spatialization Plugin: None (built-in). LogAudio: Display: Audio Reverb Plugin: None (built-in). LogAudio: Display: Audio Occlusion Plugin: None (built-in). LogAudioMixer: Display: Initializing audio mixer using platform API: 'XAudio2' LogAudioMixer: Display: Using Audio Hardware Device U34G2G4R3 (NVIDIA High Definition Audio) LogAudioMixer: Display: Initializing Sound Submixes... LogAudioMixer: Display: Creating Master Submix 'MasterSubmixDefault' LogAudioMixer: Display: Creating Master Submix 'MasterReverbSubmixDefault' LogAudioMixer: FMixerPlatformXAudio2::StartAudioStream() called. InstanceID=3 LogAudioMixer: Display: Output buffers initialized: Frames=1024, Channels=2, Samples=2048, InstanceID=3 LogAudioMixer: Display: Starting AudioMixerPlatformInterface::RunInternal(), InstanceID=3 LogAudioMixer: Display: FMixerPlatformXAudio2::SubmitBuffer() called for the first time. InstanceID=3 LogInit: FAudioDevice initialized. LogAudio: Display: Audio Device (ID: 3) registered with world 'VoiceSDKShapesDemo'. LogLoad: Game class is 'GameModeBase' LogWorld: Bringing World /VoiceSDK/Demo/Levels/UEDPIE_0_VoiceSDKShapesDemo.VoiceSDKShapesDemo up for play (max tick rate 60) at 2022.12.08-20.32.45 LogWorld: Bringing up level for play took: 0.000706 LogOnline: OSS: Created online subsystem instance for: :Context_5 LogVoiceEncode: Display: EncoderVersion: libopus unknown LogOnlineVoice: OSS: StopLocalVoiceProcessing(0) returned 0xFFFFFFFF LogOnlineVoice: OSS: Stopping networked voice for user: 0 LogVoiceSDK: Display: Init VoiceService: bIsPlatformIntegrationEnabled=false LogWit: Verbose: UVoiceResponseMatcher: Registering response callback LogWit: Verbose: UVoiceResponseMatcher: Registering response callback LogWit: Verbose: BeginPlay: adding synthesize response callback LogWit: Verbose: UTTSMemoryCache::GetClip: requesting clip with id (D493BA55894D558B4369EE10DB66AFB0F388702D) LogWit: Verbose: UTTSMemoryCache::GetClip: clip does not exist in cache LogWit: Display: ConvertTextToSpeechWithSettings: converting text (Hello TTS Demo) with voice (Charlie) LogWit: Verbose: WriteJsonData: Old reader size is (0) LogWit: Verbose: WriteJsonData: Wrote (96) bytes. New array size is (96) New reader size is (96) LogTemp: Display: https://api.wit.ai/synthesize LogWit: Verbose: SendRequest: Url is (https://api.wit.ai/synthesize?v=20221205), Content type is (application/json) and Content length is (96) LogViewport: Display: Viewport MouseLockMode Changed, LockOnCapture -> DoNotLock LogViewport: Display: Viewport MouseCaptureMode Changed, CapturePermanently_IncludingInitialMouseDown -> NoCapture PIE: Server logged in PIE: Play in editor total start time 0,063 seconds.

I tried calling API to see connectivity issues perhaps with my network, but executing a curl command with the API seems to work just fine at my end. Also, I got it working inside Unity to test it out. My firewall does not block Unreal Engine neither. And debugging UE5 shows that API calls are being transmitted as well.

OnSynthesizeResponse, which is responsible for the AudioComponent to set and play the sound wave is never called. And OnSynthesizeResponse.Broadcast is never being triggered by the EventHandler. SoundWave is never cached inside MemoryCache. The OnRequestProgress and OnRequestComplete delegate was also never triggered.

wangpingsx commented 1 year ago

Hi @GameBayoumy Thanks for helping me debugging this issue.

  1. I saw it is using "NVIDIA High Definition Audio" as speaker. Is this a real speaker?
  2. May I ask you to login into your wit.ai app and check the language you are using? (under: Management->Settings) TTS only works for English for now. Please try to create a new app with English language and try again.
  3. If this doesn't work, let's zip the project and let me try. Hope this helps.

Thanks

GameBayoumy commented 1 year ago

Sure! I try to find out as well where it could go wrong, my guess for now is the response of the SendRequest.

  1. Yes my real speaker is a monitor connected to a headset, it plays audio from UE. Playing a SoundWave inside editor works
  2. Language is EN and I tried creating a new app just now, without any success
  3. https://we.tl/t-P0ABILVv7F

If it works at your end, probably somewhere in my UE configuration/setup it wont allow connectivity of sending requests to the Wit API.

I also tried to run a simple HttpRequest myself. It fails to trigger OnProcessRequestComplete() even with this simple API request. So it's my UE built or config. I think you can close the issue, since its not related wit-unreal. No idea what this could be related to. It would be helpful if you recognize the issue and possibly share a solution, otherwise I'll keep on searching. vDf6ojE

GameBayoumy commented 1 year ago

Okay. So I found this interesting.

I managed to send a simple http request using the same code. But here is what I did to replicate the issue to break it.

  1. Remove anything related to Wit inside the map
  2. Add my own simple httpRequest actor
  3. Closing the editor
  4. Load the project again
  5. Run the editor with the simple httpRequestActor
  6. Response works with json content inside as expected
  7. Stopping the editor
  8. Adding BP_TTsSpeaker with a TTS Speak function request
  9. Run the editor
  10. Things start to break, even the simple httpRequestActor gets no response at all

Somehow the http request of Wit is related to causing the issue at my end. Like something get's stuck inside the UE editor http request whenever I add something related to Wit. No http request will work at all unless I restart my editor again and my simple httpRequest actor works again, if I were to remove Wit again.

Here are logs of BP_TTsSpeaker running for the first time with editor just opened:

LogDebuggerCommands: Repeating last play command: Selected Viewport LogPlayLevel: PlayLevel: No blueprints needed recompiling PIE: New page: PIE session: MainLevel (9 Dec 2022, 17:00:53) LogOnline: OSS: Created online subsystem instance for: NULL LogOnline: OSS: TryLoadSubsystemAndSetDefault: Loaded subsystem for module [NULL] LogPlayLevel: Creating play world package: /Game/UEDPIE_0_MainLevel LogPlayLevel: PIE: StaticDuplicateObject took: (0.001578s) LogPlayLevel: PIE: Created PIE world by copying editor world from /Game/MainLevel.MainLevel to /Game/UEDPIE_0_MainLevel.MainLevel (0.001645s) LogSubsystemCollection: Failed to initialize subsystem dependency (WorldPartitionSubsystem) LogChaos: FPhysicsSolverBase::AsyncDt:-1.000000 LogAIModule: Creating AISystem for world MainLevel LogPlayLevel: PIE: World Init took: (0.000861s) LogUObjectHash: Compacting FUObjectHashTables data took 0.86ms LogAudio: Display: Creating Audio Device: Id: 2, Scope: Unique, Realtime: True LogAudioMixer: Display: Audio Mixer Platform Settings: LogAudioMixer: Display: Sample Rate: 48000 LogAudioMixer: Display: Callback Buffer Frame Size Requested: 1024 LogAudioMixer: Display: Callback Buffer Frame Size To Use: 1024 LogAudioMixer: Display: Number of buffers to queue: 2 LogAudioMixer: Display: Max Channels (voices): 0 LogAudioMixer: Display: Number of Async Source Workers: 0 LogAudio: Display: AudioDevice MaxSources: 32 LogAudio: Display: Audio Spatialization Plugin: None (built-in). LogAudio: Display: Audio Reverb Plugin: None (built-in). LogAudio: Display: Audio Occlusion Plugin: None (built-in). LogAudioMixer: Display: Initializing audio mixer using platform API: 'XAudio2' LogAudioMixer: Display: Using Audio Hardware Device U34G2G4R3 (NVIDIA High Definition Audio) LogAudioMixer: Display: Initializing Sound Submixes... LogAudioMixer: Display: Creating Master Submix 'MasterSubmixDefault' LogAudioMixer: Display: Creating Master Submix 'MasterReverbSubmixDefault' LogAudioMixer: FMixerPlatformXAudio2::StartAudioStream() called. InstanceID=2 LogAudioMixer: Display: Output buffers initialized: Frames=1024, Channels=2, Samples=2048, InstanceID=2 LogAudioMixer: Display: Starting AudioMixerPlatformInterface::RunInternal(), InstanceID=2 LogAudioMixer: Display: FMixerPlatformXAudio2::SubmitBuffer() called for the first time. InstanceID=2 LogInit: FAudioDevice initialized. LogAudio: Display: Audio Device (ID: 2) registered with world 'MainLevel'. LogLoad: Game class is 'GameModeBase' LogWorld: Bringing World /Game/UEDPIE_0_MainLevel.MainLevel up for play (max tick rate 60) at 2022.12.09-16.00.53 LogWorld: Bringing up level for play took: 0.000567 LogOnline: OSS: Created online subsystem instance for: :Context_2 LogWit: Verbose: VoiceCapture - OnModulesChanged: module has been loaded LogVoiceEncode: Display: EncoderVersion: libopus unknown LogOnlineVoice: OSS: StopLocalVoiceProcessing(0) returned 0xFFFFFFFF LogOnlineVoice: OSS: Stopping networked voice for user: 0 LogWit: Verbose: BeginPlay: adding synthesize response callback LogWit: Verbose: UTTSMemoryCache::GetClip: requesting clip with id (5491D623C9B07072B1E29D11EF2333E6FAC6B06F) LogWit: Verbose: UTTSMemoryCache::GetClip: clip does not exist in cache LogWit: Verbose: UTTSStorageCache::RequestClip: requesting clip (5491D623C9B07072B1E29D11EF2333E6FAC6B06F) with path (C:/Users/sbayo/AppData/Local/Wit/Cache/) LogWit: Verbose: UTTSStorageCache::LoadClipFromBinaryFile: clip does exist in cache LogWit: Display: ConvertTextToSpeechWithSettings: converting text (Hello World) with voice (Charlie) LogWit: Verbose: WriteJsonData: Old reader size is (0) LogWit: Verbose: WriteJsonData: Wrote (93) bytes. New array size is (93) New reader size is (93) LogTemp: Display: https://api.wit.ai/synthesize LogHttp: Verbose: FCurlHttpRequest::SetContentFromStream() - FMemoryReader LogHttp: VeryVerbose: FHttpRequestImpl::OnRequestProgress() LogHttp: VeryVerbose: FHttpRequestImpl::OnProcessRequestComplete() LogHttp: Verbose: 00000554D5953C00: URL='https://api.wit.ai/synthesize?v=20221205' LogHttp: Verbose: 00000554D5953C00: Verb='POST' LogHttp: Verbose: 00000554D5953C00: Custom headers are present LogHttp: Verbose: 00000554D5953C00: Payload size=93 LogHttp: Verbose: 00000554D5953C00: request (easy handle:00000554D3879000) has been added to threaded queue for processing LogWit: Verbose: SendRequest: Url is (https://api.wit.ai/synthesize?v=20221205), Content type is (application/json) and Content length is (93) PIE: Server logged in PIE: Play in editor total start time 0,112 seconds. LogHttp: Verbose: 00000554D5953C00: Adding header 'User-Agent: voice-sdk-47.0.2,voicesdk-unreal,Windows-10.0.22621.1.768.64bit,Unknown,7420282D4C70A64AE9CB3F8197991890,Unknown,Editor' LogHttp: Verbose: 00000554D5953C00: Adding header 'Accept: audio/wav' LogHttp: Verbose: 00000554D5953C00: Adding header 'Content-Type: application/json' LogHttp: Verbose: 00000554D5953C00: Adding header 'Content-Length: 93' LogHttp: Verbose: 00000554D5953C00: Adding header 'Expect: ' LogHttp: 00000554D5953C00: Starting POST request to URL='https://api.wit.ai/synthesize?v=20221205' LogWit: Verbose: SetupRequestOverrides: trying to set overrides LogWit: Verbose: SetupRequestOverrides: overriding upload callback LogHttp: Verbose: 00000554D5953C00: request (easy handle:00000554D3879000) has started threaded processing LogHttp: Verbose: Started running threaded request (00000554D5953C00). Running threaded requests (1) Rate limited threaded requests (0) LogHttp: VeryVerbose: 00000554D5953C00: ' Trying 2a03:2880:f245:c9:face:b00c:0:6fe...' LogHttp: VeryVerbose: 00000554D5953C00: 'TCP_NODELAY set' LogHttp: VeryVerbose: 00000554D5953C00: 'Connected to api.wit.ai (2a03:2880:f245:c9:face:b00c:0:6fe) port 443 (#0)' LogHttp: VeryVerbose: 00000554D5953C00: 'ALPN, offering http/1.1' LogHttp: VeryVerbose: 00000554D5953C00: 'Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH' LogHttp: VeryVerbose: 00000554D5953C00: Sent SSL data (5 bytes) LogHttp: VeryVerbose: 00000554D5953C00: 'TLSv1.3 (OUT), TLS handshake, Client hello (1):' LogHttp: VeryVerbose: 00000554D5953C00: Sent SSL data (512 bytes) LogHttp: VeryVerbose: 00000554D5953C00: Received SSL data (5 bytes) LogHttp: VeryVerbose: 00000554D5953C00: 'TLSv1.3 (IN), TLS handshake, Server hello (2):' LogHttp: VeryVerbose: 00000554D5953C00: Received SSL data (122 bytes) LogHttp: VeryVerbose: 00000554D5953C00: Received SSL data (5 bytes) LogHttp: VeryVerbose: 00000554D5953C00: Received SSL data (5 bytes) LogHttp: VeryVerbose: 00000554D5953C00: 'TLSv1.3 (IN), TLS Unknown, Certificate Status (22):' LogHttp: VeryVerbose: 00000554D5953C00: Received SSL data (1 bytes) LogHttp: VeryVerbose: 00000554D5953C00: 'TLSv1.3 (IN), TLS handshake, Unknown (8):' LogHttp: VeryVerbose: 00000554D5953C00: Received SSL data (21 bytes) LogHttp: VeryVerbose: 00000554D5953C00: Received SSL data (5 bytes) LogHttp: VeryVerbose: 00000554D5953C00: 'TLSv1.3 (IN), TLS Unknown, Certificate Status (22):' LogHttp: VeryVerbose: 00000554D5953C00: Received SSL data (1 bytes) LogHttp: VeryVerbose: 00000554D5953C00: Received SSL data (5 bytes) LogHttp: VeryVerbose: 00000554D5953C00: 'TLSv1.3 (IN), TLS Unknown, Certificate Status (22):' LogHttp: VeryVerbose: 00000554D5953C00: Received SSL data (1 bytes) LogHttp: VeryVerbose: 00000554D5953C00: 'TLSv1.3 (IN), TLS handshake, Certificate (11):' LogHttp: VeryVerbose: 00000554D5953C00: Received SSL data (2947 bytes) LogHttp: VeryVerbose: 00000554D5953C00: 'TLSv1.3 (IN), TLS handshake, CERT verify (15):' LogHttp: VeryVerbose: 00000554D5953C00: Received SSL data (264 bytes) LogHttp: VeryVerbose: 00000554D5953C00: 'TLSv1.3 (IN), TLS handshake, Finished (20):' LogHttp: VeryVerbose: 00000554D5953C00: Received SSL data (36 bytes) LogHttp: VeryVerbose: 00000554D5953C00: Sent SSL data (5 bytes) LogHttp: VeryVerbose: 00000554D5953C00: 'TLSv1.3 (OUT), TLS change cipher, Client hello (1):' LogHttp: VeryVerbose: 00000554D5953C00: Sent SSL data (1 bytes) LogHttp: VeryVerbose: 00000554D5953C00: Sent SSL data (5 bytes) LogHttp: VeryVerbose: 00000554D5953C00: 'TLSv1.3 (OUT), TLS Unknown, Certificate Status (22):' LogHttp: VeryVerbose: 00000554D5953C00: Sent SSL data (1 bytes) LogHttp: VeryVerbose: 00000554D5953C00: 'TLSv1.3 (OUT), TLS handshake, Finished (20):' LogHttp: VeryVerbose: 00000554D5953C00: Sent SSL data (36 bytes) LogHttp: VeryVerbose: 00000554D5953C00: 'SSL connection using TLSv1.3 / TLS_CHACHA20_POLY1305_SHA256' LogHttp: VeryVerbose: 00000554D5953C00: 'ALPN, server accepted to use http/1.1' LogHttp: VeryVerbose: 00000554D5953C00: 'Server certificate:' LogHttp: VeryVerbose: 00000554D5953C00: ' subject: C=US; ST=California; L=Menlo Park; O=Facebook, Inc.; CN=.wit.ai' LogHttp: VeryVerbose: 00000554D5953C00: ' start date: Sep 17 00:00:00 2022 GMT' LogHttp: VeryVerbose: 00000554D5953C00: ' expire date: Dec 16 23:59:59 2022 GMT' LogHttp: VeryVerbose: 00000554D5953C00: ' subjectAltName: host "api.wit.ai" matched cert's ".wit.ai"' LogHttp: VeryVerbose: 00000554D5953C00: ' issuer: C=US; O=DigiCert Inc; OU=www.digicert.com; CN=DigiCert SHA2 High Assurance Server CA' LogHttp: VeryVerbose: 00000554D5953C00: ' SSL certificate verify ok.' LogHttp: VeryVerbose: 00000554D5953C00: Sent SSL data (5 bytes) LogHttp: VeryVerbose: 00000554D5953C00: 'TLSv1.3 (OUT), TLS Unknown, Unknown (23):' LogHttp: VeryVerbose: 00000554D5953C00: Sent SSL data (1 bytes) LogHttp: VeryVerbose: 00000554D5953C00: Sent header (346 bytes) - POST /synthesize?v=20221205 HTTP/1.1Host: api.wit.aiAccept-Encoding: deflate, gzipAuthorization: Bearer JBUHSQLG245LRZP7LD7OU4LXNDCKM4LEUser-Agent: voice-sdk-47.0.2,voicesdk-unreal,Windows-10.0.22621.1.768.64bit,Unknown,7420282D4C70A64AE9CB3F8197991890,Unknown,EditorAccept: audio/wavContent-Type: application/jsonContent-Length: 93 LogHttp: VeryVerbose: 00000554D5953C00: Sent SSL data (5 bytes) LogHttp: VeryVerbose: 00000554D5953C00: 'TLSv1.3 (OUT), TLS Unknown, Unknown (23):' LogHttp: VeryVerbose: 00000554D5953C00: Sent SSL data (1 bytes) LogHttp: VeryVerbose: 00000554D5953C00: Sent data (93 bytes) LogHttp: VeryVerbose: 00000554D5953C00: 'We are completely uploaded and fine'

And here is running the same program again, by just stopping and playing from editor again:

LogDebuggerCommands: Repeating last play command: Selected Viewport LogPlayLevel: PlayLevel: No blueprints needed recompiling PIE: New page: PIE session: MainLevel (9 Dec 2022, 17:02:52) LogPlayLevel: Creating play world package: /Game/UEDPIE_0_MainLevel LogPlayLevel: PIE: StaticDuplicateObject took: (0.001608s) LogPlayLevel: PIE: Created PIE world by copying editor world from /Game/MainLevel.MainLevel to /Game/UEDPIE_0_MainLevel.MainLevel (0.001675s) LogSubsystemCollection: Failed to initialize subsystem dependency (WorldPartitionSubsystem) LogChaos: FPhysicsSolverBase::AsyncDt:-1.000000 LogAIModule: Creating AISystem for world MainLevel LogPlayLevel: PIE: World Init took: (0.000777s) LogUObjectHash: Compacting FUObjectHashTables data took 1.00ms LogAudio: Display: Creating Audio Device: Id: 3, Scope: Unique, Realtime: True LogAudioMixer: Display: Audio Mixer Platform Settings: LogAudioMixer: Display: Sample Rate: 48000 LogAudioMixer: Display: Callback Buffer Frame Size Requested: 1024 LogAudioMixer: Display: Callback Buffer Frame Size To Use: 1024 LogAudioMixer: Display: Number of buffers to queue: 2 LogAudioMixer: Display: Max Channels (voices): 0 LogAudioMixer: Display: Number of Async Source Workers: 0 LogAudio: Display: AudioDevice MaxSources: 32 LogAudio: Display: Audio Spatialization Plugin: None (built-in). LogAudio: Display: Audio Reverb Plugin: None (built-in). LogAudio: Display: Audio Occlusion Plugin: None (built-in). LogAudioMixer: Display: Initializing audio mixer using platform API: 'XAudio2' LogAudioMixer: Display: Using Audio Hardware Device U34G2G4R3 (NVIDIA High Definition Audio) LogAudioMixer: Display: Initializing Sound Submixes... LogAudioMixer: Display: Creating Master Submix 'MasterSubmixDefault' LogAudioMixer: Display: Creating Master Submix 'MasterReverbSubmixDefault' LogAudioMixer: FMixerPlatformXAudio2::StartAudioStream() called. InstanceID=3 LogAudioMixer: Display: Output buffers initialized: Frames=1024, Channels=2, Samples=2048, InstanceID=3 LogAudioMixer: Display: Starting AudioMixerPlatformInterface::RunInternal(), InstanceID=3 LogAudioMixer: Display: FMixerPlatformXAudio2::SubmitBuffer() called for the first time. InstanceID=3 LogInit: FAudioDevice initialized. LogAudio: Display: Audio Device (ID: 3) registered with world 'MainLevel'. LogLoad: Game class is 'GameModeBase' LogWorld: Bringing World /Game/UEDPIE_0_MainLevel.MainLevel up for play (max tick rate 60) at 2022.12.09-16.02.52 LogWorld: Bringing up level for play took: 0.000535 LogOnline: OSS: Created online subsystem instance for: :Context_3 LogVoiceEncode: Display: EncoderVersion: libopus unknown LogOnlineVoice: OSS: StopLocalVoiceProcessing(0) returned 0xFFFFFFFF LogOnlineVoice: OSS: Stopping networked voice for user: 0 LogWit: Verbose: BeginPlay: adding synthesize response callback LogWit: Verbose: UTTSMemoryCache::GetClip: requesting clip with id (5491D623C9B07072B1E29D11EF2333E6FAC6B06F) LogWit: Verbose: UTTSMemoryCache::GetClip: clip does not exist in cache LogWit: Verbose: UTTSStorageCache::RequestClip: requesting clip (5491D623C9B07072B1E29D11EF2333E6FAC6B06F) with path (C:/Users/sbayo/AppData/Local/Wit/Cache/) LogWit: Verbose: UTTSStorageCache::LoadClipFromBinaryFile: clip does exist in cache LogWit: Display: ConvertTextToSpeechWithSettings: converting text (Hello World) with voice (Charlie) LogWit: Verbose: WriteJsonData: Old reader size is (0) LogWit: Verbose: WriteJsonData: Wrote (93) bytes. New array size is (93) New reader size is (93) LogTemp: Display: https://api.wit.ai/synthesize LogHttp: Verbose: FCurlHttpRequest::SetContentFromStream() - FMemoryReader LogHttp: VeryVerbose: FHttpRequestImpl::OnRequestProgress() LogHttp: VeryVerbose: FHttpRequestImpl::OnProcessRequestComplete() LogHttp: Verbose: 000005547B85CD00: URL='https://api.wit.ai/synthesize?v=20221205' LogHttp: Verbose: 000005547B85CD00: Verb='POST' LogHttp: Verbose: 000005547B85CD00: Custom headers are present LogHttp: Verbose: 000005547B85CD00: Payload size=93 LogHttp: Verbose: 000005547B85CD00: request (easy handle:00000554ADF04000) has been added to threaded queue for processing LogWit: Verbose: SendRequest: Url is (https://api.wit.ai/synthesize?v=20221205), Content type is (application/json) and Content length is (93) PIE: Server logged in PIE: Play in editor total start time 0,088 seconds.

wangpingsx commented 1 year ago

Hi @GameBayoumy, How are you? hope you had a good weekend. Sorry for my slow reply. I got COVID and fully offline for a couple of days. I will take a deep look at these logs. Meantime, could you please do me a favor, try my latest code? Just clone our repos into the Plugins folder. (or use my below scripts)

cd Plugins
git clone https://github.com/wit-ai/voicesdk-unreal
git clone https://github.com/wit-ai/wit-unreal
GameBayoumy commented 1 year ago

Hi @wangpingsx, been doing well, thank you! I hope things are going better with your illness.

I've cloned the plugins inside my project and it gives me the same logs as mentioned above. For now I haven't looked further into the issue.

wangpingsx commented 1 year ago

Hi @GameBayoumy , How are you? Hope you had a good Xmas and happy New Year! I some more tests, but couldn't really reproduce this issue. Are you still facing the same issue? thanks

mszadko commented 1 year ago

I think i'm having same/similar issue. I tried to make couple different aproaches and with all of them I got same issue. I tired to follow this https://www.youtube.com/watch?v=UB4Uzmw8XA8 with no luck. I tried with sample project (one that comes in plugin from marketplace) and one that you need to download from github and then clone repositories into Plugins folder. I always added those 2 lines needed to Engine.ini file. First call of ActivateVoiceInput returns true. It triggers OnStartVoiceInput then OnMinimumWakeTresholdHit then OnStopVoiceInput and OnStopVoiceInputDueToInactivity/Timeout depends on how long I speak. And then no matter how long I wait I don't get any callback from wit.ai. No OnWitResponse or OnEntitiesMatched. Second try to call ActivateVoiceInput returns false with additional log LogWit: Display: SendTranscription: sending transcription (make the cube red) LogWit: Warning: SendTranscription: cannot send transcription because a request is already in progress or LogWit: Display: VoiceCapture - Init: init start - available (1) LogWit: Display: ActivateVoiceInput: activated voice input LogVoiceCapture: Warning: Resetting UncompressedAudioBuffer. LogWit: Display: BeginStreamRequest: starting stream request LogWit: Display: TickComponent: deactivating voice input - too long since activation (0) - too long since voice input (1) LogWit: Display: DeactivateVoiceInput: deactivated voice input LogWit: Warning: ActivateVoiceInput: cannot activate voice input because a request is already in progress

and this is from my app (not sample) where I'm calling print string whenever event is calles on VoiceEvents component on AppVoiceExperience

LogWit: Display: VoiceCapture - Init: init start - available (1) LogWit: Display: ActivateVoiceInput: activated voice input LogBlueprintUserMessages: [BP_Voice_2] OnStartVoiceInput LogWit: Display: BeginStreamRequest: starting stream request LogBlueprintUserMessages: [BP_Voice_2] OnMinimumWakeTresholdHit LogBlueprintUserMessages: [map_C_1] true LogVoiceCapture: Warning: Resetting UncompressedAudioBuffer. LogWit: Display: TickComponent: deactivating voice input - too long since activation (1) - too long since voice input (0) LogWit: Display: DeactivateVoiceInput: deactivated voice input LogBlueprintUserMessages: [BP_Voice_2] OnStopVoiceInput LogBlueprintUserMessages: [BP_Voice_2] StopVoiceInputDueToTimeout

any ideas how to fix that?

p.s. i thought it might be some connectivity issue but i typed curl ^ -H "Authorization: Bearer serveraccestoken" ^ "https://api.wit.ai/message?***" in cmd and it return proper response.

p.s.2 I'm working on 4.27

p.s.3 even with fresh download of github sample project i cannot press ui buttons send/activate because IsRequestInProgress returns true

mszadko commented 1 year ago

Also I noticed that after I closed project I cannot delete log file and something is still writing to it [2023.01.25-14.16.56:388][773]LogHttp: Display: Http module shutting down, but needs to wait on 11 outstanding Http requests: [2023.01.25-14.16.56:388][773]LogHttp: Display: verb=[GET] url=[https://api.wit.ai/message?v=20230104&q=make%20cube%20red] refs=[1] status=Processing [2023.01.25-14.16.56:388][773]LogHttp: Display: verb=[GET] url=[https://api.wit.ai/message?v=20230104&q=make%20the%20cube%20red] refs=[1] status=Processing [2023.01.25-14.16.56:388][773]LogHttp: Display: verb=[POST] url=[https://datarouter.ol.epicgames.com/datarouter/api/v1/public/data?SessionID=&AppID=UEEditor.Rocket.Release&AppVersion=4.27.2-18319896%2B%2B%2BUE4%2BRelease-4.27&UserID=&AppEnvironment=datacollector-binary&UploadType=eteventstream] refs=[1] status=Processing [2023.01.25-14.16.56:388][773]LogHttp: Display: verb=[POST] url=[https://api.wit.ai/speech?v=20230104] refs=[1] status=Processing [2023.01.25-14.16.56:388][773]LogHttp: Display: verb=[POST] url=[https://datarouter.ol.epicgames.com/datarouter/api/v1/public/data?SessionID=&AppID=UEEditor.Rocket.Release&AppVersion=4.27.2-18319896%2B%2B%2BUE4%2BRelease-4.27&UserID=&AppEnvironment=datacollector-binary&UploadType=eteventstream] refs=[1] status=Processing [2023.01.25-14.16.56:388][773]LogHttp: Display: verb=[POST] url=[https://api.wit.ai/speech?v=20230104] refs=[1] status=Processing [2023.01.25-14.16.56:388][773]LogHttp: Display: verb=[POST] url=[https://datarouter.ol.epicgames.com/datarouter/api/v1/public/data?SessionID=AppID=UEEditor.Rocket.Release&AppVersion=4.27.2-18319896%2B%2B%2BUE4%2BRelease-4.27&UserID=&AppEnvironment=datacollector-binary&UploadType=eteventstream] refs=[1] status=Processing [2023.01.25-14.16.56:388][773]LogHttp: Display: verb=[GET] url=[https://api.wit.ai/apps?v=20230104&offset=0&limit=10] refs=[1] status=Processing [2023.01.25-14.16.56:388][773]LogHttp: Display: verb=[POST] url=[https://api.wit.ai/speech?v=20230104] refs=[1] status=Processing [2023.01.25-14.16.56:388][773]LogHttp: Display: verb=[GET] url=[https://api.wit.ai/message?v=20230104&q=make%20the%20cube%20red] refs=[1] status=Processing [2023.01.25-14.16.56:388][773]LogHttp: Display: verb=[POST] url=[https://datarouter.ol.epicgames.com/datarouter/api/v1/public/data?SessionID=&AppID=UEEditor.Rocket.Release&AppVersion=4.27.2-18319896%2B%2B%2BUE4%2BRelease-4.27&UserID=&AppEnvironment=datacollector-binary&UploadType=eteventstream] refs=[1] status=Processing [2023.01.25-14.16.56:388][773]LogHttp: Display: cleaning up 11 outstanding Http requests. [2023.01.25-14.16.56:388][773]LogHttp: Display: Sleeping 0.500s to wait for 11 outstanding Http requests. [2023.01.25-14.16.56:889][773]LogHttp: Display: Sleeping 0.500s to wait for 11 outstanding Http requests. [2023.01.25-14.16.57:390][773]LogHttp: Display: Sleeping 0.500s to wait for 11 outstanding Http requests. ...

GameBayoumy commented 1 year ago

I haven't looked further into it. Also think it is not related to wit plugin, but the HttpRequest like mentioned here: Not getting any response from wit.ai