gtreshchev / RuntimeAudioImporter

Runtime Audio Importer plugin for Unreal Engine. Importing audio of various formats at runtime.
MIT License
336 stars 69 forks source link

when I use Stream Sound Wave,append streaming audio from websocket,can not playback by order #52

Closed oneanime closed 1 year ago

oneanime commented 1 year ago

when I use Stream Sound Wave,append streaming audio from websocket,can not playback by order。Is it adding data asynchronously and unordered。 Feeling like the data in front is being overwritten by the data in the back.

And When I use Azure tts to generate audio data that format is wav and send it to ue through websocket,use this plugin streaming play sound,and the log show the wav format is wrong,but this audio can play in my java code

gtreshchev commented 1 year ago

If you want to ensure that the order is entirely accurate and sequential, I recommend you to append the audio data once the OnPopulateAudioData delegate is broadcast. That is, your algorithm might look like this: bind to the OnPopulateAudioData delegate, append audio data, and once the delegate is broadcasted, append the next part of the audio data again, and so on. This will ensure that the sequence is always correct.

Regarding the WAV audio data, you can attach it here and I can manually check what's wrong with it if you want. Regardless, I recommend you to verify that the format is 100% correct.

oneanime commented 1 year ago
1

It is my blueprint,I use the OnPopulateAudioData that reurn value is float array, I dont konw how to play it.
The binary is byte array that size is 16000 from java websocket,be pushed to ue5 one by one

gtreshchev commented 1 year ago

You don't need to play back an array of floats, this is not needed for your case. This OnPopulateAudioData delegate is broadcasted once the audio is populated by calling your AppendAudioDataFromEncoded function, which can serve as an indicator that the append function has finished. After this, you can call the AppendAudioDataFromEncoded again, ensuring a consistent and sequential order of appending.

gtreshchev commented 1 year ago

For your specific scenario, you can implement a queue for audio data, such as using a two-dimensional array wrapped in a USTRUCT if you're handling this logic in Blueprints. The logic could be to append the audio data from the queue only once the OnPopulateAudioData is broadcasted each time.

gtreshchev commented 1 year ago

Starting from 2ea924e40a0eaaa39b00f07a1f93573b09b5efd8, it should always append the audio data in order, no matter how frequently you perform the append.