gtreshchev / RuntimeAudioImporter

Runtime Audio Importer plugin for Unreal Engine. Importing audio of various formats at runtime.
MIT License
346 stars 69 forks source link

Audio compression #77

Closed ApoorvOtherhalf closed 1 month ago

ApoorvOtherhalf commented 2 months ago

Any way to compress the audio recorded from capturable audio component using vorbis and repopulating the audio data in float array?

gtreshchev commented 2 months ago

If I understand your question correctly, then yes, it is possible. Please follow these steps:

  1. Export the audio data to the Vorbis format, such as a buffer presented as an array of bytes (TArray), as documented here. Capturable sound waves support exporting audio data in the same way as imported sound waves.
  2. Import the audio data using the default RuntimeAudioImporter approach, which is described here.
  3. The resulting imported sound wave will have the audio data in its internal buffer, which can be retrieved as an array of floats using the GetPCMBuffer function mentioned at the top of this page.
ApoorvOtherhalf commented 2 months ago

Ah, thanks

ApoorvOtherhalf commented 2 months ago

Well, compression works so that's great and thanks for it. Although, while exporting capturable sound wave to vorbis, it converts whole audio data. What I need is to convert only the new recorded audio chunk and not the complete data stored in capturable sound wave component. For eg, let's say data "x" was recorded in 1st frame, I used export data to buffer in exported capturable sound wave to export in vorbis and did my further processing. New data "y" was appended in the next frame in the same capturable sound wave component. Now I need only the "y" data in vorbis and not "x + y" and so on. So the target is like:

Recorded data#1 -> export to vorbis -> process byte array recorded data#2 -> export to vorbis -> process byte array and so on..

Hope I was able to express. Thanks

gtreshchev commented 2 months ago

There is currently no functionality to specify the range of audio data samples to export, but you can release the memory of the sound wave so that no audio data is present, and then capture and export the audio data, ensuring that only the relevant portion is exported. The function is called ReleaseMemory, and you can find more information about it on this page: https://github.com/gtreshchev/RuntimeAudioImporter/wiki/3.-Playing-Audio

ApoorvOtherhalf commented 2 months ago

Thanks, much appreciated. Loving the plugin so far!

ApoorvOtherhalf commented 2 months ago

Problem: Audio chunk exported using ogg vorbis contains a lot of hitches Sorry to bother again, compression works flawlessly when done on a complete recorded audio at the end but, it exports chunks having a lot of hitches when done chunk by chunk. My procedure is:

  1. StartCapture()
  2. Listening to OnPopulateAudioData()
  3. As soon as it's triggered, I export CapturableSoundWaveComponent to ogg vorbis keeping quality at 100
  4. Using that data to do my processes
  5. Releasing memory when I'm done
  6. Repeat from Step 3

For testing, when I append all these chunks, audio is having too many hitches I believe OnPopulateAudioData() is the only event which is triggered whenever a chunk of audio is recorded. So my need is to:

  1. Know as soon as any audio is recorded
  2. export that chunk using ogg and performing my further calculations.
ApoorvOtherhalf commented 1 month ago

I tried creating new ImportedSoundWave component on start of OnPopulateAudioData(), populating it will PCM data and broadcasting inside the very same OnPopulateAudioData(). On exporting new component to vorbis and merging later, sound's still noisy and jittery.

ApoorvOtherhalf commented 1 month ago

Achieved by exporting PCM data to another custom PopulateEvent.