search

gtreshchev / RuntimeAudioImporter

Runtime Audio Importer plugin for Unreal Engine. Importing audio of various formats at runtime.
MIT License
309 stars 67 forks source link

oddness with the Calculate Frequency Spectrum plugin #8

Closed CANA-Dan closed 2 years ago

CANA-Dan commented 2 years ago

im unsure why your plugin gives issues because it seems to work comfortably with everything else. i think the calculate frequency spectrum node directly accesses the samples in order to create the spectrogram, and the audio likely needs to be fully decompressed. EDIT: i did some looking and indeed it looks like unreal thinks the audio file contains 0 samples.

do you have any ideas what could be causing the issue? iv also printed the Enum and it would always say success. on a secondary note, do you think the most recent versions of your program would be compatible with 4.26? i see that you added ogg support, and that would be amazing to have on that older version of the editor.

this is what i have for my node setup. image

for a quick comparison, this is a file imported using your plugin (i get this result from a regular audio file, and from a raw audio file) image

and this is an asset in unreal engine. image

gtreshchev commented 2 years ago

If you mean the Sound Visualization plugin, then you need to take into account that it only works in the editor and will not work in packaged build (it is all wrapped in WITH_EDITORONLY_DATA and the buffer used is relevant only in the editor).

The problem is that it uses a RawData buffer that is not used in the plugin at all. The plugin uses its own buffer to store uncompressed data in 32-bit IEEE floating point PCM format.

I would recommend writing own implementation for calculating the required values. UImportedSoundWave has a PCMBufferInfo, which contains all the required data.

BTW I have a plugin that can help with envelope analysis and beat detection that works in conjunction with the Runtime Audio Importer: https://github.com/gtreshchev/AudioAnalysisTools

CANA-Dan commented 2 years ago

hmm. well good to know that the sound visualization plugin wont work as a packaged project. i looked around in blueprints but PCM Buffer info didnt seem to exist, so im going to assume that its a C++ only thing.

for your Audio Analysis plugin, does it give spectrum data? im looking through it but i dont see anything of that sort.

i do know someone who can likely help me make an analyzer using another plugin as a base (that plugin ins currently broken for the latest version) https://github.com/leereilly/eXiSoundVis. that would also allow me to multi thread it too (current spectrogram is pretty slow if i start upping the quality).

gtreshchev commented 2 years ago

Yes, working with an audio buffer is an expensive operation for blueprints, so it is cpp-only.

Audio Analysis Tools can't analyze the audio spectrum at the moment, but there are plans to implement it in one of the next updates.

CANA-Dan commented 2 years ago

any ideas for when that may happen? unsure if i should spend the next month learning how to work with C++ in unreal engine and how to make that spectrogram, or if i should wait for that spectrum if you are planning on adding.

gtreshchev commented 2 years ago

I have very little time, so I'm not sure exactly when. I would recommend starting learning C++ any way :)

In the future I will implement a more advanced work on analyzing audio data such as FFT Magnitude Spectrum, but this will not be soon since I have little free time.

CANA-Dan commented 2 years ago

alright. thanks a ton as well. was sorta lost what to do there for a bit.