kniEngine / kni

KNI is a cross-platform C# game framework.
Other
128 stars 9 forks source link

Add a SoundEffectProcessor Parameter to convert sounds to Mono channel. #1478

Open nkast opened 5 months ago

nkast commented 5 months ago

For those reasons it would be nice to have an option in the Pipeline, instead of converting the original asset.

damian-666 commented 4 months ago

I'd be happy with just MONO, the default WAV file, and the reverb filter and the bit of code that adjusts the filter from the SILK presets to the OpenAL one to make the values in the preset match sound a bit more like it...and have dB and units documented. I bet it's easier in XNA but I was hosting in Avalonia then using GL and OpenAL. The best thing is someone tried to make Android sound like XNA via manual tweaks.. a map of the XNA values to the equivalent values in OpenAL.

the best thing is someone tried to make android sound like xna via manual tweask. a map of the xna values to the equiv values in openAL. i would hope it finds whatever low latency dsp like asio is there. i only care about pc, adroid, then PC ARM 64 qualcom which has DSPs ,

so since im tweaking sounds in real time , based on physics impulses and ,i dont use the 3d Audio listener , i have my own pan , zoom , doppler. because my game is 2d.. i adjust the rate at which volume falls with distance also. the XAudio uses a dsp on intel if its available..

so since im tweaking sounds in real time , based on physics impulses and ,i dont use the 3d Audio listener , i have my own pan , zoom , doppler. because my game is 2d.. i adjust the rate at while volume falls with distance also. i add a bit more dispersion that if itwas pure 2d.

for short sounds i just load them from a DLL, add to own sound bank i made.. i add on any pans , or ptichs to the hash key and map them to isntances to see if a sound is already playing ( noises moistly)

teh audio presets should all be mono since this is for physiics sounds in my case.

with audacity reverbe doenst work on stereo i dont think or mp3

i has to be a clean signal and some high frequency may will not work w reverb.

so theh i redo the sound in audacity, download anotehr effect. there are EQ fitlers in there low pas and high, but i dont see a n interactive game use case for it much

the use case is guns shot and what ever volume cave you are in. i do some other ambience with winds and stuff.. or if something is between you and the sound. so i think 3daudio lister is too high level .

if its not real time i would desing the sound out of my level editor.

damian-666 commented 4 months ago

I'm doing some research on the Hexagon processor that comes with Qualcomm and now ships on most Windows tablets, Surface, and OEM devices. I don't see the value in people putting stereo sound effects into their package, and maybe it should just be a warning. I don't see the use case for stereo with 3D effects.

You could do that, but just output a warning that your track is being mixed down to mono. Then, they can just do that with Audacity and put the track back in. I don't think they should compress it, especially lossy. I'm not sure if 16-bit audio is good enough. But somebody advocated bringing it up to 24-bit. The Hexagon processor might convert it to fixed point, and then you don't have to, but you need that headroom just for your calculations, and then you can put it back to 16-bit PCM integers.

In some packages, like middleware Onyx or TensorFlow Lite, there is just a switch that says, "optimize for Hexagon." But I think OpenAL should have bugs logged if it's not utilizing that resource. Because OpenAL Soft is open source with hundreds or at least 30 actual hotword kernels that are listening and detecting OS with callbacks that go to the CPU and use the same memory. This isn't anything you can do with GPU acceleration, and it's been around for 10 years. So there's a lot of work to be done, especially in middleware, and it's huge because these DSPs have been underused over a decade. So all these reverb things are very expensive to do on software.

We can see that in OpenAL Soft, with the snaps, crackles, pops, and the garbage collection kicking in and using a lot of watts, and not being able to apply more than one voice to a sound isn't good enough for what is in your document on how to do that kind of reverb that uses the actual angles of the rays that you cast, which is way better than what I proposed. Compress this and make it work on Hexagon, and hopefully, it will do the best it can. Hexagons can handle integer math and any type of vector. They can be extremely low power, and that means you can listen and have 30 kernels listening at once to your microphone.

damian-666 commented 1 month ago

I updated the comment above. I've been doing more research. I'm trying to implement a hot word ,not a wake word, but a real hot word. Low latency machine learning thing and see if I can learn anything from that. Maybe even running 60 like 30 kernels at once, listening for up, up, left, left, right, right, down, down. All that stuff. I'll. I'll put it in the discussion. https://www.quora.com/Is-16-bit-44-1-kHz-audio-good-enough