Request: more flexibility for handling sound sources in filled voxels

[borogove]

From the "Bake Resolution" doc:

Sound sources can't be located inside "filled" voxels (i.e. voxels that contain geometry). This results in no sound.

I'm concerned that we're going to "lose" a lot of audio emitters in walls this way. For a level/sound designer, the most natural placement of some ambient sound emitters will be slightly inside a wall or object (consider a bulky piece of machinery built as static mesh; conceptually, the hum of the machinery comes from within). I don't want to burden designers with having to check the voxelization for every emitter placed. Similarly, a moving object whose collision volume is smaller than the voxel grid might end up against a wall with the emitter embedded.

I understand that an unlimited search for the nearest open voxel could be both potentially very expensive and could give unwanted results, but some facility to automatically cope with just-barely-embedded sources would be great. A possible solution: have an "open voxel search radius" parameter per emitter, cache the result of an open voxel search until the next time the emitter moves, and log a warning if an object is doing a lot of searches.

At a minimum, if an API was provided to determine if a given location is filled or open in the voxelization, app-specific solutions could be assembled pretty straightforwardly.

[nikunjragh]

@borogove: This is a good point and a common practical issue, we agree that fixing this would boost practical usability of the system.

Disregarding CPU expense, an automated search has the following downside. Suppose there is wall clock and sound source is right on the wall. The search for a free voxel can succeed on either side of the wall, and if the automatic search starts behind the wall and succeeds, we have caused more harm than good and worse, the system messed up but thinks it has succeeded, which would make debugging basically impossible.

But user provided app-specific hints seems quite a viable direction. What do you think of this. A hint in the API for a "push" direction and distance. If the query fails, the system will search along a straight line in the push direction up to the provided distance. This bounds cost as it is a linear search (as opposed to volumetric) up to some distance, and presumably, this can be assigned based on surface normals or other app-dependent parameters, such as opposite to the impact velocity of a projectile.

A related option that you can explore is to look at the debug drawing code for voxel data in editor. This shows you how to get at the internal voxel data, so it won't be too hard to rig up a system that warns the designer when placing emitters that they are inside and will fail at runtime.

As a note, for dynamic emitters the system should already do a half decent job. It persists last-known-good values, so a lobbed grenade will keep the good values it had right before hitting the ground.

Let us know what you think of all this - this is super informative feedback for us, thanks!

[borogove]

Suppose there is wall clock and sound source is right on the wall. The search for a free voxel can succeed on either side of the wall, and if the automatic search starts behind the wall and succeeds, we have caused more harm than good and worse, the system messed up but thinks it has succeeded, which would make debugging basically impossible.

I hear what you're saying but I only half agree; in most cases I'd rather have the ticking clock sounding in the wrong place than not at all.

But user provided app-specific hints seems quite a viable direction. What do you think of this. A hint in the API for a "push" direction and distance. ... this can be assigned based on surface normals

Gut feeling: If I'm not expecting the emitter to be embedded, I don't have much to go on to pick a direction; to use a surface normal I have to do a volumetric search in mesh-land instead of voxel-land. This might be a good argument for providing just a simple GetVoxelAtPosition() ->open, closed, or unavailable function -- either a box search or a line search is easy enough to write from that primitive. (Also possibly useful for other purposes, like the QueryDistance function!)

A related option that you can explore is to look at the debug drawing code for voxel data in editor. This shows you how to get at the internal voxel data, so it won't be too hard to rig up a system that warns the designer when placing emitters that they are inside and will fail at runtime.

I did poke around in there a little today. I'll see if I can set up a warning system.

As a note, for dynamic emitters the system should already do a half decent job. It persists last-known-good values, so a lobbed grenade will keep the good values it had right before hitting the ground.

That's good to hear.

I appreciate the responsiveness y'all have shown for my issues, by the way -- thanks!

[nikunjragh]

I hear what you're saying but I only half agree; in most cases I'd rather have the ticking clock sounding in the wrong place than not at all.

I read the code and I think the docs are wrong on this one, I will verify later and probably file a docs bug. You could also try placing a source inside geometry and see if you get the following behavior: Acoustics will recognize its failure and not do any Wwise occlusion or reverb calls. So the net effect should be an un-occluded, dry sound. This was certainly the original intent since disappearing-sounds-is-BAD is a lesson we learned thoroughly on Gears 4.

Gut feeling: If I'm not expecting the emitter to be embedded, I don't have much to go on to pick a direction; to use a surface normal I have to do a volumetric search in mesh-land instead of voxel-land. This might be a good argument for providing just a simple GetVoxelAtPosition() ->open, closed, or unavailable function -- either a box search or a line search is easy enough to write from that primitive. (Also possibly useful for other purposes, like the QueryDistance function!)

QueryDistance actually is much faster than any ray marching since it uses a completely pre-computed and smoothed spherical distance map. But your point is understood.

A primitive voxel query API in the form you're asking is already there in the voxel drawing code in AcousticsDebugRender.cpp. In fact it demonstrates the exact scenario you'd like: it takes a voxel volume in a box region around a point (the player location in this case, replace with emitter location for our scenario) using this call: auto voxelSection = tritonDebug->GetVoxelmapSection(minCornerIn, maxCornerIn); which returns a lightweight VoxelSection data structure which represents a box-section of the global scene voxel map.

Then this call: voxelSection->IsVoxelWall(x, y, z) can be used to do random-access indexing into the voxel section and will return true if voxel is solid, false otherwise. This voxel data is the exact same used for deciding if emitters are inside wall internally so if you find a free voxel and place an emitter there, query should succeed.

[borogove]

The search for a free voxel can succeed on either side of the wall

Now that I've tried it I see this is a bigger problem than I'd guessed -- a large static mesh box voxelizes as hollow. It would be nice if it could leverage the collision volume here, but I guess that's a big can of worms since collision often extends well beyond visual mesh.

[nikunjragh]

Yes, and to be sure, we could fill all closed volumes easily during voxelization - but then we will be imposing the requirement of water-tight meshes, with the risk of filling out the whole world due to just one non-water-tight mesh. So that's also a can of worms.

Which is why I was suggesting biasing the open-voxel-search to a line that consults some kind of reliable normal or other metadata on the game side. Inside the acoustics engine, we'd like to do something that really solved this problem without caveats (or at least, very clearly predictable caveats), which probably requires supplementing the voxel data with some kind of other metadata, like some actual triangle information, which will then increase RAM usage.

[nikunjragh]

Also - it is super cool that you were able to use the voxel API! I think its a matter of finding some reliable audio event metadata to cue the voxel search. For sounds generated by impact events, stuff like sound source velocity can be useful (search the other way). Our wallclock example is the tough one actually since there's no dynamic info. But thinking of that specifically, even in that case the clock has a preferred facing direction - so in the prefab/blueprint actor you could pull out the actual emitter location slightly in front of the watch - just needs ~10cm nudge, and it will work in all cases. In cases where pulling around sound sources doesn't work, like say a turbine fan, excluding the fan/axle geometry from the bake will work. Its a bag-of-tricks rather than one solution for sure, but on Gears we were able to get by with all this. The required nudge is small enough that it doesn't mess spatialization. Not the most satisfying answer, I know, but best we've understood this problem till now.

[nikunjragh]

As another thought, actually collision volume is a very promising solution, not for using directly for baking, but for nudging the emitter location - assuming it is easy for you to locate which collision volume to use. Then it amounts to finding the direction to the closest point on the collision volume surface and do your voxel search in a line along that direction. So collision volume provides guidance on how to exit, and you still use the detailed visual mesh voxels. Hope I'm explaining this right.

[MikeChemi]

The latest release provides some support for source-inside-voxel issues. #130