[1.16.4] Multiple waterfall sounds prevent other sounds from playing.

[Ozzuneoj]

DynamicSurroundings-1.16.4-4.0.4.1

forge-1.16.5-36.1.0

https://pastebin.com/PFwR17pX

(no crash)

I will just make this a quick description, in hopes that the problem is simple enough that it is easily repeatable.

If I am near a slowly sloping waterfall (1 block steps basically) that is several blocks wide and has a lot of vertical room to flow, the waterfall sounds seem to overload the sound engine and prevent any other sounds from playing. For example, blocks breaking, lava sounds etc.

I do have several other mods installed (Terraforged, Optifine, Biomes O Plenty and others), but this is the only one that focuses on sound.

It is fairly simple to test. Place many water buckets on a hill that isn't very steep (roughly 1x1 block steps) to create a wide waterfall with many "steps", then try to break blocks to make sounds. The closer I am to the waterfall (starting at about 7-12 blocks distance depending on the situation), the more sounds are missing.

I also noticed that in the debug screen it is showing "Sound Control Sound Processor" hovering in the 5-6ms range when it starts missing sounds. In the center of a waterfall I've seen it reach over 9ms.

This is on a system with a Ryzen 5 3600, 16GB DDR4, Xonar DX sound card on Windows 10.

It isn't constant but I frequently see messages like these in the log: [31Mar2021 01:55:15.664] [SoundControl Sound Processor/WARN] [SoundControl/SoundFXProcessor]: SoundControl Sound Processor is lagging; behind 32 msecs [31Mar2021 01:55:15.792] [SoundControl Sound Processor/WARN] [SoundControl/SoundFXProcessor]: SoundControl Sound Processor is lagging; behind 26 msecs

I also noticed this: [31Mar2021 01:54:17.254] [Render thread/INFO] [SoundControl/SoundUtils]: Version: 1.1 ALSOFT 1.19.1 [31Mar2021 01:54:17.254] [Render thread/INFO] [SoundControl/SoundUtils]: Renderer: OpenAL Soft

Is it possible to use OpenAL hardware acceleration for this mod or for Minecraft in general? Many sound cards support OpenAL, but I'm not sure if it's possible to make a game actually use this in Windows 10. Can we bring back Aureal now? ;)

Suggestion: If it is an issue of there being too many sounds playing at the same time (seems most likely), it may work to have the sound engine check for nearby waterfall sounds (or other duplicate sounds, like when there are 50 similar mobs in close proximity) and then only play a set number of them (specifically the ones closest to the player). With waterfalls, an alternative would be to add a few deeper sounding "bigger waterfall" sounds that get progressively larger-sounding and have one of those play in place of several smaller sounds. It may be helpful if these values are configurable, in case a specific situation (or system performance?) requires an adjustment.

[OreCruncher]

Can you hit F3 and post a screenshot of when this is happening? There would be info in that output that would tell me the load on the sound system.

As for OpenAL I leave it up to Minecraft to do the heavy lift on config. Only thing that my mod would do is adjust the config so that there are more slots available for sound effects. I can look into it - I know enough about OpenAL to be dangerous. :)

Regarding alternative approaches to waterfall, I have been giving this thought ever since I implemented waterfalls. I haven't come up with a good efficient way to do what you are suggesting.

[Ozzuneoj]

It looks like it is hitting the limit of 245 sounds. Even the shots that are far away are hitting the limit, even though I can't actually hear the waterfall at that distance. If there is a way to reduce the number of sounds coming from flowing water I think that'd be the most elegant way to fix the problem (since it would reduce the load on the system in general), but maybe a temporary fix could be to allow the maximum number of sounds to be increased in the config screen. Is there a way to edit this manually?

Also, I'm not sure if it helps or not, but the weather2 mod by Corosus (for 1.12.2) had waterfall sounds that got "larger" as the size of the waterfall increased. Yours sounds much smoother for small waterfalls because it is literally emitting more sound with each block, but if you're able to tell how weather2 does it, maybe that could give you some ideas as to how to handle large waterfalls.

https://www.curseforge.com/minecraft/mc-mods/weather-storms-tornadoes

I haven't done any game programming in many years, but it seems like it might be necessary to add some kind of special logic for the water sounds in particular, so that when a water sound is triggered it generates something to indicate "water sound is playing here". Then it checks for the number of those water indicators within 4 blocks or something. For every X number of indicators it finds, then pick the location closest to the player and have it play the next "largest" water sound, to make the stream\waterfall sound appropriate for it's size. This could potentially work progressively up to huge waterfalls with deep roaring sounds. To lessen the load (from all of the individual ones checking?), maybe something could be worked in that makes the larger sounds actually prevent the smaller ones from checking so often.

I don't know how much of a load this would create, or if this is even possible in Java, but I thought it might be worth posting.

Thank you for making such a nice mod and being so responsive with these questions. This mod really does a great job of enhancing the feel of the game. And thank you for also making it compatible with Sound Filters! I'm not using Sound Filters right now (while troubleshooting this), but I like the two slightly different takes on reflections and occlusions, and I'm so glad that I don't have to choose between Dynamic Surrounds and Sound Filters. These along with Ambient Sounds make for quite an experience. :)

[OreCruncher]

That is a pretty big fall. Most of the ones I come across max out at about 170 sounds or so.

There are 256 sound channels. Some portion is reserved for streaming sounds - that's the "+ n/10" in the debug display, and include things like background music and biome sounds that are streamed from data within the JAR. The rest are essentially "one shot" sounds (monster ambient, rain drops, etc.) that are read in their entirety and then played. 256 is the max number of sound channels that most sound systems support. There isn't a way to increase due these hardware limitations.

In terms of optimizing sound output from falls I had pretty much similar thoughts that you have. The environment is pretty dynamic and to make these types of calculations on the fly is a challenge. You can get a sense of how the sound effect processing (reverb, occlusion, etc.) is taking - about 8ms of a tick, and that is between two threads. And it only processes about 25% of the sounds during that interval. (Note that it does not take up client tick time - just comparatively it would take up 8ms of the client tick if done on the render thread.)

Back in 1.12.2 I had an option to cut off waterfall sounds based on intensity. The end result is that it would omit playing sounds for low strength - which based on your screenshots would be the majority of what you are experiencing. It wouldn't sound right, but it would prevent maxing out the sound system.

And yes, Sound Filters has a bit more reverb in their sound production as compared to DS. It's a "choose the sound you like best" type of thing. For me personally I like DS a bit more because it feels a bit smoother and reminds of some of the caves I have been in.

Overall I am still on the hunt for a "better way" of handling the waterfall sounds.

[Ozzuneoj]

Thank you again for taking the time to reply. That's a great explanation.

Is it possible to have a rule for those specific effects so that it won't play any more than 200 of them at a time? For example, when the water block is flowing, rather than just call that sound event, have it do something like this with a global variable\counter (except in actual useful Java programming language): if (water-sounds-playing < 201) then (water-sounds-playing + 1) and (play watersound.ogg) and (after X ticks (water-sounds-playing -1))

For that last bit, you'd be reducing the count by one a set amount of time\ticks after the sound has started playing (roughly the length of the effects?). This would keep the counter from going up forever. I have no clue if it's possible to count ticks like this without bogging down the game, but if it is this would eliminate any need for checking whether sounds are playing. It is somewhat hacky but any situations where this trick would be noticeable would be quite rare. As it is though, a large waterfall totally blots out all effects (mobs, blocks, etc.) which can really impact gameplay. If it's possible to simply put a cap on the number of water sounds playing, that could fix it.

Alternatively... you could simply have it check the number of the sound counter that already exists. If the number is over 220, don't play the water sound, as it is unlikely that any other sounds would do this. Surely that must be possible?

Had I any experience with Java, I would volunteer to get in there and look at it myself, but I'm probably years away from modifying other peoples' code without making it worse. heh. :)

[Ozzuneoj]

I saw that version 4.0.4.2 was just released and had the following change listed: "Option to configure the number of streaming sound channels. Defaults to 10. If a modpack has additional streaming sounds you may want to bump this value a little."

Does that setting have any impact on the issue mentioned in this thread?

Thanks. :)

[OreCruncher]

No, it does not. What that setting does is increase the number of sound channels reserved for streaming sounds. Some modpacks have a bunch of ambient sound mods installed, each wanting to use streaming sound channels. When the sound engine runs out of streaming sound channels these sounds will not play. (Streaming sounds are things like music, biome sound effects, possibly machine or other environmental sounds.)

I am still looking at the waterfall issue. I may reintroduce a setting that would allow you to select a sound cutoff for waterfall intensity. What this would allow you to do is not play small waterfall effects. The majority of the sounds that play in large falls are these small effects.

[Ozzuneoj]

I see. Thank you for the speedy explanation! :)

I have also been wanting to pick your brain regarding how feasible it is to make all ambient bird or village sounds have a source in the world. I have been using AmbientSounds3 along with Dynamic Surroundings and I recently noticed that almost all of the sounds in AS3 are stereo and play directly from the player (is this what you mean by streaming?). The problem with this is that any stereo sounds that are panned left or right stay there as you turn, and if you go into a cave they ignore all occlusion or reverb effects as far as I can tell. I haven't taken the time to dig into DS to see if it does something similar for birds and other things, but I know that at the very least your mod includes an option to convert stereo to mono (which is absolutely essential to place a sound in 3D space), so I figured you were familiar with this.

Also, I have recently discovered the mind blowing effects of enabling OpenAL's HRTF feature in Minecraft (something I was accustomed to with A3D and Sensaura 20 years ago), but it really makes me wish all of the ambient sounds were located in 3D space rather than just playing the way music would.

I'm imagining adding all sorts of things to the world now... but I don't know how much of it is even possible. For example, adding a random chance that one leaf block in a given area will generate a bird sound (with more logic than that, but you get the idea) so that all the ambient birds have a location in 3D space (distant ones could be more echo-y or muffled), making the sound of a blacksmith play when the blacksmith goes near the forge, randomly playing crumbling\rock sounds when walking in caves (adding to the footsteps) or even adding the sound of falling\tumbling rock bits on top of blocks below you when you are near the edge of a cliff\crevice or walking down (in caves the reverb would then be added by the system already in place... super immersive!). If I could program I'd love to experiment with this stuff. I think your mod is by far the closest to adding things like this (in fact you probably already do some of this).

Anyway, please forgive the tangent... I have been into game modding (mostly sound effects) and 3D sound for over 20 years, so all these things that breathe new life into Minecraft in this way are very interesting to me. :)

[OreCruncher]

I'll have to look into HRTF.

As for streaming 3D non-positioned sounds you are correct that panning can give it a bit of direction and does not sound good. In DS I compensate by setting the sound about 32 blocks above the players head, so when turning its not noticeable. Note that non-positioned (or global) sounds do not have to be streaming. Streaming just means that the OGG is streamed from storage (from with the asset folder in the JAR) rather than loaded all at once.

DS does have the ability to generate sounds based on block type. This is how the frog sounds are generated around lily pads, and maniacal laughter for soul sand in the Nether. For bird sounds DS has what is called a spot sound. It has a chance to play at a random location around the player and is based on the biome that the player is in. This differs from other sound mods because DS biome sounds essentially create the background sound for a biome, and things like spot sounds provide the accent.

Positioned repeating sounds is possible. In fact that is what a waterfall sound really is. It's on my list to work out. A suggestion made a long time ago was to have the jukebox occasionally play sounds similar to number stations when located in a wasteland type biome.

(I also want to be able to have custom biome sounds based on region, but I think that would require server side support.)

[OreCruncher]

@Ozzuneoj Check this JAR out. I added an option to enable HRTF in the Sound Control settings. By default it is disabled. If enabled it will turn on HRTF if it is in a disabled state. If the option is disabled DS will not do anything - it will leave the setting alone. For my system HRTF is automatically enabled due to the headset I am using.

[Ozzuneoj]

Very cool! I assume it is the same effect as setting "hrtf=true" in the alsoft.ini file as mentioned in the link above, except you've made an easy in-game toggle for it, right?

I think if you have a sound card or USB headset with virtual surround modes enabled that is why you've already got HRTF. It's crazy how few people make use of this though, as most of the time the settings are aimed at headphone use exclusively. I find that they work great for speakers as well (though my speaker setup is above average... not sure how it sounds on laptop or monitor\TV speakers). Of course, it can make individual effects sound a bit less clear because the sounds are being filtered constantly... I'm not sure if there is any way to adjust OpenAL's HRTF settings to do anything about that, but I remember games sounding like this 20 years ago with these effects enabled, so I doubt there's much that can be done about that. I think it's definitely worth the trade off for the amazingly accurate placement of sounds though. I put my headphones on my 7 year old daughter and she was pointing at stuff with her eyes closed with HRTF turned on... even as I spun and walked around. Very cool stuff. :)

Also, can you clarify what you meant before about wanting custom biome sounds based on region? Did you mean customizable by the player\client? Because it seems like you already have different sounds depending on the biome.

[OreCruncher]

Yes - does the same as adding/modifying the external ini. The OpenAL SDK that Minecraft is using has extensions that make it easy to configure.

By region I am meaning an area defined by coordinates and size (not biome). For example, a server may set up server spawn at a specific location and they would want to play a custom sound clip when a player is in that area. To do this would require server side support as well as some sort of ability to stream that sound from the server to the client. (I have to check into whether a resource pack can be downloaded from the server to client, and have the OGG files delivered through that mechanism. Need to do research.)

EDIT: I am adding back the value for waterfall cutoff. Setting the slider to 2-3 should eliminate a lot of the sounds that get played around large falls. It's not ideal, but helps out with sound overload.)

[Ozzuneoj]

Ah, I see what you mean now about the custom sounds.

I don't believe there is any way to deliver resources directly via the server. I have been running a customized modded server for some friends for a few years and looked into this pretty extensively. My wife and I do custom textures and I've done several sound effects, as well as modifying the lang file to create a certain theme. We have always either had to either send each player a resource pack to enable manually or deliver a resource pack through a remote download through the server.properties file.

Neither is really seamless and both have downsides. Downloading manually works basically flawlessly once the pack is enabled, but obviously requires the players to do extra steps and there's nothing stopping them from simply not using the resource pack (or installing it improperly).

The second option involves uploading a resource pack to, for example Dropbox, and then editing the server.properties file with the URL to download it (resource-pack=https://www.dropbox.com/s/ol4fdg34t3mt/your_resource_pack.zip?dl=1). When people join, if the URL is formatted properly and the website supports downloads from links (dropbox does as long as the link ends with a ?dl=1), the players will be asked if they want to download the resource pack. They can still choose not to of course. The most annoying downside is that it will have to reload the game resources once the players have joined, which causes a second or third really long loading screen (totally locks the system) if Optifine is being used. In the end, since I'm helping all my friends set up their clients, I just opted to give them the resource pack manually to avoid the irritating loading screens every time they join or leave the game.

In short... sending resource to clients can be done, but I don't think there is any way other than the two methods above.

Another alternative could be to use the game's built in Narrator speech synthesis system. That's a pretty incredible feature to have in a game and I think it is highly underutilized. I have no clue how much flexibility mod makers have to use it, but I'd love to see the ability to make it read back messages even when the player doesn't have it enabled. If it's possible for a server or datapack to initialize Narrator to read a specific message aloud without the client having a mod, that'd be amazing and potentially hilarious. I would definitely use it on our private server. 🤣 This is sounding like an entirely different, possibly server-side only mod though.

[Ozzuneoj]

By the way, the HRTF setting works great for me. I was able to remove the file I added to the Roaming folder and just use the setting in DS. :)

[Ozzuneoj]

One more question\suggestion.

Is it possible to make the "larger" waterfall sounds audible from farther away? Other than adjusting the volume\scale, I don't see any way to do this. Is it a Minecraft limitation or something that could be tweaked?

If a large waterfall runs into a chasm, you can't actually hear it because the sound seems to come from the bottom (which makes sense), but the sound fades really quickly as you walk away (and this is without occlusion for waterfall sounds).

I'm a big fan of roaring waterfalls (very atmospheric), so I've been tinkering with this a bit. I had mentioned before about making the waterfall sound meatier if it was bigger, but after doing some more testing I see that you actually did make it do this. I'm sorry I missed it before. I see now that it depends on how far a specific "flow" of water falls. I was curious as to what could be done with this, so I actually made some bigger\deeper waterfall sounds for waterfall3, 4 and 5 to test it out and it makes the larger falls sound more powerful. I made sure that they had a smooth loop (no obvious repeating sounds) and that the volume blended decently with your other water sounds. If you're interested I can send you the files to check out. The samples all came from royalty free effects found online that don't require any credits or anything. I edited, mixed and played with the EQ to get them to sound decent using Goldwave.

... thinking more about the louder sounds being audible farther away, now I'm thinking it'd be really neat if it could be done for explosions too... how about distant explosion sounds that have a delay based on how far away they are... that'd be amazing! Ah, I better stop. ;)

[OreCruncher]

Starting with 1.16.4 there is a way to do it, but requires modifying the sound.json file. Sounds now have an attenuation distance that could be set. For falls I have it configured to 22 (blocks). Normal attenuation distance is 16. This is something I keep tweaking.

[Ozzuneoj]

Interesting. Thank you!

Also, I just went back and played an old 1.12.2 setup that I used for years and it had a few sound related mods as well. It used Sound Physics for reverb and occlusion. To my surprise, this mod was actually updated for 1.16.3 (October 2020) so I was able to test it along side Dynamic Surroundings. There are obviously some conflicts, but if you have the opportunity could you test this out and tell me what you think? https://github.com/djpadbit/Sound-Physics/releases

This is the reverb\occlusion setup I was used to when playing the game for a few years and one thing that stuck with me was that sounds that play in a separate "area" (inside a house or down inside a huge ravine) carry their reverb through to the outside (or other rooms) to a greater degree than DS or Sound Filters. This is probably up to preference, but if you have a moment to take a peak at that mod, I'd be curious to hear your take on how different it sounds. The difference is most noticeable with mobs or waterfalls in a ravine or cave while the player is actually outside the cave.

I honestly had no idea that mod was supported beyond 1.12.2, so this came as a surprise to me. I'd be curious to know if there's any way to adjust settings in DS to have the reverb from caves\buildings more audible from the outside like this. If not, I understand. I'm willing to do the tinkering myself if possible, I'd just need some direction. Alternatively, if it isn't a huge amount of work to make DS play nice with this mod the way it does with Sound Filters, that would be amazing. I simply can't go without DS at this point. 😁

[OreCruncher]

Interesting. The sound reverb processing of DS is based on Sound Filters. I did do some tweaking of the algorithms for performance though, and I know that some of the occlusion/reflection numbers are different.

[Ozzuneoj]

Interesting. The sound reverb processing of DS is based on Sound Filters. I did do some tweaking of the algorithms for performance though, and I know that some of the occlusion/reflection numbers are different.

I'm not sure if Sound Physics does things differently than Sound Filters or if it just has some specific reverb settings cranked up. You should definitely give it a listen though. To my ears it sounds very different, especially when multiple areas are concerned (being outside of big rooms, caves, hallways, chasms, etc.).

EDIT: I see the creator of the mod actually mentioned using DS with his mod: https://github.com/djpadbit/Sound-Physics/discussions/64 I think some interesting things could come from a discussion between the two of you. 👍

[OreCruncher]

Sound Filters does things entirely differently in terms of calculations. To me Sound Filters has more echo rather than reverb. (It also has some performance challenges.) If Sound Physics is available for 1.16.x I can add code similar to what I did for Sound Filters (auto-enhanced sound processing).

[Ozzuneoj]

I just wanted to mention, I tweaked just a couple of settings in DS (with no other sound mods) and... I'm blown away.

I increased "block range" to 64, while also editing sounds.json to push the larger waterfall sounds (4 and 5) much higher (I did 42 and 64 respectively), with waterfall occlusion enabled, also using the "deeper" sounding waterfall effects I mixed together. Now, when anywhere near a big waterfall... it's super obvious. It's really impressive actually. I could actually hear a deep roaring sound inside a cave and I followed it to the end of the cave where it got slightly less muffled, then I dug through some blocks and the waterfall outside poured in... absolutely incredible!!! The occlusion muffled the effect appropriately, but allowed the lower frequencies of the deeper waterfall effect to not only play in the cave but reverberate through it...

When water is running into a ravine the reverb effects are now able to make up for the occlusion effects in such a way that you can hear the roar of the waterfall when approaching the ravine.

I have a fairly fast CPU (Ryzen 5 3600), so it's possible that these settings would bog down a slower system, but I noticed no performance difference.

This is what I've been missing! I will make a recording and post it here as soon as I can.

Is it possible to adjust the attenuation settings per-effect via a config file rather than having to edit the sounds.json in the DS .jar? Now that i think about it, simply adding the edited sounds.json file in a resourcepack would probably work...