I played around with the settings a bit. I think I set some of these numbers too high, but I don't want to mess with it as it sounds much better. I think what really solved it was changing bitrate-type="constrained-vbr".
Here's all the stuff I threw in there:
From Discord: