adaptive Music Utility for Godot

[betalars]

Describe the project you are working on

A game that has an adaptive Soundtrack.

Describe the problem or limitation you are having in your project

Syncing Audio Players, using Animation-Trees to manage Music Clips, doing transitions/intros/endings ... it's really awkward right now. I have not even found a native way to keep a piece of music playing when an animation that started it ends, because even when using a PropertyTrack to set an AudioPlayer to be plying, Godot will be smart and treat it as an AudioPlaybackTrack. I may be severely overlooking something, but I also have not found great answers in online resources.

Describe the feature / enhancement and how it helps to overcome the problem or limitation

I can imagine taking a look at Wwise and trying to parody some of what they are doing as a single powerful dynamicSoundtrackPlayer will greatly improve Godots Utility while keeping it compatible with Industry Standard Workflows, Composers are already used to.

But right now I think making strategic changes to the existing Animation Utilities to make them easier to work with when creating dynamic Soundtracks may the solution, that is going to be easier to maintain long term while also providing more interoperability for instance when making rhythm games.

Describe how your proposal will work, with code, pseudo-code, mock-ups, and/or diagrams

I'd honestly like to procastinate on the specifics unless we have a general discussion if this should be a core feature and if it is supposed to become it's own thing or fit into the existing Animation System.

Will probably appending this section after collecting some feedback.

If this enhancement will not be used often, can it be worked around with a few lines of script?

Bodgy Music Players are pretty easy to script, but it gets really difficult really fast.

Is there a reason why this should be core and not an add-on in the asset library?

It's just a really fundamental Feature. I know there have been add-ons before, but none of them have made it to godot4 yet and having a good way to do adaptive Music is IMO as fundamental as things like SCG-Shapes.

[betalars]

So I've just realized this whole thing might become a whole much bigger beast with rhythm based games, like for instance with Remember Me.

Having a core utility I would call a "conductor" that allows you to have a rythm in your Scene, to cue sound effects based on that rythm, to handle input based on rythm while having koyote-frames (so it can handle inputs even when they are slightly off beat) sounds like an incredible utility, that will make developing a lot of games a lot easier.

But also ... the scope of this would be substantial.

[Mickeon]

Not all games, not even most, may necessarily benefit from this. Therefore it may be better off as an addon.

[bend-n]

I disagree, I think this is quite a useful feature, and should be in core. As godot already has many audio utilities (generator, bus, doppler, attentuation) i think adding some more is okay. However, this proposal isnt very well fleshed out, and the issue seems to be being unable to keep a stream playing with a animation.

[betalars]

Not all games, not even most, may necessarily benefit from this. Therefore it may be better off as an addon.

The thing is: implementing it on an engine level by improving the animation systems to work better with rythm is imo a much better way to go about it, and choosing this approach would be really clunky to do as an AddOn.

And like ... most games made in Godot will not use localisation, but there's a reason it's natively supported and I think this falls into that category.

[AThousandShips]

And like ... most games made in Godot will don't use localisation

While it is arguably off topic I would like to see where you get this from? Also the editor uses localization.

How does this https://github.com/godotengine/godot/pull/64488 fit this proposal?

[bend-n]

Quite well imo! i already have a thumbup on that pr.

[AThousandShips]

Be aware that an addon can be built as a module an be fully integrated, though requiring you to rebuild the engine, so performance doesn't really come into it

What does come into it however is the workload in maintaining any feature, the size taken up by it, etc.

[betalars]

And like ... most games made in Godot will don't use localisation

While it is arguably off topic I would like to see where you get this from? Also the editor uses localization.

How does this godotengine/godot#64488 fit this proposal?

Quite well looking at it broadly, I only checked for adaptive music. Will probably close this issue once I've had a closer look at it.

[betalars]

How does this godotengine/godot#64488 fit this proposal?

I've looked at it an I'm not going to close the issue right now, as I'm not sure if the Merge is going to go trough and if it is the best approach. I feel like retro-fitting an animation-system to an audio player is really redundant when we have an animation system already, that we could just make more compatible with music.

[AThousandShips]

Then the issue of feature bloat and maintainability remains, will quote myself from another proposal: As this looks like a relatively niche feature, most projects are likely to use pre-processed audio as far as I know, there's many arguments against making it part of the engine, it adds to the size of the source and the executable and exports, and is harder to maintain, as a dedicated plugin or module maintained by people focused on it makes this much easier and smoother without complicating the situation for those who don't need it or use it.

[betalars]

Then the issue of feature bloat and maintainability remains

I don't think so honestly. I've looked further into AnimationTree and AnimationPlayer and I honestly think these components in conjunction are at 90% of what we'd need for a great adaptive Music System.

I really need to update my initial comment, but I think it's kind of wasteful to not fix the remaining issues with these systems, that prevent them from being useful to play music instead of building a new Animation System into the AudioStreamPlayer, witch seems to be what the Godot Devs are currently trying to do to accommodate adaptive music.