Feature Request: Ability to play arbitrary frequencies to play sounds without opening a file

[Aplosion]

What problem does this solve or what need does it fill?

Ability to play arbitrary sounds (frequencies) without using any assets

A description of why this particular feature should be added.

Debugging is a big reason. I can tell easily what's broken if, for example, a sound asset doesn't play but a frequency I wrote in my code directly does. I'm sure there's more advanced debugging uses as well, but I'm not advanced enough to know them.

But it's also got lots of use cases as a sound effect and/or music engine.

https://www.seventhstring.com/resources/notefrequencies.html Using fairly simple math, it's easy create any note you like, and if you want to use notes not on the standard scale, defining a new set of basic notes is as simple as dividing by a different number in this formula: desired_note=base_note * 2^(half_steps_from_base_note/12) (usually, A4 =440hz is the base note, which is roughly in the middle of the musical range. A half step below A4 is G#4, a half step above is Bb4, which are represented by -1 and 1 in this case.)

Example use cases that work better with directly coded sounds than imported files:

(would need too many files)

I want to make a level where the player's position corresponds to a specific frequency and if they jump, it distorts that frequency.

I want my underwater level's music to go slower as the player goes deeper into the ocean, but I don't want to distort the sound of the music by slowing the playback down.

I want to use the characters of a player's name to generate a unique theme.

(faster to just code it)

I have a breakout-like game, and I want to make a sound whenever the ball hits a brick. I want to make the sound get higher and higher with each successive hit.

I'm rapid prototyping a 2d platformer. I want to have a sound for jumping, taking damage, and getting an item, but I don't especially care what they sound like, only that they're easy to tell apart.

(I'd rather use math than learn how to use advanced music software)

To get a spooky variation of the game's main theme, I want to run it through a sine wave.

I'm making a rhythm game and I want the notes to sound slightly off-key if the player hits them slightly off-beat, and I want it to scale with how far off they are.

I want to get a compressed-sounding version of a song by rounding notes wrong.

What solution would you like?

I don't know what the under-the-hood implementation is (but https://docs.rs/twang/latest/twang/ can probably help with that), but I basically want, at minimum, a function that looks like this:

play_note(frequency_in_hertz , duration);

with support for playing overlapping notes

I'd additionally like

play_sequence();

which is a bit more complicated, and I'm not sure exactly how you'd implement it. It'd take some data structure containing note floats, and instructions on when and how long to play them, and ideally have a easy way to indicate some sequence of notes is a chord.

The solution you propose for the problem presented.

What alternative(s) have you considered?

You can import a sound file note as a base note and operate on it to get all other sounds, but this means that the code relies on that file, so debugging, code review, and getting help are all more difficult. Depending on the file type and the operations you're performing on the file, it might also cause undesirable sound distortions and performance issues.

Other solutions to solve and/or work around the problem presented.

Importing files works fine for a lot of things, but doesn't help new users who want to copy-paste code, is a bit hard to debug. Importing other crates to do this work is also fine, but having an integrated system makes it easier to promise it will function in the context of this game engine.

Additional context

Any other information you would like to add such as related previous work, screenshots, benchmarks, etc. all pretty much inspired by this (to run this code on a normal python version, winsound on windows, not sure on other platforms): https://blog.wokwi.com/play-musical-notes-on-circuitpython/

I also used this for reference on the note frequency formula I used:

https://www.translatorscafe.com/unit-converter/en-US/calculator/note-frequency/ this video on music trackers and david wise's work on Donkey Kong Country: https://www.youtube.com/watch?v=jvIzIAgRWV0

[Aplosion]

@alice-i-cecile Enhancement PR, let me know if it looks all right, I'm happy to rework it a bit if not.

[alice-i-cecile]

I'm quite happy with this issue description. Clearly motivated, some helpful links. Similarly, I'd be happy to see a PR for this and would use it in the Breakout example :)

[LiamGallagher737]

Would we be able to achieve the playing a single note part using rodio's SineWave? Definition: https://docs.rs/rodio/latest/rodio/source/struct.SineWave.html Usage: https://docs.rs/rodio/latest/rodio/index.html#:~:text=SineWave%3A%3Anew(440.0)

[alice-i-cecile]

That looks like a very promising place to start.

[mockersf]

Exposing a basic function to play a SineWave should be simple.

I've been toying with rodio functions to find a more general API for sound generation but didn't manage a nice one yet...

[zoalst]

I made a quick hack of bevy's breakout example which uses dasp to generate and play 1 second sine waves in C Major instead of playing an audio file on each collision. It can be a little harsh sounding because I let the sine waves stack on top of each other, but the general idea for 'play_note()' is there. I'd imagine a fully fleshed out feature would allow for custom synths using dasp instead of just sine waves, since dasp has all the building blocks for sound synthesis.

For the 'play_sequence()' functionality I'd imagine it would be a separate, more complex feature using midi, which could play sequences of dasp sounds or audio files, and would be able to dynamically control aspects such as tempo or pitch bend.

[basilefff]

Hello! Is this issue still relevant? If so, I would like to try my hand at it. I have two ideas of how to do that:

• Implementing NoteBundle = AudioSourceBundle<Note>, where Note has two parameters: frequency and duration.
• Implementing some way to forward rodio::source for use with bevy systems. This would allow users much more options for mixing different sounds, but I ran into some obstacles when trying to do it this way.

Which approach would be better?

[alice-i-cecile]

Yep, this is still needed :)

I would prefer the former: there's a very high chance that we move off rodio at some point, so presenting a backend agnostic API is a better path forward.

[Aplosion]

I'd still like to have it, yes. I'm okay with the simpler option, as there are fully-featured audio packages in Rust, but I just wanted something basic for prototyping.

On Thu, Jul 20, 2023 at 2:02 PM Василий Чай @.***> wrote:

Hello! Is this issue still relevant? If so, I would like to try my hand at it. I have two ideas of how to do that:

• Implementing NoteBundle = AudioSourceBundle, where Note has two parameters: frequency and duration.
• Implementing some way to forward rodio::source for use with bevy systems. This would allow users much more options for mixing different sounds, but I ran into some obstacles when trying to do it this way. Which approach would be better?

— Reply to this email directly, view it on GitHub https://github.com/bevyengine/bevy/issues/7515#issuecomment-1644363011, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGEBQWYRIAEBYIVWVLOVZPDXRFXB5ANCNFSM6AAAAAAUR4JIO4 . You are receiving this because you authored the thread.Message ID: @.***>

[basilefff]

I would love to get your feedback for my PR

[basilefff]

So, should this issue be closed or something more needs to be done first?

[alice-i-cecile]

Yep, should be closed. I missed it when merging!