WVAviator
commented
8 months ago Update: The plan is to use Pixabay sounds and music. Each game should have the following:
- Ambient music for the menu screen
- Ambitious music for the narrative
- Background ambient sounds or music for each scene
A selection of music and ambience should be carefully curated for the game. A JSON file should provide information about each asset - including the following:
- id: Probably just integers, or omit if using an array
- keywords: a series of keywords that describe the theme and feel of the song or sounds
- title
- artist
- link: A link to the artist page or song page
Most of the above information would make the most sense to include in a "credits" section later. The keywords will be how an AI chooses the appropriate sound or music for a scene. The AI will be provided a list of descriptive text for each scene, and be asked to select the most appropriate song or sound by id. The AI should be encouraged to choose similar sounds for similar areas in the game - for example if there are three scenes all located inside an ancient ruin, they should probably use the same ambient track. A well placed useEffect will keep the sound playing without interruption even when the scenes change in this case.
There should be sound and music in the app. To start with, just some typical burnt/mysterious music should play, and sounds for things like clicking, maybe footsteps when traveling to a new scene.
Potentially, as a stretch feature, music can be selected for a game. An AI model can be prompted to choose music based on the game summary. They would be provided with a list of music IDs and a description of of the type of mood or feeling that music conveys, and it would choose the best match.
Potentially, the same could be done for sounds, and the narrator could provide a sound in response to an action or scene change. This could get pricey regarding tokens though. If added, it would need to be opt in for the user.