schrum2 / MM-NEATv2

MM-NEAT version 2.0 is no longer supported. Please get MM-NEAT 3+ from https://github.com/schrum2/MM-NEAT
Other
11 stars 5 forks source link

SoundUtil #271

Closed schrum2 closed 7 years ago

schrum2 commented 7 years ago

Create a new class called SoundUtil inside of the package edu.utexas.cs.nn.util. This class will be full of public static methods that can do various tasks related to sound processing. We know that we want to eventually evolve sounds/music in some manner, but first you need to learn what is possible using sounds in Java. Since we aren't fully certain what we need yet, go ahead and explore a variety of options. Here are some ideas for good methods to start with:

Methods that load/play MIDI files. Methods that load/play WAV files. Methods that load/play MP3 files. Methods that take a given sound format (WAV or MP3) and adjust the volume, speed, etc. Methods that take a MIDI, WAV, MP3 file and extract some more primitive representation from them, such as an array or arrays of numbers. Methods that create MIDI, WAV, and/or MP3 files from a primitive representation, such as an array of numbers. Methods that combine two WAV, MP3, MIDI files together into a single file, sequentially. Methods that blend/layer two sound files on top of each other. Whatever else you can think of ...

Be sure to specify yourself as the author at the top of the file, and to thoroughly comment all code/methods so that you and future students will understand the code.

Please commit the code frequently. As long as the code still compiles, it is ok to commit it, even if it doesn't work fully ... but be sure to leave comments explaining problems when you commit non-functional code. Also make sure you update this GitHub "issue" to track progress on implementing the methods above.

schrum2 commented 7 years ago

It looks like an external library might be the best way to go, but which one to use? Let's not commit to one yet, but here are several interesting options: https://code.google.com/archive/p/mp3transform/downloads http://www.javazoom.net/javalayer/sources.html (this one seems to be the most popular) http://www.javazoom.net/jlgui/api.html

schrum2 commented 7 years ago

Went with the JLayer library. Hopefully this will serve our purposes

schrum2 commented 7 years ago

Useful page for understanding the WAV file format: http://soundfile.sapp.org/doc/WaveFormat/

schrum2 commented 7 years ago

I found this useful site: http://ganeshtiwaridotcomdotnp.blogspot.com/2011/12/java-extract-amplitude-array-from.html I think you can copy the code more-or-less directly into the SoundUtil class, but change all of the methods to static methods, and avoid the global instance variables. Some of the code be even be redundant (you may already have some code that accomplishes these steps) ... it takes a sound file and breaks it down in many steps, until eventually you have an array of amplitude values.

Once this all works, we can see if there is a way to do the process in reverse.

schrum2 commented 7 years ago

I believe that this code provides a means of creating sounds derived from an array of amplitude values: http://introcs.cs.princeton.edu/java/stdlib/StdAudio.java.html

Pay particular attention to the play(double sample) and play(double[] samples) methods

schrum2 commented 7 years ago

This webpage also looks useful: http://archive.oreilly.com/oreillyschool/courses/data-structures-algorithms/soundFiles.html However, it is a bit dense. Some of it looks like it already duplicates procedures we are already aware of, but I saw a mention of a Fast Fourier Transform, which is something I think we might need eventually. Save this site for later in case we need it.

schrum2 commented 7 years ago

Another page that was too dense to look closely at, but that might be useful later: https://0110.be/releases/TarsosDSP/TarsosDSP-1.7/TarsosDSP-1.7-Manual.pdf

Also, regarding the StdAudio.java file mentioned above. Rather than copy bits and pieces of that code, we may want to copy the whole file into our project ... it could be useful as a utility class of its own.

Move SoundUtil.java and the new StdAudio.java files into a package edu.utexas.cs.nn.util.sound

schrum2 commented 7 years ago

Have a methods that do the following:

Take an amplitude array, and save it as a WAV file Take an amplitude array, and save it as an MP3 file

(the next two methods can use the previous two)

Take a CPPN, generate an amplitude array, and save that as a WAV file Take a CPPN, generate an amplitude array, and save that as an MP3 file

twerisa commented 7 years ago

http://www.labbookpages.co.uk/audio/javaWavFiles.html useful link regarding writing wav files

twerisa commented 7 years ago

Solved issues with saving playable CPPN file (issue #310)

schrum2 commented 7 years ago

I'm going to close this issue. Even though you will continue to add code to these utility classes, I think that those should be associated with more specific issues. However, the general framework for sound manipulation clearly exists already ... even if there are still some improvements needed.