Implement recording samples from AudioIn (and other sound sources)

[dhowe]

I'm trying to build a simple demo where you start recording with a keypress from AudioIn, then stop the recording, see the waveform of the recorded sound, and hear it playing. Seems like it should be trivial, but I don't find the needed methods... How to record a sample then play it? thanks

AudioIn input;
Amplitude loudness;

void setup() {
  size(100, 400);
  noStroke();
  input = new AudioIn(this, 0);
  loudness = new Amplitude(this);
}

void keyPressed() {
  if (keyCode == 32) {     // start record
    loudness.input(input);  
  }
  if (keyCode == 10) {     // stop record

    input.stop();   // How to convert input to AudioSample ?
  }
}

void draw() {
  float f = loudness.analyze() * 5;
  background(125, 255, 125);
  rect(0, height, 100, -height * f);
}

[dhowe]

Any thoughts?? This seems like it should be quite simple

@kevinstadler

[kevinstadler]

This can be achieved using the Waveform analyzer and then copying data from there to an AudioSample. The main issue is that, unless you add your own code to incrementally gather new sample data, it will be necessary to decide the length of the sample (or at least the maximum length) beforehand.

The basic structure would look like this (not tested):

AudioIn input;
Waveform waveform;
AudioSample sample;

// allocate a buffer for up to a 10 second sample
int maxSampleSize = 44100 * 10;

void setup() {
  size(100, 400);
  noStroke();
  input = new AudioIn(this, 0);
  waveform = new Waveform(this, maxSampleSize);
}

void keyPressed() {
  if (keyCode == 32) {     // start record
    waveform.input(input);
  }
  if (keyCode == 10) {     // stop record
    float[] samples = waveform.analyze();
    int startIndexOfRecordedSample = maxSampleSize - waveform.getLastAnalysisOffset();
    // if we recorded for >= 10 seconds this index will be 0, otherwise it will be later into the float[] array
    sample = new AudioSample(this, Arrays.copyOfRange(samples, startIndexOfRecordedSample, maxSampleSize));
  }
}

[kevinstadler]

Just looking into this again.

In the short term, DIY solutions can be made easier by exposing AudioIn's internal JSyn ChannelIn through a public method (documented only in the Javadoc, for advanced users).

In the long run it would be great to have dedicated user friendly methods to record not just from AudioIn, but potentially any sound source. Here's a rough idea for an interface that I cooked up, would be great to hear @dhowe's opinions on this if you're still working with this?

Assuming the user has already created a fixed length AudioSample (or SoundFile), they could record into that buffer using the following new methods:

• RecordingSession as.recordFrom(SoundObject o) fills the existing AudioSample buffer once from start to finish
• RecordingSession as.recordFrom(SoundObject o, float duration) if the duration is negative, record indefinitely into the fixed size buffer, effectively implementing #54 (not just for AudioIn but for any sound source)
• RecordingSession as.recordFrom(SoundObject o, float start, float duration) same but with a target buffer start offset
• RecordingSession as.recordFromFrame(SoundObject o, int start) same but with offsets specified in frame numbers
• RecordingSession as.recordFromFrame(SoundObject o, int start, int nframes)

RecordingSession is a new interface which provides a controller for the recording session. It has the following methods:

• boolean rs.isFinished() returns false as long as the writing of samples from the source to the AudioSample is ongoing
• void/boolean rs.stopRecording() ends an indefinite, circular recording session (or stops a fixed length recording prematurely)
• possibly other methods for getting information about the recording duration so far/to go, extending/restarting the recording etc.

In the case of one-off fixed length recordings the user can just fire and forget without keeping the RecordingSession object, since the recording will finish automatically after duration. The nice thing about having an external interface/object for the recordings is that several recordings (even from different audio sources) can be written into the same AudioSample at the same time to create crazy mashups (and even while the AudioSample itself is being played back!

To allow for the creation of recordings of unknown/dynamic length (like described above in this issue), one could add a new class AudioRecording extends AudioSample implements RecordingSession which is returned by the following new AudioIn methods:

• AudioRecording ain.record(float seconds) essentially a short-hand for creating a new AudioSample of that length and then calling as.recordFrom(ain) on it
• AudioRecording ain.record() starts an indefinite recording into a new AudioSample, which needs to be stopped explicitly by calling stopRecording() on the resulting AudioRecording. An open implementation question is whether the (expanding) sample buffer of this object will be available before stopRecording() is called, or whether the internal buffer is only created once the effective size of the recording is known.