Closed jaustin closed 1 month ago
These are awesome!
Lots to digest here! My preference is always to keep things as simple as possible, but my audio background is pulling me to want to keep the concept of sample rate, if not the samples.
Do we know if altering the sample rate will produce clearly audible differences that are still useful / intelligible in some way? The ability to change sample rate is only useful if you can hear a difference and learn that you can make a trade-off between recording length and quality.
keep the concept of sample rate, if not the samples
I agree, it's probably going to be necessary to adjust the recording sample rate in certain cases. Building on example 4 from above, this could be done via optional arguments to construct a RecordingBuffer
:
sample_buf = audio.RecordingBuffer(duration=2.0, rate=8000)
Based on the multiple options discussed above I think there are a couple of things I'd like to highlight:
audio.play(...)
that would not work with other sound types, like MicrobitSound
, MicrobitSoundEffect
or AudioFrame
. 1.
is applied, then we would have to either:
audio.play()
can access things like sampling rate from the buffer class instead of additional arguments
ii.
as it continues to unify the usage of audio.play()
First proposal: A new buffer type
With that in mind, I think I would propose to create a new buffer class. We can call it AudioBuffer
for now, the name is open for suggestion, but I don't think it should be tied to playback and recording specifically, as it could also be filled as an array/list and contain additional parameters, like sampling rate.
my_buffer = audio.AudioBuffer(samples=16000, rate=8000)
microphone.record(my_buffer)
audio.play(my_buffer)
To modify the playback sampling rate, we could modify its attribute:
# Play at half the speed
my_buffer.rate = my_buffer.rate // 2
audio.play(my_buffer)
A couple of questions:
microphone.record()
to return a newly created buffer?
my_buffer = audio.AudioBuffer(samples=16000, rate=8000)
microphone.record(my_buffer)
# Vs
my_buffer = microphone.record(samples=16000, rate=8000)
AudioBuffer
and microphone.record()
my_buffer = audio.AudioBuffer(ms=2000, rate=8000)
other_buffer = microphone.record(duration=2000, rate=8000)
duration
? ms
? something else?Second proposal: microphone.record() to be able to both create or use an AudioBuffer
Setting a duration value for a recording makes sense, but it's probably a less common unit type for a buffer (specially if the data can be get or set like an array). One option could be to let the AudioBuffer
to only deal with the number of samples and the rate, while having the option to record for a specific amount of time via microphone.record()
.
# Returns a new buffer for 2 seconds at 8k sampling rate, so 16k samples
my_buffer = microphone.record(duration=2000, rate=8000)
# By having a reasonable default sampling rate this two-liner works well
my_buffer = microphone.record(duration=2000)
audio.play(my_buffer)
# The user can still create their buffer first and record until the buffer is full
other_buffer = audio.AudioBuffer(samples=16000, rate=8000)
same_buffer = microphone.record(other_buffer)
# record() should return the reference to other_buffer, which is unnecessary, but that way it matches the return signature from the previous example
# An advantage of creating your own buffer is that you can also record only a portion of the buffer
other_buffer = audio.AudioBuffer(samples=16000, rate=8000)
microphone.record(other_buffer, duration=500)
# Or if the argument would be called `ms` instead of `duration`
microphone.record(other_buffer, ms=500)
# And changing the sampling rate at the point where we record can still be valid
microphone.record(my_buffer, duration=8000, rate=2000)
# We don't even need to know how much time would fit, we could record until full with a lower sampling rate
microphone.record(my_buffer, rate=my_buffer.rate // 2)
And being able to record only a portion of the buffer brings me to the third proposal.
Third proposal: microphone.record() to be able to record at a buffer offset
This proposal is about having an offset
parameter in microphone.record()
method so that a single buffer could be filled via multiple recordings.
This is not necessarily tied to using the duration
/ms
argument type of microphone.record()
, but for continuity the snippets here will use proposal.
For this to work effectively we need a way to keep track where in the buffer the last recording stopped. Normally this could be the return value from microphone.record()
, but if we'd like this method to be able to create and return a new buffer, then we'd have to either return a tuple, or find a different way.
The AudioBuffer
class could track its index (or audio.play()
could update this value), so that the next recording could continue where the previous left over.
The default value for offset
should be zero, as this is an advance feature and simpler programmes that constantly overwrite a single buffer (for example, press A to record, and B to playback) should work as expected with the default values.
Not 100% sure what the "continue where you left off" value should be, offset=None
would be the simplest option, but it's not intuitive for this use-case, maybe offset="auto"
? Very open for suggestions.
my_buffer = microphone.record(samples=16000, rate=8000)
# We could start a new recording specific we left off or select it
microphone.record(my_buffer, duration=500)
microphone.record(my_buffer, duration=500) # Overwrites the first 500ms again
microphone.record(my_buffer, duration=500, offset="auto") # Continues where the previous left off
microphone.record(my_buffer, duration=1000, offset=8000) # The exact sample could be inputted directly
A few quick points discussed in a call that I will expand next week:
audio.play()
we can then add that as function parameter, and a audio.set_rate()
function microphone.record()
offset
argument, as being able to create multiple AudioBuffers
fulfils the same purposemicrophone.record()
return a newly created buffer is a good idea, but instead of having the same function take a buffer or created, we can also have a microphone.record_into(my_buffer)
functionAudioBuffer
type, then having matching arguments for the buffer and the microphone.record()
keeps things simple and symmetrical. We can also add a duration
parameter to microphone.record()
.We can close this issue as specific discussion about implementation have been carried out in other issues/PRs.
...transferring from the past private Repo from when V2 was still not announced.... @jaustin commented on Mon Sep 21 2020
We'd like a few Python code examples of how the user could record samples from the microphone and then play them back in the style of a parrot (eg listen until the buffer is full, or there's a silence)
@dpgeorge was going to have a go at putting forward a few different sample models and @microbit-giles and @microbit-carlos could we please also contribute here things from our API prototyping?
@finneyj commented on Mon Sep 21 2020
Yep - sounds like a good probe.
@dpgeorge commented on Thu Sep 24 2020
Example 1: simple wait, record, stop, play.
Example 2: recording via the audio object instead of microphone (to support an external mic?), and the record function has the ability to stop on a given event.
Example 3: attempt to keep some samples before the loud event so the first part of the recording is not cut off.
@dpgeorge commented on Thu Sep 24 2020
Example 4: Removing the concept of sample rate and samples.
Example 5: allocating the recording buffer automatically.