WAV over Bluetooth Serial - Sampling Limitation?

[joe-althaus]

Hello, many thanks for this amazing work.

I have been having good luck with the existing methods of the library.

I am attempting to encode sampled audio into WAV format and simply transmit the bytes via bluetooth serial port profile leveraging BluetoothSerial.h. This works well for sampling rates above 9600Hz, but when I try to sample at something below 9600Hz, say 8000Hz, I cannot recreate the wav file correctly because the true sampling rate is 9600Hz. The audio is fine, save for the frequency mismatch rendering the file at a lower pitch when tried to playback at 8000Hz.

Is this because the Bluetooth Serial "Baud" is driving the stream to stream copy?

I have been attempting to drive the data transfer into the SerialBT buffer with .available, .readBytes, and .write methods, but to no avail as yet.

Any insights?

Thanks!

#include "AudioTools.h"
#include "AudioLibs/AudioKit.h"
#include "BluetoothSerial.h"

#if !defined(CONFIG_BT_ENABLED) || !defined(CONFIG_BLUEDROID_ENABLED)
#error Bluetooth is not enabled! Please run `make menuconfig` to and enable it
#endif

BluetoothSerial SerialBT;

AudioKitStream kit;           // Access I2S as stream

EncodedAudioStream out(&SerialBT, new WAVEncoder());
StreamCopy copier(out, kit);  // copies sound into i2s

void setup() {
  Serial.begin(115200);

  auto cfg = kit.defaultConfig(RXTX_MODE);
  cfg.input_device = AUDIO_HAL_ADC_INPUT_LINE1;
  cfg.bits_per_sample = 16;
  cfg.sd_active = false;
  cfg.sample_rate = 8000;
  cfg.channels = 1;
  kit.begin(cfg);
  Serial.println("Kit Started");

  out.begin();

  SerialBT.begin("ESP32Sender"); //Bluetooth device name
  while(!SerialBT.connected()){
     Serial.print(".");
     delay(1000);
  }
}

void loop() { 
   // copier.copy();
    if(kit.available()){
      copier.copy();
    }
}

[pschatzmann]

I actually started to work on the communication topic as well. The issue that you have, is that you need to make sure that the amount of data that is produced can be transmitted w/o causing any I2S buffer overflows. Since the bluetooth speed is limited this can only be controlled in the following ways:

• limit the sampling rate
• reducing the channels (you are already at 1 - so no improvement there)
• use a CODEC that compresses data

The Codecs are currently work in progress, but there are already quite a few options (e.g SBC, LC3) that you can try...

[joe-althaus]

Thanks for the reply - I do see buffer overflow issues at higher sampling rates and multiple channels, but I'm actually trying to operate at 8000Hz sampling. The recreation of the WAV file on the other end of the bluetooth link only works when I run it at 9600Hz - there are no overflows just pitch shifted audio. So my sampling rate is bottom limited? Unfortunately, I'm trying to feed a device that is expecting a WAV file with the 8000Hz, 16bit, 2 channels (playing with one for now), so I can't really explore other encoding options.

I run this same code configured for 16000 Hz and I get a perfect reconstruction on the other side of the BT link.

[pschatzmann]

I think you did not tell me how (on what device) you run it. WAV itself does not have any limitations and if you play it e.g. on a desktop it should work. There is nothing that would stop you to decode from another format (e.g. SBC) to PCM and then PCM to WAV. It's just one more step.

If the receiving side is an AudioKit as well: here are the supported rates

[joe-althaus]

Thank you. I am using the LyraT 4.3 board with onboard mics at the moment to encode and send audio data to an Android app that has been setup to expect the audio data as described. At the moment, for development purposes, I have written a receiver script that ingests the Bluetooth serial data and constructs the WAV file. I will look into some sort of intermediate encoder as you have suggested.

[pschatzmann]

OK, in this case MP3 might be an option as well and I expect that Android should be able to deal with this easier then SBC or LC3

[joe-althaus]

Quick update - Working all aspects of this, trying to get down to the 8000Hz sampling rate objective and I tried to remove the i2s Stream (Audio Kit) to see if that was causing an issue - replaced with a sine wave generator. This works as expected and I am able to successfully recreate audio (by generating a wav file from a byte stream received over the bluetooth serial connection) on the app side. So there is something sneaky happening when an i2s stream is copied versus a generated audio stream.

#include "AudioTools.h"
#include "AudioLibs/AudioKit.h"
#include "BluetoothSerial.h"

uint16_t sample_rate = 8000;
uint8_t channels = 1;  
SineWaveGenerator<int16_t> sineWave( 32000);  // subclass of SoundGenerator with max amplitude of 32000
GeneratedSoundStream<int16_t> sound( sineWave);                      // Stream generated from sine wave

BluetoothSerial SerialBT;

//AudioKitStream kit; // Access I2S as stream

EncodedAudioStream out(&SerialBT, new WAVEncoder());
StreamCopy copier(out, sound);  // copies sound into i2s
//StreamCopy copier(out, kit);  // copies sound into i2s

void setup() {
  Serial.begin(115200);
  //AudioLogger::instance().begin(Serial, AudioLogger::Info);

//    auto cfg = kit.defaultConfig(RXTX_MODE);
//    cfg.input_device = AUDIO_HAL_ADC_INPUT_LINE1;
//    cfg.bits_per_sample = 16;
//    cfg.sd_active = false;
//    cfg.sample_rate = 8000;
//    cfg.channels = 1;
//    kit.begin(cfg);
//    Serial.println("Kit Started");

    out.begin();
    sineWave.begin(channels, sample_rate, N_D4);

    SerialBT.begin("ESP32Sender"); 
    while(!SerialBT.connected()){
         Serial.print(".");
         delay(1000);
  }
}

void loop() { 
    copier.copy();
}

I receive the same erroneous effect when I apply a byte-wise copy such as, instead of copier.copy():

if (kit.available()>0){
            int len = kit.readBytes(buffer, 128);
            encoder.write(buffer, len);
        } 

[pschatzmann]

Can you share one of the wav files that is not working ? You write a file first and listen to it later - right?

• You can add the copy size as third argument to the copier (to define the copy buffer size). So with 128 you should be able to get the same negative effect. Can you double check this to verify that the issue is not related to any tasks
• You could also add a buffer_size and buffer_count parameter to the I2S config to double check if this makes a difference.
• If you just read from I2S you should open it in RX_MODE (and not RXTX_MODE)

The main difference between I2S and the Generator is that I2S produces samples at a constant rate, and the generator is able to produce samples at any rate (independent of the sample rate)...

[joe-althaus]

Yes, I am writing the captured bytes to an array from which I build the wav file for listening in Audacity. Here are two different files - I recorded a stereo byte stream for 2.5 seconds using the same source and then built the wav file using 8000Hz and 9600Hz as the sampling rate in the header. The source audio is simply a sine wave from my speakers and then picked up using the onboard mics of the LyraT. Both files were generated from bytes captured from the code used below.

It is possible to hear that the file with the 9600Hz sampling header is the correct end result frequency. To be clear - I am running the code below (configured for 8000 Hz sample rate) and capturing the bluetooth byte stream on my end device. When I receive the bytes and tell my wave file that I received the same configuration (2ch, 16-bit, 8000Hz) that was intended I get audio data that is pitch shifted down when I load that same wav file into Audacity. If I capture that same data but then put 9600 in the sampling rate field, I get the correct pitch/frequency in Audacity.

I have been trying to come up with MASTER/SLAVE configurations to see if I can get the i2s stream to be the MASTER in the copier.copy() transaction thereby slowing down sample rate to my desired 8000Hz.

I did try playing with the various buffer commands and configurations, but it didn't seem to have an effect.

#include "AudioTools.h"
#include "AudioLibs/AudioKit.h"

#include "BluetoothSerial.h"

#if !defined(CONFIG_BT_ENABLED) || !defined(CONFIG_BLUEDROID_ENABLED) || !defined(CONFIG_BT_SPP_ENABLED)
#error Bluetooth configuration issue! Please run `make menuconfig` to and enable it
#endif

BluetoothSerial SerialBT;
AudioKitStream kit; // Access I2S as stream
StreamCopy copier(SerialBT, kit);  // copies sound into i2s

void setup() {
    Serial.begin(115200);

    auto cfg = kit.defaultConfig(RX_MODE);
    cfg.input_device = AUDIO_HAL_ADC_INPUT_LINE1;
    cfg.bits_per_sample = 16;
    cfg.sd_active = false;
    cfg.sample_rate = 8000;
    cfg.channels = 2;
    //cfg.master_slave_mode = AUDIO_HAL_MODE_MASTER;
    kit.begin(cfg);
    Serial.println("Kit Started");

    SerialBT.begin("ESP32Sender"); //Bluetooth device name
    while(!SerialBT.connected()){
         Serial.print(".");
         delay(1000);
  }
}

void loop() { 
    copier.copy();
}```
[AudioFiles.zip](https://github.com/pschatzmann/arduino-audio-tools/files/8655618/AudioFiles.zip)

[pschatzmann]

Oh, finally i understand your issue, which is strange indeed: If you use AUDIO_HAL_MODE_MASTER, the ES8311 in Master mode and the ESP32 the slave, then depending the documentation the sampling frequency of 8000 should work (see here page 8). I am not sure however if this has been ever tested because I concentrated on having the ESP32 as the master (which is the default setting)

If I understand you correctly the issue is that the recording sampling rate is not the indicated 8000 but 9600Hz.

It would be interesting to see if this issue is only on the input side or also on the output side of the AudioKit. The output of a generated sine tone might also be pitch shifted at a rate of 8000. And one more question: which Arduino ESP32 version are you using and have you tried with a more recent one ?

Testing the output with 8000 on ESP32 version 2.0.3, I did not notice this issue on the output side...

[pschatzmann]

Closed due to inactivity