search

lancaster-university / codal-microbit-v2

CODAL target for the micro:bit v2.x series of devices
MIT License
44 stars 52 forks source link

Gibberish in the beginning of the recording data #442

Open microbit-carlos opened 2 months ago

microbit-carlos commented 2 months ago

When recording data from the microphone, the beginning of the data is clearly not right.

I've taken measurements with a recording and playback sampling rate of ~44.1 KHz, using the StreamRecording and retrieving the data with the SerialStreamer. During sound recording, I've placed my finger on top of the microphone hole to try to minimise background noise, and once the data stabilises, it stays very close to the zero line.

The top graph plots 11k samples, 250ms at 44.1Khz. The second graph zooms into the first 2560 samples:

image

The ADC DMA buffers are 512 bytes, and with a 14-bit sampling rate, we get 256 samples per buffer. In most captures that I've done it looks like the data is only stable after 2560 samples, which is interestingly exactly 10 DMA buffers.

Source code to replicate
This example code needs this fix in the SerialStreamer to work correctly: - https://github.com/lancaster-university/microbit-v2-samples/pull/92 ```cpp #include "MicroBit.h" #include "StreamRecording.h" #include "SerialStreamer.h" MicroBit uBit; int main() { uBit.init(); static SplitterChannel *splitterChannel = uBit.audio.splitter->createChannel(); static StreamRecording *recording = new StreamRecording(*splitterChannel); static MixerChannel *mixerChannel = uBit.audio.mixer.addChannel(*recording); static SerialStreamer *streamer = new SerialStreamer(*recording, SERIAL_STREAM_MODE_DECIMAL); MicroBitAudio::requestActivation(); mixerChannel->setVolume(75.0); uBit.audio.mixer.setVolume(1023); uBit.serial.printf("Recording rate: %d\n", (int)recording->getSampleRate()); uBit.serial.printf("Mixer rate: %d\n", (int)uBit.audio.mixer.getSampleRate()); while (true) { if (uBit.buttonA.isPressed()) { uBit.sleep(500); recording->recordAsync(); uBit.display.print("R"); uBit.sleep(250); recording->stop(); } else if (uBit.buttonB.isPressed()) { uBit.display.print("P"); recording->connect(*mixerChannel); recording->play(); } else if (uBit.logo.isPressed()) { recording->connect(*streamer); recording->play(); } uBit.display.clear(); uBit.sleep(100); } } ```
Serial output Didn't include all the serial output at the end, stopped around the time the data stabilises. ``` Recording rate: 45454 Mixer rate: 44100 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 26 18 6 -7 -22 -37 -50 -65 -80 -94 -108 -121 121 108 94 82 70 56 44 34 22 9 -2 -13 -24 -33 -44 -55 -65 -77 -85 -94 -103 -111 -122 126 119 108 103 92 86 76 69 63 55 48 42 34 27 22 12 8 0 -5 -13 -17 -22 -28 -33 -37 -45 -50 -53 -59 -65 -70 -73 -77 -82 -88 -90 -97 -100 -104 -107 -111 -115 -119 -122 -127 127 123 121 116 114 109 108 104 102 100 96 95 89 88 85 83 82 79 75 75 71 70 66 66 64 61 60 58 57 53 54 50 49 47 47 44 42 40 39 40 38 33 34 32 32 28 29 28 26 25 26 23 22 22 20 21 18 18 18 18 16 16 13 13 12 13 10 11 9 9 9 8 5 7 8 6 6 5 5 4 3 1 1 0 1 -1 0 0 0 -1 -3 -3 -2 -2 119 118 118 119 119 119 119 119 119 120 118 119 118 120 118 118 119 117 117 118 120 117 123 66 -106 -106 -106 -106 -106 -106 -106 -106 -106 -106 -106 -106 -106 -106 -106 -106 -106 -106 -106 -106 -106 -106 -106 -106 -106 -106 -106 -106 -106 -106 -106 -106 -106 -106 -106 -106 -106 -106 -106 -106 -106 -106 -106 -106 -106 -106 -106 -106 -106 -106 -106 -106 -106 -106 -106 -106 -106 -106 -106 -106 -106 -106 -106 -106 -106 -106 -106 -106 -106 -106 -106 -106 -106 -106 -106 -106 -106 -106 -106 -106 -106 -106 -106 -106 -106 -106 -106 -109 -116 -128 114 99 86 71 56 40 26 13 -1 -16 -30 -43 -56 -70 -81 -93 -106 -118 -128 115 104 92 81 71 60 51 39 30 22 11 2 -7 -15 -26 -35 -41 -49 -58 -65 -74 -82 -90 -98 -104 -110 -117 -125 124 117 111 104 98 95 87 78 75 69 64 58 53 49 45 38 34 28 25 20 16 11 7 3 -2 -4 -8 -14 -16 -20 -22 -28 -31 -33 -35 -41 -43 -46 -49 -51 -54 -56 -60 -63 -65 -67 -69 -72 -75 -77 -78 -81 -82 -85 -87 -90 -91 -95 -94 -97 -99 -102 -102 -103 -105 -108 -109 -109 -111 -112 -114 -116 -115 -117 -118 -119 -121 -123 -124 -124 -126 -127 -127 -128 126 126 124 123 124 120 122 120 121 120 117 -121 -121 -122 -123 -121 -122 -124 -123 -124 -124 -125 -124 -124 -125 -126 -126 -128 -128 -127 -128 -127 127 124 125 126 126 126 125 123 123 124 125 124 123 124 123 123 123 123 122 121 122 121 122 122 121 121 121 119 122 120 120 119 119 120 119 120 117 120 119 119 119 119 120 120 119 121 121 119 119 120 120 120 119 120 119 120 120 119 121 119 120 121 120 119 120 119 119 119 120 120 119 119 120 120 120 119 119 119 118 120 121 122 119 120 120 122 119 121 122 120 120 120 121 120 120 121 122 122 122 122 119 122 122 122 121 122 122 121 121 122 122 123 123 122 121 124 124 122 124 123 124 124 123 121 121 123 122 126 123 123 124 124 123 123 123 124 124 124 124 124 125 124 125 124 124 126 124 125 125 125 126 126 124 123 126 124 124 123 123 124 125 125 124 126 125 125 127 127 125 126 124 126 125 126 126 127 126 -128 127 126 127 -128 127 127 127 127 127 -128 125 125 127 125 125 127 127 -127 -128 127 127 127 126 -128 127 -127 127 126 -127 127 -127 127 127 126 -126 -127 127 127 -127 127 -128 -127 127 -127 -128 -128 -128 -127 -127 -128 -128 -127 127 -128 -127 -128 -128 67 67 68 67 67 67 68 67 69 68 66 67 68 69 68 69 67 70 68 69 69 70 68 68 70 67 69 66 69 68 68 69 67 67 70 68 69 69 71 69 70 72 71 70 70 69 69 70 70 71 69 68 70 70 70 70 70 69 70 70 70 70 69 69 71 71 70 71 71 71 70 71 72 67 71 70 71 71 72 70 69 71 69 71 70 70 71 70 72 73 71 71 71 71 73 71 70 71 71 70 72 72 71 70 72 70 72 72 71 73 73 72 72 72 71 73 73 72 71 69 71 71 72 73 72 71 72 72 73 72 71 72 71 72 71 73 71 72 72 74 73 72 72 72 72 72 71 73 72 73 73 72 73 73 71 73 73 72 74 72 72 73 71 73 71 72 72 72 74 73 74 74 73 72 74 72 74 73 74 73 73 74 73 71 73 75 74 74 74 72 73 74 74 74 73 73 74 74 74 72 74 73 74 73 73 72 72 74 72 72 74 74 73 74 73 75 76 74 75 74 75 74 74 74 74 75 74 75 74 74 74 74 75 73 74 73 73 75 74 75 74 75 75 75 74 74 76 74 76 75 74 75 75 75 77 77 -88 -89 -90 -90 -89 -91 -89 -90 -89 -90 -89 -89 -89 -90 -89 -91 -88 -89 -89 -90 -89 -90 -89 -88 -90 -90 -90 -90 -89 -88 -91 -91 -90 -88 -89 -89 -89 -90 -88 -89 -88 -89 -91 -89 -92 -89 -87 -89 -88 -88 -88 -89 -88 -90 -90 -88 -89 -88 -88 -89 -89 -88 -87 -88 -88 -89 -88 -88 -89 -88 -89 -88 -88 -88 -89 -88 -88 -88 -89 -87 -88 -89 -87 -88 -88 -88 -89 -91 -88 -90 -88 -88 -90 -87 -88 -87 -88 -89 -88 -88 -87 -88 -88 -87 -89 -88 -88 -89 -87 -88 -88 -88 -88 -88 -88 -88 -89 -87 -89 -87 -89 -89 -88 -85 -87 -87 -89 -87 -87 -87 -88 -87 -88 -87 -88 -87 -88 -88 -89 -87 -86 -87 -87 -87 -88 -87 -85 -87 -87 -88 -87 -87 -86 -87 -88 -87 -87 -88 -87 -88 -86 -86 -87 -88 -87 -87 -88 -86 -87 -87 -87 -87 -88 -88 -87 -86 -86 -88 -87 -87 -86 -88 -88 -86 -86 -86 -86 -87 -87 -87 -87 -86 -87 -86 -87 -88 -87 -86 -86 -85 -89 -88 -87 -87 -85 -88 -87 -87 -86 -88 -86 -87 -86 -87 -88 -87 -87 -87 -87 -87 -86 -87 -86 -87 -87 -87 -89 -88 -87 -88 -88 -86 -88 -88 -88 -87 -86 -86 -86 -86 -87 -86 -86 -87 -88 -89 -87 -87 -87 -86 -87 -88 -86 -85 -86 -88 -43 -43 -42 -44 -42 -42 -44 -44 -42 -43 -43 -43 -42 -42 -41 -42 -42 -43 -43 -42 -42 -41 -43 -43 -41 -42 -41 -43 -43 -43 -42 -42 -42 -43 -43 -42 -42 -42 -42 -41 -42 -43 -41 -42 -45 -42 -42 -42 -42 -42 -44 -41 -42 -44 -42 -42 -44 -43 -41 -43 -41 -43 -43 -43 -43 -42 -42 -42 -43 -42 -41 -42 -42 -43 -42 -43 -42 -41 -42 -43 -42 -41 -42 -40 -42 -41 -42 -42 -42 -41 -43 -42 -41 -42 -42 -43 -42 -42 -43 -40 -41 -41 -42 -41 -42 -43 -42 -44 -42 -42 -42 -40 -41 -43 -42 -42 -41 -41 -43 -41 -41 -41 -41 -40 -43 -41 -42 -41 -41 -41 -42 -41 -40 -41 -40 -42 -42 -42 -44 -42 -41 -41 -41 -42 -42 -41 -42 -42 -41 -43 -41 -40 -42 -44 -40 -41 -41 -42 -41 -41 -40 -40 -41 -42 -42 -41 -42 -42 -42 -41 -43 -41 -42 -41 -42 -41 -42 -41 -41 -39 -41 -40 -41 -40 -40 -42 -41 -41 -40 -41 -40 -42 -43 -41 -41 -41 -42 -42 -42 -41 -42 -40 -41 -43 -41 -40 -41 -41 -42 -40 -40 -42 -41 -39 -40 -42 -41 -40 -40 -42 -40 -40 -39 -41 -40 -42 -39 -41 -40 -42 -42 -41 -41 -40 -39 -42 -42 -42 -42 -41 -41 -41 -41 -39 -40 -39 -41 -41 -40 -40 -41 -41 -41 -40 -39 -42 -20 -20 -21 -21 -20 -17 -20 -21 -20 -19 -18 -18 -19 -19 -18 -19 -19 -21 -22 -21 -20 -20 -20 -20 -19 -21 -20 -19 -21 -19 -20 -19 -18 -19 -18 -19 -20 -20 -18 -20 -20 -20 -20 -20 -20 -20 -19 -20 -20 -20 -19 -20 -19 -20 -18 -19 -19 -20 -18 -19 -19 -19 -20 -20 -21 -19 -21 -17 -19 -20 -21 -18 -19 -19 -21 -21 -21 -18 -18 -19 -22 -19 -19 -19 -20 -18 -20 -19 -16 -20 -18 -19 -19 -18 -20 -18 -19 -19 -19 -20 -20 -19 -18 -19 -18 -18 -18 -17 -18 -19 -18 -18 -18 -18 -19 -19 -17 -20 -19 -18 -19 -17 -19 -19 -19 -20 -22 -19 -20 -20 -20 -19 -19 -19 -19 -18 -17 -20 -19 -18 -17 -18 -18 -19 -18 -19 -18 -19 -19 -19 -18 -19 -19 -20 -19 -19 -20 -19 -17 -18 -21 -18 -20 -19 -19 -19 -18 -19 -19 -19 -21 -19 -17 -18 -18 -19 -19 -18 -18 -18 -18 -19 -19 -17 -19 -18 -17 -21 -18 -19 -19 -19 -18 -21 -18 -19 -18 -18 -19 -19 -19 -18 -19 -18 -19 -19 -19 -18 -19 -18 -18 -20 -19 -19 -18 -19 -18 -18 -19 -19 -18 -18 -18 -17 -17 -18 -19 -20 -19 -18 -18 -19 -18 -19 -19 -17 -18 -17 -20 -18 -18 -18 -19 -19 -18 -17 -18 -20 -19 -22 -19 -19 -18 -18 -19 -20 -8 -8 -10 -10 -10 -11 -9 -10 -10 -10 -10 -8 -10 -9 -10 -9 -8 -9 -10 -10 -11 -10 -10 -9 -9 -9 -9 -10 -9 -9 -9 -9 -9 -10 -9 -10 -9 -9 -10 -10 -10 -9 -9 -11 -10 -10 -9 -8 -8 -8 -8 -8 -9 -9 -10 -10 -9 -9 -7 -10 -9 -9 -8 -8 -7 -9 -10 -8 -9 -10 -9 -9 -9 -7 -8 -9 -9 -8 -8 -9 -9 -9 -10 -9 -7 -8 -10 -10 -9 -8 -9 -8 -10 -10 -9 -10 -10 -9 -8 -10 -8 -9 -10 -8 -9 -8 -9 -8 -9 -10 -10 -8 -9 -9 -9 -10 -8 -9 -9 -9 -10 -8 -9 -8 -10 -9 -8 -8 -8 -9 -9 -10 -8 -8 -8 -7 -8 -8 -9 -8 -9 -8 -9 -8 -9 -9 -7 -8 -8 -9 -10 -9 -8 -10 -10 -8 -9 -9 -10 -9 -8 -7 -7 -8 -9 -9 -9 -7 -8 -9 -9 -10 -9 -10 -7 -8 -9 -9 -9 -8 -8 -9 -7 -9 -8 -8 -10 -8 -8 -8 -9 -8 -10 -7 -10 -7 -8 -10 -9 -9 -8 -8 -8 -10 -8 -9 -7 -9 -10 -9 -7 -8 -8 -8 -9 -10 -9 -10 -9 -10 -10 -8 -7 -8 -10 -10 -9 -8 -8 -8 -9 -9 -9 -9 -10 -8 -10 -8 -8 -9 -8 -9 -8 -7 -7 -8 -8 -9 -9 -10 -7 -8 -8 -8 -8 -8 -4 -4 -4 -4 -4 -4 -2 -4 -3 -4 -4 -4 -5 -5 -4 -3 -4 -4 -4 -5 -5 -4 -3 -2 -4 -4 -3 -4 -6 -4 -5 -5 -5 -6 -4 -5 -4 -3 -4 -2 -4 -3 -3 -5 -4 -4 -4 -4 -3 -2 -4 -3 -5 -3 -4 -3 -3 -4 -4 -2 -4 -3 -5 -3 -3 -2 -3 -1 -4 -4 -3 -3 -3 -3 -3 -6 -4 -4 -4 -4 -4 -4 -3 -4 -4 -4 -3 -3 -3 -3 -4 -4 -2 -3 -3 -3 -3 -5 -2 -4 -3 -2 -5 -4 -2 -4 -4 -4 -4 -3 -4 -3 -4 -3 -4 -3 -5 -2 -4 -4 -3 -3 -1 -4 -3 -4 -3 -1 -4 -3 -3 -3 -2 -3 -3 -3 -2 -4 -4 -3 -3 -1 -5 -2 -3 -5 -2 -1 -4 -2 -4 -3 -4 -2 -3 -4 -2 -4 -3 -3 -3 -4 -3 -4 -4 -4 -3 -3 -5 -3 -3 -3 -2 -4 -3 -3 -3 -4 -3 -4 -4 -3 -3 -4 -5 -4 -3 -4 -4 -5 -4 -2 -4 -2 -5 -4 -3 -3 -4 -4 -2 -4 -3 -3 -3 -2 -2 -2 -5 -3 -3 -3 -5 -3 -4 -3 -3 -3 -2 -2 -2 -3 -4 -3 -3 -2 -3 -2 -3 -2 -2 -2 -2 -3 -4 -3 -2 -2 -3 -3 -2 -2 -3 -3 -5 -4 -4 -3 -3 -3 -3 -4 -1 -2 -1 -4 -2 -2 -2 -1 -1 -1 -2 -2 -2 0 -2 -4 -3 -1 -1 -1 -1 -2 -3 -1 -2 -1 -3 -1 -2 -2 -1 -4 -1 -2 -1 -2 -1 -1 -4 -2 -1 -2 -3 -3 -2 -2 -2 -1 -1 -2 -2 -1 -2 -1 0 -1 -2 0 -2 -1 -1 -2 -1 0 -1 -3 -1 0 -1 0 -1 -2 -1 -3 -2 0 -1 -2 -1 -1 -1 -1 0 -2 -1 0 -2 -1 -3 -1 -1 -1 -1 -1 -3 0 0 -2 -1 -1 -1 0 -1 0 -1 -1 -1 -1 0 0 -3 -2 -1 -1 -1 0 0 0 -1 -2 0 -1 -2 -2 -1 0 -1 -1 -2 0 0 -1 -1 -1 -2 -3 0 -1 0 -2 -2 0 0 -1 0 -1 0 -1 0 0 -2 -1 -2 0 -1 -2 -1 -2 -1 -1 -2 0 -2 0 0 0 -1 -1 0 0 0 0 -1 -1 0 0 -1 -1 -1 -2 -1 -1 0 0 -2 -1 -1 -2 -1 -1 -2 -1 0 -1 -1 -1 -1 -1 0 -1 -1 -2 0 0 0 -1 -1 -1 -2 -1 0 0 -2 -1 0 0 0 0 0 -1 -1 -2 0 -2 -3 -1 0 -1 -1 -1 0 -1 -1 0 -2 -2 -1 0 0 0 0 0 0 0 0 -2 0 -1 -1 -2 -1 -2 -2 -1 -1 -2 -3 -1 -2 0 -2 0 -1 -2 -2 -1 0 -1 0 -1 -1 0 0 -1 -1 -1 -1 0 0 -2 0 -2 -1 0 0 0 0 0 0 0 -1 0 0 0 0 -1 0 0 -2 -2 -1 -1 -1 -1 -1 -3 -2 -1 -1 -1 -1 0 -1 0 -1 -1 -1 0 0 0 0 0 1 -1 -1 0 0 -1 0 -1 0 -2 -1 -1 0 -1 0 0 -1 -1 -2 -2 0 0 0 -1 -1 0 0 -1 0 0 0 -1 0 0 0 -1 -1 0 0 0 0 0 0 0 0 0 -1 -3 0 -2 -1 -1 0 0 0 0 0 -1 0 0 1 0 -1 -1 0 0 0 -1 -1 0 0 -1 0 -1 0 0 0 -1 -1 -1 0 0 0 1 -1 -1 -1 0 0 -2 -1 0 0 0 -1 -1 -1 0 0 0 0 0 0 1 0 0 0 0 -1 -1 0 0 -1 -1 0 0 -1 0 -2 -2 0 -1 1 0 0 0 1 0 0 -1 -1 0 -1 0 0 0 0 0 0 0 0 -1 -1 0 0 0 -1 0 0 0 -1 0 -1 -2 0 0 0 0 0 0 0 0 0 0 -1 -1 -2 0 -1 -1 0 0 0 0 0 -1 0 0 0 0 0 0 0 0 -1 -1 -1 0 0 0 0 0 -1 0 0 -1 0 -2 0 0 1 0 0 0 0 -1 -1 -2 -1 0 0 -1 0 0 0 0 0 0 0 -1 0 0 0 0 0 0 0 0 0 0 0 0 -1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 -1 0 1 -1 0 0 1 0 -1 0 1 0 0 1 0 0 0 0 0 0 0 0 1 0 1 -1 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 -1 0 0 1 0 0 0 -1 0 1 1 0 -1 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 1 1 0 0 1 0 0 0 -2 0 0 1 0 0 0 ```

MICROBIT.hex.zip

martinwork commented 2 months ago

I think the gibberish is produced in NRF52ADCChannel::demux. The first buffer in each recording is an old one stored by mic→output. The next few are recorded before, while or shortly after the mic is activated. What settle time does the mic need? The long "tail" of steps is perhaps the StreamNormalizer adjusting the calculated zero offset.

Two possible workarounds without changing CODAL:

I think this is how MicroBitAudio mic input is operating...

As the program starts, the mic is

The mic ADC channel continues to generate buffers and drive the pipeline, which is blocked in mic→output, and only results in MicroBitAudio calling deactivateMic() twice for each buffer (see SPLITTER_TICK below).

No buffers pass through the StreamNormalizer until something pulls on the splitter.

When StreamRecording is started, it simply starts accepting buffers. Though the pull request is prompted by the ADC producing a new buffer, if the pipeline was not flowing, the first buffer pulled is whatever mic→output was holding when it stopped flowing. Pulling that unblocks mic→output, and causes the splitter and rawsplitter to message MicroBitAudio to activate the mic.

Once StreamRecording has ended, deactivateMic() is called repeatedly again because mic→output blocks when no buffers are pulled.

uBit.audio.levelSPL can activate the mic and provide a pull on rawsplitter to keep mic→output unblocked and the mic active.

The first block of the first recording after RESET is always slightly off, because it’s the first one through the StreamNormalizer, establishing the zero offset.

Even after uBit.audio.levelSPL→activateForEvents(true), at the end of a StreamRecording, MicroBitAudio sees events 1032, 4 and 10, all of which lead to deactivateMic(). So the mic light briefly flickers off before subsequent events activate it again, which seems to generate some gibberish.

Details...

I put a serial number in the first byte of each NRF52ADCChannel::demux buffer, and copied it in StreamNormalizer::pull(). That revealed that the first buffer in a recording was old. DMESGing the first few bytes of each buffer in those places seemed to confirm they haven’t been corrupted.

I don’t understand why the non-blocking mic→output DataStream holds the first buffer it receives until it is pulled, and drops newer buffers. I expected the opposite. https://github.com/lancaster-university/codal-core/blob/509086cc8590465041b15493ab52b56e7071c110/source/streams/DataStream.cpp#L188

While the ADC is driving the pipeline, rawsplitter and splitter continually send SPLITTER_TICK messages that cause MicroBitAudio to call activateMic or deactivateMic twice for each ADC buffer. https://github.com/lancaster-university/codal-core/blob/509086cc8590465041b15493ab52b56e7071c110/source/streams/StreamSplitter.cpp#L255 SPLITTER_TICK was added quite recently. https://github.com/lancaster-university/codal-core/commit/c064d96883a04605ba8d0a3cd12428ccf021d5ad