something not quite right...

[themindfactory]

I downloaded the code and added parts of it to an Arduino build (ESP32) I have done with a digital I2S microphone. The microphone interface is stable and giving the correct values.

What I notice is:

1. the brightness of the LEDS are only going to around 70 out of 255 when it calls the HSV(h,s,v) routines, so v is never larger than around 70

2. I configured it to do 7 octaves from 27.5Hz so that should get me to about 3300hz, however when using a tone generator it only gets to around 500hz and then no LEDS display no matter the volume.

I am running at 16000Hz sample rate, I make sure it feeds the "ADC" -4095 to +4095 I do get proper colors out at the correct notes for the first 3-4 octaves, So assume that I have correctly set up sample frequency.

When I take a look at the embeddedbins32[] array I expect if I play a 3200hz (upper end of the 7 octaves its set for) then the last few entries in the array should be larger than the rest.. they are not, I get to around 700Hz when this happens. FIXEDBINS is at 168 which makes sense also as this is 7 octaves 12 notes/octave 2 ... so I am a little stumped.

[cnlohr]

(1) You may want to look at changing the settings on NOTE_FINAL_AMP as this will likely have the impact you want.

(2a) I believe the embedded system wants smaller values for the ADC in... I recommend looking at https://github.com/cnlohr/colorchord/blob/master/embedded8266/user/user_main.c#L100 what does your entry look like?

(2/3) Hmm... I don't know if I ever tested any other sizes for the # of fixbins, there may be some other constant hidden somewhere awful.

(4) Would you mind posting the output of your embeddedbins32?

[themindfactory]

(1) don't really get where the 12 comes from... #define NOTE_FINAL_AMP 12 //Number from 0...255

(2a) to be clear I am using an ESP32 but I have used snippets of your code in my Arduino project... my feeding of the samples is more like https://github.com/cnlohr/colorchord/blob/8e628ab602367a3da92049a23ac4ff671be5247d/embeddedstm32f303/main.c#L83

I looked at the embeddedstm32f303 it looked simpler to follow without all the tasks :-)

The example you sent shows the sounddata[] as unsigned 8 bits, in my example I used it was signed 16 bits and the function prototype used was void PushSample32( int16_t dat );

(2/3) was thinking it might be a rollover issue somewhere when a uint8 was used and now needs to be uint16 due to the extra octaves.... that said, I watched a youtube video of your friend playing on the keyboard and my leds would not react at all to the video.... (ColorChord 2: Fugue In D Minor) weird, played lots of music tracks and got some action, but zero with that keyboard

(4) I will post this when I have access to the hardware again

[themindfactory]

also, why does this call HandleInt twice with same data???

void PushSample32(int16_t dat) { HandleInt(dat); HandleInt(dat); }

[themindfactory]

(2a) what filter is that just before the PushSample32() my system has a high pass at 50Hz, removed it at one point to see if I could see a difference and there was none....

[cnlohr]

re: calling twice, IIRC, this was because the way the algorithm works it updates the top? octave every frame, and all the other octaves 1/2^their octave down frames. I believe it just offers slightly better stability.

I believe 2A is the filter which removes the DC offset. I don't know if it's actually needed with the current algorithm, but I think it is.

[themindfactory]

I have no DC offset on my data stream from microphone, I had already been doing lots of FFT and DFT stuff with the stream so I am very aware of how its working, and there is no issue with that part I know.

Will keep at it here, I had done plenty of troubleshooting before I posted... to better understand and discuss the problem more intelligently :-)

[cnlohr]

If you don't have a significant DC offset, it shouldn't be an issue, so you can feel free to remove it!

[themindfactory]

I played notes using a tone generator from A0 to A7, while the software was running it would output to the screen the contents of embeddedbins32[] which is the DFT bins for each frequency that corresponds to a note. This was done in the UpdateOutputBins32() function. The tones played were pure tone, so I expect to see peaks at certain locations in the array that correspond to the note played.

I show also the location the peak was found for each note.

I expected: A0 0 A1 24 (24 apart, 12 notes x 2 locations per entry) A2 48 A3 72 A4 96 A5 120 A6 144 A7 168

Again setup for 7 octaves starting at A0 (27.5hz), after the note is the position of the max value, I am playing pure tones.

HOWEVER the correct colors are shown... just stop being displayed around C5

A0 75th 24,28,24,26,26,26,26,28,26,26,26,26,26,26,26,24,26,28,26,28,26,26,26,28,28,28,27,28,27,27,28,28,26,27,28,27,27,27,28,28,30,28,28,29,31,31,32,32,31,32,32,32,33,33,33,33,32,34,36,38,42,45,48,52,54,56,57,58,59,65,76,90,108,128,144,153,148,134,121,111,99,86,71,58,49,46,44,41,36,33,31,29,29,28,29,29,28,30,37,44,42,33,29,20,14,12,13,12,12,12,15,17,18,25,29,22,15,8,7,6,4,7,10,15,26,21,10,6,4,3,2,8,17,15,9,6,3,3,10,11,7,7,7,9,10,4,5,6,11,6,3,4,11,6,0,2,10,7,4,4,3,2,3,6,2,7,3,3,

A1 67th 70,72,72,72,72,72,70,72,72,72,72,74,72,72,74,76,74,76,76,76,78,78,78,78,68,69,69,70,68,68,70,69,69,70,68,68,69,68,70,70,71,73,74,75,76,78,81,82,77,79,82,86,89,94,100,106,112,118,126,135,148,165,183,207,231,251,267,272,266,247,219,184,158,140,131,117,99,93,92,84,76,74,73,68,56,45,36,31,29,29,30,32,37,41,50,65,75,65,49,35,28,21,20,19,27,28,23,17,14,9,6,16,28,24,18,11,6,13,21,17,14,9,7,20,17,14,7,17,18,16,12,3,8,5,6,11,10,2,9,10,8,12,11,2,5,0,7,8,5,8,4,8,2,5,2,6,4,7,3,6,2,3,6,3,5,3,2,1,

A2 91th 24,24,22,24,24,24,24,22,24,24,22,24,24,22,22,22,24,22,26,26,22,22,24,26,28,27,29,29,30,28,28,30,27,29,29,28,28,28,28,27,28,26,26,25,24,22,23,20,17,14,12,9,6,4,1,4,8,11,16,19,23,27,30,32,34,36,37,37,37,35,35,33,32,34,41,50,53,49,43,45,54,64,72,76,77,81,98,127,166,204,235,247,238,206,156,103,83,95,89,43,70,70,42,55,62,53,63,90,98,79,47,25,15,11,4,11,7,14,15,33,42,30,17,12,10,9,3,5,20,26,14,9,6,6,2,8,13,9,5,3,1,15,14,8,8,3,7,10,6,5,1,12,6,6,2,4,4,3,9,7,5,4,5,4,2,4,4,1,

A3 124th 10,12,12,12,12,12,10,8,12,10,12,10,12,10,10,10,10,10,8,10,8,10,10,8,10,11,11,11,10,11,11,11,10,12,10,11,11,12,10,12,12,12,12,12,11,12,11,13,10,10,9,10,10,10,10,11,11,10,11,12,12,11,12,13,14,15,16,17,20,22,27,32,33,37,39,39,37,37,37,36,34,33,32,30,27,26,25,25,24,23,24,24,25,26,27,28,32,38,45,43,33,28,25,24,27,27,28,33,36,42,53,68,97,151,144,103,68,55,46,38,55,58,89,165,208,119,69,47,38,30,22,24,42,30,26,25,25,28,28,9,8,10,15,29,14,6,5,3,13,3,6,5,3,6,6,5,3,4,6,16,5,4,4,2,3,4,4,5,

A4 145th 4,2,4,2,6,4,4,4,6,6,6,4,2,6,4,4,6,4,8,6,6,6,6,6,7,6,6,6,5,7,6,6,5,5,5,6,4,5,4,6,5,4,3,5,4,5,4,4,7,7,8,7,7,7,7,6,6,6,5,5,5,4,3,3,1,1,1,1,2,3,3,4,6,6,6,7,7,7,7,7,8,8,8,7,6,4,3,3,5,7,9,10,11,12,12,11,9,7,7,8,8,9,10,12,16,17,14,10,5,0,3,6,7,5,4,4,3,11,21,23,17,12,8,11,11,10,8,10,22,22,20,16,20,25,32,10,14,21,25,40,101,71,43,54,99,216,143,64,37,26,31,21,22,23,11,5,10,22,7,2,4,8,6,6,4,6,9,9,

A5 167th 0,2,2,2,2,0,2,2,2,2,2,2,2,2,2,0,0,2,2,0,2,2,2,2,3,3,4,2,3,4,3,3,2,4,3,3,4,4,4,4,4,4,3,5,4,4,3,4,1,1,0,0,0,1,0,1,2,2,3,3,5,6,7,7,7,7,6,7,7,7,7,7,1,1,1,1,0,0,0,0,0,0,0,1,1,1,2,3,3,3,3,3,4,5,4,4,2,2,2,2,2,2,2,2,3,3,5,4,2,2,5,7,6,6,5,5,5,5,6,2,3,10,11,8,7,7,7,6,3,9,10,6,5,4,3,3,7,1,0,1,2,4,10,7,6,6,4,18,10,9,7,22,9,7,7,13,1,3,13,15,12,19,30,27,60,36,52,175,

A6 ?? 2,2,2,4,2,2,4,2,2,2,2,4,4,6,4,2,4,2,4,4,4,2,2,2,4,2,2,3,3,2,3,3,2,2,2,1,1,2,2,1,0,1,1,1,0,1,1,1,2,1,2,1,1,2,1,2,1,2,2,2,2,2,2,2,2,2,2,2,2,1,1,0,1,1,1,1,1,0,0,0,0,0,0,0,1,1,1,1,1,1,1,1,2,2,2,3,3,2,2,2,2,2,2,2,3,2,2,2,2,1,1,1,0,1,0,0,0,0,0,0,1,1,1,1,1,2,3,4,3,2,3,3,2,2,3,3,3,2,2,2,3,2,2,1,1,1,0,1,1,2,3,2,2,2,3,2,2,2,1,1,1,0,1,2,2,3,3,3,

A7 ?? 2,2,0,2,0,0,0,2,0,2,2,0,2,2,2,2,0,2,2,2,2,0,2,0,1,1,1,0,2,0,1,1,1,2,1,2,2,2,2,2,3,3,3,1,2,1,3,3,1,1,1,1,1,2,2,2,2,2,2,2,1,1,1,1,2,1,2,2,2,3,2,2,2,2,2,2,2,1,2,2,2,2,2,2,1,1,1,2,3,3,3,3,2,2,1,1,1,1,1,1,0,0,0,0,0,0,0,0,0,1,1,0,0,0,0,0,0,1,1,2,2,1,0,0,0,1,1,1,1,2,1,0,0,1,1,1,1,1,1,1,0,0,1,0,0,1,2,1,0,0,1,1,0,0,1,0,0,0,2,0,1,2,0,0,0,1,1,0,

[themindfactory]

OK set it up for 3 octaves... very interesting after A2,A3,A4 look good.... thats 3, but the first 2 are wacked!

A0 0 80,80,76,74,72,68,68,66,62,60,58,56,52,50,50,48,48,44,42,42,42,38,36,32,32,29,29,25,24,24,21,19,19,19,18,18,18,18,18,20,21,21,22,22,24,23,23,25,25,25,24,24,23,23,23,23,22,22,22,21,21,19,19,19,19,18,18,17,17,16,15,15,

A1 0 180,176,168,164,160,152,146,138,132,126,124,116,112,108,100,94,92,86,80,78,72,68,70,64,60,55,53,52,49,48,46,42,39,38,37,35,33,31,30,28,25,24,22,21,19,18,17,16,16,16,17,17,18,18,17,16,16,16,16,16,17,17,15,14,13,13,12,13,13,13,13,13,

A2 0 210,208,204,204,202,200,194,196,188,186,184,180,174,172,166,160,154,148,142,138,130,124,116,112,107,101,97,93,88,85,83,81,78,76,73,70,68,65,61,57,53,50,48,47,44,43,40,40,39,37,35,33,32,31,31,30,30,28,26,24,23,21,21,21,20,19,18,18,19,19,19,19,

A3 24 184,190,192,196,198,204,206,212,218,222,230,236,244,250,256,262,270,276,284,288,292,300,300,300,303,302,298,294,286,280,270,262,252,243,233,222,211,200,189,179,169,161,153,144,137,133,127,122,113,109,103,97,92,89,85,82,79,76,73,69,65,62,59,56,53,51,49,47,46,45,43,41,

A4 48 62,60,64,64,62,64,64,64,66,64,66,68,70,70,70,72,72,74,76,76,78,78,80,84,102,105,107,111,114,118,122,127,132,136,140,146,153,160,167,178,191,205,219,236,253,267,276,282,285,281,273,262,248,232,213,194,175,158,144,135,127,119,110,101,92,87,82,78,74,68,62,60, 48

A5 72 8,8,8,8,8,8,10,8,10,8,10,8,10,8,10,8,10,8,10,8,10,12,14,12,29,29,28,27,30,30,28,29,29,29,29,29,30,30,31,31,32,33,35,35,37,37,38,40,49,50,52,54,55,57,60,63,66,69,73,76,81,87,94,101,110,121,136,153,174,200,228,255,

[themindfactory]

also just tried to decrease the amplitude of the input (went from 13bit to 8 bit) got basically the same results with just smaller #'s in the array and the LEDS did not light as the #'s being small are not very bright... but the position where the notes appear on the array stayed constant.

[cnlohr]

I'm sorry I don't understand exactly what the arrays are showing under what conditions, do you mind going back and editing the post to include additional information?

[themindfactory]

Sorry, I editted https://github.com/cnlohr/colorchord/issues/82#issuecomment-457888442

[cnlohr]

Hmm... So, first I encourage you to use the embeddedlinux tool and feed the data in procedurally... But I don't agree with your analysis of the data. Something may be wrong with the first one at higher frequencies but you seem to be exceeding the ranges frequency-wise.

First data sets:

Second data sets:

Though I do agree something is broken with the first data set. And am not exactly sure how this could happen, I wonder if the frequency selection for the bins is somehow damaged? Can you tell me the settings of the following preprocessor values? PRECOMPUTE_FREQUENCY_TABLE and USE_32DFT Also, print out the contents of fbins if applicable. I just worry there may be assumptions in UpdateFreqs which should not be in the code. You may be using it outside of its originally intended usage though you should be able to do so there may be currently a bug in that.

Also - for background - lower frequencies generally are rougher, or lumpier because the response time with them is artificially high compared to higher frequencies, so it is difficult to get a really good picture. This is expected behavior and generally does not have an adverse impact on the overall quality of the visualization.

[themindfactory]

Understood about the low frequencies.... USE_32DFT and PRECOMPUTE_FREQUENCY_TABLE are defined

UpdateBinsForProgressiveIntegerSkippyInt() is not used and does not exists, it only uses UpdateBins32();

and fbins is precomputed (constant)

define DREQ 16000

define BASE_FREQ 27.5

define PCOMP(f) (uint16_t)((65536.0)/(DFREQ) (f BASE_FREQ) * 16 + 0.5)

static const uint16_t fbins[FIXBPERO] = { PCOMP(1.000000), PCOMP(1.029302), PCOMP(1.059463), PCOMP(1.090508), PCOMP(1.122462), PCOMP(1.155353), PCOMP(1.189207), PCOMP(1.224054), PCOMP(1.259921), PCOMP(1.296840), PCOMP(1.334840), PCOMP(1.373954), PCOMP(1.414214), PCOMP(1.455653), PCOMP(1.498307), PCOMP(1.542211), PCOMP(1.587401), PCOMP(1.633915), PCOMP(1.681793), PCOMP(1.731073), PCOMP(1.781797), PCOMP(1.834008), PCOMP(1.887749), PCOMP(1.943064) };

values: 1802,1855,1909,1965,2023,2082,2143,2206,2271,2337,2406,2476,2549,2623,2700,2779,2861,2945,3031,3120,3211,3305,3402,3502

[themindfactory]

Just played a 3.3Khz tone G#7 and captured the digital microphone data and ran it online to get a FFT of it to see how it looked, looks pretty clean...

Just trying to rule things out....

[themindfactory]

Hey! So... I rewrote all the DFT functions and went away from the progressive method and went for the brute force, calculate all the bins at once approach.... now it all works 100%, it takes approx. 25% ESP32 CPU to run the effect. I was getting about the same with the non-working implementation also.

My DFT function returns real numbers, I assumed the embeddedbins32[] array with its 2 entries was for real and complex.... so I filled in real and set the complex entry to zero. If this is somehow wrong, it still works :-)

My DFT function looks like, and uses a table to select the frequency coefficient of each note.

float DFT(int16_t *samples, int nSamples, int n) { float coeff = CCcoeffTable[n]; float Q0 = 0, Q1 = 0, Q2 = 0;

while (nSamples--) { Q0 = coeff Q1 - Q2 + ((float)(samples++) / 256.0); Q2 = Q1; Q1 = Q0; }

return Q2 Q2 + Q1 Q1 - coeff Q1 Q2; }

[cnlohr]

Hmm... Would you consider doing a PR for me so this could be an option for others, also I would really appreciate to see the code. I still find this quite perplexing, but, that is kind of ... insane ... that the ESP32 can handle the brute force approach.

[themindfactory]

I will clean up my code and make it available.

I have done even more complex things with the ESP32 and it always shines through, I have a 64x32 RGB panel (HUB75) pumping out 18 bit color at 100Hz while doing 64 channels of DFT for spectrum also at 100Hz and also maintaining about 5 layers of animation that are all updated in realtime. AND the kicker... all with floats! and it also has about 3 digital audio filters running on the 16Khz audio also done with floats, and an AGC :-) all this runs around 35% CPU

[themindfactory]

Well... i found one of my routines was being redundantly called and only needed to be called 10X less... we are now at 4% CPU :-) and that's still using float DFT... I could half this with 32 bit ints... but its not worth it... so it will stay floats!

[cnlohr]

Then I strongly would love it if we could add an ESP32 thing to ColorChord!

[themindfactory]

I will work on the cleanup soon, and document the source code.

The only question I have now is the assumption I made on the embeddedbins32[] being in the format of:

embeddedbins32[0] = DFT0 real embeddedbins32[1] = DFT0 complex embeddedbins32[2] = DFT1 real embeddedbins32[3] = DFT1 complex ...

Is this correct? If not can you describe this data structure and why it has 2 entries per DFT bin?

EDIT: On further checking when I see this type of code.... this looks like the real/complex getting calculated and the result added to embeddedbins[32];

int32_t ipt = &Sdatspace32BOut[0]; int16_t isps = (ipt++)>>16; int16_t ispc = (ipt++)>>16; uint32_t rmux = ( (isps) (isps)) + ((ispc) * (ispc)); embeddedbins32[i] = rmux;

So I think now embeddedbins32[] holds either one octave of 1/2 notes, or 2 octaves of notes....

Let me know.

[cnlohr]

Sdatspace32BOut is the sin/cosine thing. You can see it here: https://github.com/cnlohr/colorchord/blob/master/embeddedcommon/DFT32.c#L212

Sdatspace32BOut should be 2*notes

int32_t Sdatspace32BOut[FIXBINS*2]; //(isses,icses)

Embeddedbins32 is the combined data, the scalar portion. Should be a vector of # of notes.

uint16_t embeddedbins32[FIXBINS];

[PureTek-Innovations]

@themindfactory , it would be really handy if you could share your code. Don't worry if you haven't cleaned it up or added comments. Thank you

[scruffynerf]

Any code to share, @themindfactory ?