ESP32-S2-Kaluga-1 v1.2 + ESP-LyraT-8311a v1.2 - Doesn't work

[gonzoveliki]

1. It can't compile because ESP32-S2-Wrover module which is used on the Kaluga-1 v1. 2 board doesn't have two I2S, and compilation breaks at line 150 in audiokit_board.h
2. I've commented out lines 150-155, now it compiles ok, it talks via i2c on pins 7 and 8 with the codec 8311 successfully, but there is no sound from the generator output example.
3. PA_Enable is on pin10, it is enabled with kit.setSpeakerActive(true);

[pschatzmann]

Since I don't have any of this board, I can't really test. I found that the i2s masterclock was mapped to 0 instead of 35. If my correction is still not working, I suggest that you double check the mapping of all pins...

[gonzoveliki]

Thank you for your quick reply, indeed MCLK must be 35, and now I have sinewave audio output, but it is skipping and sounds distorted... it seems like an I2S fifo's buffer overrun... I'm digging on it now, I will report when I catch where it comes from. Then I will do full tests with microphone input as well. Best Regards

[gonzoveliki]

If I override default config's sample rate with "cfg.sample_rate = AUDIO_HAL_08K_SAMPLES;" just before "kit.begin(cfg);" SineWaveGenerator tests sounds normal. With any other possible sample rate value it does skip and click noisily. ..... I've tested input->output too... it doesn't catch any sound from microphone, but there is something like breathing white noise, goes up and down like an AGC is pumping... I will dump all registers and check thoroughly the configuration of ADC side...

[pschatzmann]

Did you change the log level ? This would explain the reduced sample rate. Not sure which setting is the microphone: did you try both cfg.adc_input = AUDIO_HAL_ADC_INPUT_LINE2 and cfg.adc_input = AUDIO_HAL_ADC_INPUT_LINE1

[gonzoveliki]

Info: bool audiokit::AudioKit::begin(audiokit::AudioKitConfig) Info: Selected board: 4 Info: i2c_sda 8, i2c_scl: 7 Info: SPI: cs: 13 Info: clk: 14, miso: 2, mosi: 15 Info: headphone detection not supported Debug: audio_hal_init Debug: audio_calloc: 1 36 Info: i2c port: 0 Info: i2c sda: 8 Info: i2c scl: 7 Debug: void i2c_bus_create(i2c_port_t, i2c_config_t*) Debug: i2c_bus_write_bytes: addr=48 reglen=1 datalen=1 - reg=1 value=48 Debug: i2c_bus_write_bytes: addr=48 reglen=1 datalen=1 - reg=2 value=0 Debug: i2c_bus_write_bytes: addr=48 reglen=1 datalen=1 - reg=3 value=16 Debug: i2c_bus_write_bytes: addr=48 reglen=1 datalen=1 - reg=22 value=36 Debug: i2c_bus_write_bytes: addr=48 reglen=1 datalen=1 - reg=4 value=16 Debug: i2c_bus_write_bytes: addr=48 reglen=1 datalen=1 - reg=5 value=0 Debug: i2c_bus_write_bytes: addr=48 reglen=1 datalen=1 - reg=11 value=0 Debug: i2c_bus_write_bytes: addr=48 reglen=1 datalen=1 - reg=12 value=0 Debug: i2c_bus_write_bytes: addr=48 reglen=1 datalen=1 - reg=16 value=31 Debug: i2c_bus_write_bytes: addr=48 reglen=1 datalen=1 - reg=17 value=127 Debug: i2c_bus_write_bytes: addr=48 reglen=1 datalen=1 - reg=0 value=128 Debug: i2c_bus_read_bytes: addr=48 reglen=1 datalen=1 - reg=0 Info: ES8311 in Slave mode Debug: i2c_bus_write_bytes: addr=48 reglen=1 datalen=1 - reg=2 value=0 Debug: i2c_bus_read_bytes: addr=48 reglen=1 datalen=1 - reg=5 Debug: i2c_bus_write_bytes: addr=48 reglen=1 datalen=1 - reg=5 value=0 Debug: i2c_bus_read_bytes: addr=48 reglen=1 datalen=1 - reg=3 Debug: i2c_bus_write_bytes: addr=48 reglen=1 datalen=1 - reg=3 value=16 Debug: i2c_bus_read_bytes: addr=48 reglen=1 datalen=1 - reg=4 Debug: i2c_bus_write_bytes: addr=48 reglen=1 datalen=1 - reg=4 value=16 Debug: i2c_bus_read_bytes: addr=48 reglen=1 datalen=1 - reg=7 Debug: i2c_bus_write_bytes: addr=48 reglen=1 datalen=1 - reg=7 value=0 Debug: i2c_bus_read_bytes: addr=48 reglen=1 datalen=1 - reg=8 Debug: i2c_bus_write_bytes: addr=48 reglen=1 datalen=1 - reg=8 value=255 Debug: i2c_bus_read_bytes: addr=48 reglen=1 datalen=1 - reg=6 Debug: i2c_bus_write_bytes: addr=48 reglen=1 datalen=1 - reg=6 value=3 Debug: i2c_bus_read_bytes: addr=48 reglen=1 datalen=1 - reg=1 Debug: i2c_bus_write_bytes: addr=48 reglen=1 datalen=1 - reg=1 value=63 Debug: i2c_bus_read_bytes: addr=48 reglen=1 datalen=1 - reg=6 Debug: i2c_bus_write_bytes: addr=48 reglen=1 datalen=1 - reg=6 value=3 Debug: i2c_bus_write_bytes: addr=48 reglen=1 datalen=1 - reg=19 value=16 Debug: i2c_bus_write_bytes: addr=48 reglen=1 datalen=1 - reg=27 value=10 Debug: i2c_bus_write_bytes: addr=48 reglen=1 datalen=1 - reg=28 value=106 Debug: audio_codec_initialize -> 0 Info: audio_codec_initialize-END-OK Debug: i2c_bus_read_bytes: addr=48 reglen=1 datalen=1 - reg=9 Debug: i2c_bus_read_bytes: addr=48 reglen=1 datalen=1 - reg=10 Debug: i2c_bus_write_bytes: addr=48 reglen=1 datalen=1 - reg=9 value=12 Debug: i2c_bus_write_bytes: addr=48 reglen=1 datalen=1 - reg=10 value=12 Debug: i2c_bus_read_bytes: addr=48 reglen=1 datalen=1 - reg=9 Debug: i2c_bus_read_bytes: addr=48 reglen=1 datalen=1 - reg=10 Debug: ES8311 in I2S Format Debug: i2c_bus_write_bytes: addr=48 reglen=1 datalen=1 - reg=9 value=12 Debug: i2c_bus_write_bytes: addr=48 reglen=1 datalen=1 - reg=10 value=12 Debug: SET: volume:51 Debug: i2c_bus_write_bytes: addr=48 reglen=1 datalen=1 - reg=50 value=51 Debug: setupI2S Info: i2s bck_io_num: 18 Info: i2s ws_io_num: 17 Info: i2s data_out_num: 12 Info: i2s data_in_num: 46 Info: i2s_set_pin Info: - bck_io_num: 18 Info: - ws_io_num: 17 Info: - data_out_num: 12 Info: - data_in_num: 46 Info: - mck_io_num: 35 Debug: audio_hal_ctrl_codec Info: Codec mode is 3, Ctrl:1 Debug: i2c_bus_read_bytes: addr=48 reglen=1 datalen=1 - reg=9 Debug: i2c_bus_read_bytes: addr=48 reglen=1 datalen=1 - reg=10 Debug: i2c_bus_write_bytes: addr=48 reglen=1 datalen=1 - reg=9 value=12 Debug: i2c_bus_write_bytes: addr=48 reglen=1 datalen=1 - reg=10 value=12 Debug: i2c_bus_write_bytes: addr=48 reglen=1 datalen=1 - reg=23 value=191 Debug: i2c_bus_write_bytes: addr=48 reglen=1 datalen=1 - reg=14 value=2 Debug: i2c_bus_write_bytes: addr=48 reglen=1 datalen=1 - reg=18 value=0 Debug: i2c_bus_write_bytes: addr=48 reglen=1 datalen=1 - reg=20 value=26 Debug: i2c_bus_read_bytes: addr=48 reglen=1 datalen=1 - reg=20 Debug: i2c_bus_write_bytes: addr=48 reglen=1 datalen=1 - reg=20 value=26 Debug: i2c_bus_write_bytes: addr=48 reglen=1 datalen=1 - reg=13 value=1 Debug: i2c_bus_write_bytes: addr=48 reglen=1 datalen=1 - reg=21 value=64 Debug: i2c_bus_write_bytes: addr=48 reglen=1 datalen=1 - reg=55 value=72 Debug: i2c_bus_write_bytes: addr=48 reglen=1 datalen=1 - reg=69 value=0 ..... Debug: read: 1024 Debug: write: 1024 Debug: read: 1024 Debug: write: 1024 .....

[gonzoveliki]

Yes, I did..same results, ADC reads samples full of harsh white noise, pumping up and down... Today I will dig deeper with an oscilloscope and Logic Analyzer to check I2S integrity. As I was able to trace the code flow of initialization all I2C registers were loaded properly as intended. I will make the same log for a LyraT v4.3 board which is working normally, to see are there some skipped sections of the init sequence.

[pschatzmann]

I think the mic is connected as diff input: did you try cfg.adc_input =AUDIO_HAL_ADC_INPUT_DIFFERENCE

[gonzoveliki]

I did try with LINE1 and LINE2, now with DIFFERENCE... it's the same pumping noise up and down, but now i think there is a "difference" - the microphone is picking at least something, because when the noise is going down, I can hear something - if knock on the board or whistling straight into the mic. It looks that it comes from some kind of Auto Leveling Gain control which is out of lock.

[gonzoveliki]

[gonzoveliki]

This codec supports digital mics too (Digital Data goes like single ended mic into the chip, while mic's clock is sourced from I2S) Both for analog and digital mics it offers some basic DSP's gain, filtering and equalizer features on-board. I will fiddle with corresponding registers so hear is there any change.

[pschatzmann]

Not sure if this helps, but I wondered about the following line:

define ES8311_MCLK_SOURCE 1 / 0 From MCLK, 1 From BCLK /

I would have expected this to be 0 ...

[gonzoveliki]

I've tested that already and it didn't help, but I'm about to nail it. There is ALC control which needs to be constrained in gain, and there is muting threshold levels to suppress background noise in idle.. These registers don't have proper factory defaults, so I fiddled randomly with values, and now it is recoding clean an undistorted audio. I will test thoroughly to see which are vital for proper initialization for recording, so you can add some extra functionality to HAL and the codec chip driver. I think some conditional #ifdefs must be used as well, so the problem with missing I2S_NUM1 on S2-Wrover modules shall be sorted. I will make a full report of other tests and will give you necessary stuff you wish to change the code base. Best regards

[gonzoveliki]

I'm still chasing the sampleRate issue, it is working good only with 8k. On all other possible rates it does skipping and clicking but sound pitch is unchanged - so it's not like sampling frequency mismatch, but more like I2S buffer overrun. Now I will connect the codec board LyraT to another MCU which is not S2, but WROVER-E, to check the possibility that bug comes from ESP core libs.

[gonzoveliki]

Higher samplerates than 8k, consumes buffer qiucker, so if i increase buffer to 16000, and use 16k as sample rate, it sounds 1/2 of the time, if i make buffer 32k and sample rate also 32k, it sounds 1/4 of the second... I thought it is related with that this codec is mono, and SineGenerator feeds two 16bit integers for a sample into the buffer... but it is not that... I will dig more, so far I achieved good recording/playback audio @8k samplerate, after settled mic gain and ALC min/max levels...

[pschatzmann]

Maybe it helps to compare the actual 8311.c from ADF with my (old) copy. I noticed that the coeff_div has changed.

[gonzoveliki]

I have copied "coeff" table and most of the initialization code from the latest ADF, but results were still the same. Now I'm almost 100% positive that problems are I2S related and not with the codec itself. It is improving if I manually compose the "i2s_config_t i2s_config" forcing to mono format and different buffer_count and size scheme... somehow S2 chip clocks out unstable master and bit clocks, I can hear its jitter and frequency instability... it could be clock invesion or edge slopes too smooth, there are some capacitors on I2S - i suspect these, because it works good with 8k, but it worses with rising the samplerate

and it is improving when I make stream mono, which is also reducing i2s transaction bandwidth

first schematic is LyraT mini, second is LyraT-8311A v1.2 from Kaluga-1, both showing the same issues

[pschatzmann]

Hmm, I never tried - but maybe it helps to have the ESP32 in slave mode and use the 8311 as master ? does it make a difference with i2s pll active or inactive ?

[gonzoveliki]

That's good idea I will try tonight, meanwhile I have de-soldered 4 x 22p caps C101,C32,C33,C34... now measuring with scope shows the edges became a little bit more steep, and it started to work with 11K sample rate, however a strange click is heard from time to time on random basis... could be errors from shifted clock or a jitter accumulation... it's still working perfect with 8k sr. BTW: on other boards with ESP32 coupled with similar codecs I saw resistors in series - like 22R or 33R on all I2S lines, but with Kaluga and LyraT (mini) there aren't any, but those four 22p caps which are probably intended to filter noise and smooth edges preventing overshoot/ringing...

[gonzoveliki]

AudioKitHAL.h @line 640 ... bool setupI2S(AudioKitConfig cnfg)... but later cnfg is nowhere used - instead cfg which a local protected member in the class... was it intended like that? probably setupI2S must not be called with external parameters...

[pschatzmann]

Actually I am not using this I2S output any more. In the AudioTools I rely on the I2S output functionality of the AudioTools and the AudioKit is just used to setup the codec

[gonzoveliki]

Then I have to check how it is performing with AudioTools, so far with AudioKit it is working at 8k, and after I de-soldered few caps started to work @11k too. But it is not clean and sharp like 8388, sounds like de-tuned and with random clicks of an old vinyl record :)

[pschatzmann]

I will remove the parameter for the setupI2S() method since this is not used and is confusing...

[gonzoveliki]

okay... which example of AudioTools should I use for testing output and changing sample rate ?

[pschatzmann]

I usually test the output with a sine generator: https://github.com/pschatzmann/arduino-audio-tools/tree/main/examples/examples-audiokit/streams-generator-audiokit

Or for intput and output: https://github.com/pschatzmann/arduino-audio-tools/tree/main/examples/examples-audiokit/streams-audiokit-audiokit

[gonzoveliki]

the Generator is sounding good... so it must be I2S for the audioKit write/read malfunction... but it is working with 8388 codec reasonably...

[gonzoveliki]

In fact kaluga's I2S_NUM is 0, because on ESP-LyraT-8311A v1.2 shield board there are routed two I2S pathways (for Codec-ES8311 and ADC1-ES7243), but the S2 chip which is on the base board, have only one I2S controller built-in, so it's not possible to use 8311 and the ADC1 simultaneously, and I2S_NUM_0 must be used as port, but with different pin mapping targeting ES8311 or ES7243,,, conditional #if defined(CONFIG_IDF_TARGET_ESP32S2) must be used in audiokit_board.h - function get_i2s_pins... to avoid compilation errors with S2 chips... while

I think it's a good idea to split the board profile for ESP-S2-Kaluga-1 into two board setups, one for Codec, and other for ADC both configured at same I2S channel, but with different pin mapping and driver assignment

[gonzoveliki]

Confirmed, stream-input->output forwarding with AudioTools do work on 8311, but it still does need patching mic_gain and ALC max/min registers. Still digging on I2S issues with AudioKit, I'm looking now how it is done in AudioTools, and what could be the problem...

[pschatzmann]

I found this: https://github.com/pschatzmann/arduino-audiokit/commit/d9cfb282169a06fa36e40bd803587be1471324d2

[gonzoveliki]

I have already set interrupt flags prio to 0, also added these lines at the bottom of the section.

if ESP_IDF_VERSION_MAJOR >= 4

        .fixed_mclk = 0, //(int) (cfg.fixed_mclk>0 ? cfg.fixed_mclk : 0 ),
        .mclk_multiple = I2S_MCLK_MULTIPLE_DEFAULT,
        .bits_per_chan = I2S_BITS_PER_CHAN_DEFAULT,

endif

Same results, all registers of codec are set properly, but with AudioTools via I2SESP.h it sounds good at any SR, while with build in read/write in AudioKit it does work only at 8k, and with some minor clicks and sparks at 11k... It's strange because it did improve a bit, when I have de-soldered edge smoothing caps from the I2S lines... but with AudioTools it outputs normal even caps are present.

[gonzoveliki]

It seems like buffering issue, overrun at fifo buffers, or DMA... interrupt nesting... I will not stop until it is sorted, it's good to have the more simple AudioKit usable on all platforms, with supported boards and codecs... Please consider splitting board profile for Kaluga + ESP-LyraT-8311A v1.2 in two setups, one for ES8311 and another for ES7243... so different pin mapping can be switched easy with the board selection choice... BTW: I saw there are I2SESP.h and I2SESP-New.h, switched via #define USE... flag at compile time... What is the idea ? Are you going to use the new version at future as default, or it will remain like that for backward compatibility?!?

[pschatzmann]

I would like to have this resolved as well, but currently I don't see any difference.

The idea is to use the I2SESP-New.h as soon Arduino supports the new API...

[gonzoveliki]

This afternoon I will try to isolate what is causing these strange effects, for now its 100% that it has nothing to do with codec initialization, however some corrections would be necessary to setup the proper INPUT_LINE and mic gain + alc. I will try to copy and use simplified version of the code from I2SESP, just one by one, hoping that at some moment problem will be resolved.. Its quite challenging, I love those strange situations, than both hardware and software has to fiddled to reach the root cause of the issue.

[gonzoveliki]

You won't believe it, but the cause of noises clicks and sounding de-tuned in pitch, was caused by the SineWaveGenerator.h included in the output.ino test... it is from the "float", I've changed angle and deltaTime to double, and noises and clicks did stop immediately, however there is de-tune effect, but when i changed all floats to double, it sounds clean and nice at all sampleRates... because deltaTime is precomputed at setSampleRate... it lacks of resolution, and later when angle is accumulated, because it's not wrapped around at a full cycle, floats do cause these distortion, and clicks at higher sampleRates... Unbelievable... With Wrover chips probably compiler is set to use an extended resolution for floats, or does some compile time patching for calculations - this example is working fine on other platforms, but S2 :) ... also Arduino core can pull different sin(x) implementation depending on the processor, so that for S2 can be vulnerable about angular roll-over... I saw you are using sinf(x) in AudioTools... if (m_cycles > 1.0) m_cycles -= 1.0; that's is the missing crucial part in the SineWaveGenerator.h if working with floats... they tend to loose resolution too fast as the value grows... and you were adding minor "pre-calculated" fraction of delta_time...

[gonzoveliki]

i propose that as a replacement for readSample() in the SineWaveGenerator class which is bundled with output.ino example

int16_t readSample() { if ((m_cyc += m_frequency / sample_rate) > 1.0) m_cyc -= 1.0; return m_amplitude sinf(double_Pi m_cyc); }

[pschatzmann]

Strange: why would this not be noticeable on the regular ESP32. Can you submit a pull request ?

[gonzoveliki]

Probably it is because of a different compiler options, and/or versions/implementations of the pulled-in libraries into the Arduino core. It could be from configs for specific CPU/Boards or can be caused by a any pragma statement somewhere in all included source code for a given core variant. I could hunt it down, but it's better first to make code in the example - to be as portable as possible. Okay I will submit a pull request a bit later when I fix the input recording to work out of the box as well.

[gonzoveliki]

output.zip I'm sending a modified version of the AudioKit/examples/output for preview. It comes with an improved SineWaveGenerator class, prone to sin(x) with float issues on ESP32-S2, and also is bundled with a serial Control module. You can easily change on-the-play - SampleRate, Volume, Frequency, Amplitude, Muting and Speaker activation, Buffer reducing...etc. It is not final, I have some more things to rectify, if you like it I will create a pull request

[pschatzmann]

Hmm, if you want to submit this, I suggest that you create a new directory in the examples instead of changing the existing one. For the SineWaveGenerator I prefer to use the logic of my current implementation from the AudioTools, so I will take care of this. I suggest that you submit this with a pull request along with your other corrections to the driver

[gonzoveliki]

okay. i will put this example in a separate folder, you can just fix generator of the existing one. now I'm working on input and microphone line and gain setups for ESP-LyraT-8311A board, and to enable ES7243 as well in separate board profile

[pschatzmann]

If your ESP32-S2-Kaluga-1 v1.2 changes are working, I propose to submit it. It is much easier to review some individual changes than many different things