search

marcel-licence / ML_SynthTools

ML_SynthTools
GNU General Public License v3.0
147 stars 24 forks source link

ESP32-AudioKit ES8388 - working organ; not working ePiano or FM synth #51

Closed jordanlewis closed 1 week ago

jordanlewis commented 1 month ago

Hi, I've purchased an ESP32-Audiokit (v2.2, "a417" on the board) and trying to work through your great example projects. Thanks for providing this software!

The Organ example works really well, after setting:

#define BOARD_ESP32_AUDIO_KIT_ES8388

and, in the es8388 config file:

#define ES8388_CFG_I2C  2
#define ES8388_CFG_I2S  6

I'm using the "ESP32 Dev Module" board provided by the esp32 2.0.2 library, on Arduino 1.8.13.

However, both the FM synth and ePiano examples produce sound that's extremely distorted - there's some static, and additionally the notes are kind of detuned - a fifth played via midi sounds more like a third through the synths.

By the way, I've tried with and without PSRAM, and with CPU frequencies 160/240MHz, with no discernable difference in the output.

What might be going wrong? What are some differences in the organ and FM/epiano examples that could explain the troubled output? Are there other configuration settings I should be looking at editing? I'm not quite sure where to look to start isolating the problem.

Thank you!

marcel-licence commented 1 month ago

Hi,

I can confirm that PSRAM may slow down the performance causing these kind of issues. Is there any other modification? I could imagine that there are some buffer underruns. Usually, it should work on the ESP32 Audio Kit.

Do you have a short audio of the observed problem?

Best regards,

Marcel

jordanlewis commented 1 month ago

Hi Marcel,

Here is a video of the fm synth example: https://photos.app.goo.gl/GQjyPSYdunyiaLWQ9

There aren't any other modifications. I did turn on the startup tone to get a sound, that's the background noise you hear.

Here is the boot log:

ets Jul 29 2019 12:21:46

rst:0x1 (POWERON_RESET),boot:0x1f (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:1
load:0x3fff0030,len:1324
ho 0 tail 12 room 4
load:0x40078000,len:13508
load:0x40080400,len:3604
entry 0x400805f0
[   858][D][esp32-hal-psram.c:84] psramInit(): PSRAM enabled
Heap summary for capabilities 0x00000004:
  At 0x3ffb8000 len 6688 free 3932 allocated 556 min_free 3932
    largest_free_block 3892 alloc_blocks 3 free_blocks 1 total_blocks 4
  At 0x3ffb0000 len 25480 free 15064 allocated 8228 min_free 15064
    largest_free_block 14836 alloc_blocks 2 free_blocks 1 total_blocks 3
  At 0x3ffae6e0 len 6192 free 3980 allocated 36 min_free 3980
    largest_free_block 3956 alloc_blocks 1 free_blocks 1 total_blocks 2
  At 0x3f800000 len 4194303 free 4192139 allocated 0 min_free 4192139
    largest_free_block 4128756 alloc_blocks 0 free_blocks 1 total_blocks 1
  At 0x3ffb6388 len 7288 free 1716 allocated 2976 min_free 104
    largest_free_block 1172 alloc_blocks 36 free_blocks 2 total_blocks 38
  At 0x3ffb9a20 len 16648 free 5512 allocated 8672 min_free 456
    largest_free_block 4852 alloc_blocks 25 free_blocks 3 total_blocks 28
  At 0x3ffc97a0 len 92256 free 87584 allocated 2472 min_free 87060
    largest_free_block 86004 alloc_blocks 3 free_blocks 1 total_blocks 4
  At 0x3ffe0440 len 15072 free 12908 allocated 0 min_free 12908
    largest_free_block 12788 alloc_blocks 0 free_blocks 1 total_blocks 1
  At 0x3ffe4350 len 113840 free 111676 allocated 0 min_free 111676
    largest_free_block 110580 alloc_blocks 0 
Loading data
Connect to ES8388 codec...
  SDA: 33
   SCL: 32
  freq: 400000
0x00: 0x05
0x01: 0x40
AdcCh1!
MixChAMPL!
ES8388 setup finished!
Reg 0x00 = 0x05
Reg 0x01 = 0x40
Reg 0x02 = 0x00
Reg 0x03 = 0x00
Reg 0x04 = 0x3c
Reg 0x05 = 0x00
Reg 0x06 = 0x00
Reg 0x07 = 0x7c
Reg 0x08 = 0x00
Reg 0x09 = 0x88
Reg 0x0a = 0x50
Reg 0x0b = 0x02
Reg 0x0c = 0x0c
Reg 0x0d = 0x02
Reg 0x0e = 0x30
Reg 0x0f = 0x20
Reg 0x10 = 0x00
Reg 0x11 = 0x00
Reg 0x12 = 0x16
Reg 0x13 = 0xb0
Reg 0x14 = 0x32
Reg 0x15 = 0x06
Reg 0x16 = 0x00
Reg 0x17 = 0x18
Reg 0x18 = 0x02
Reg 0x19 = 0x22
Reg 0x1a = 0x00
Reg 0x1b = 0x02
Reg 0x1c = 0x08
Reg 0x1d = 0x00
Reg 0x1e = 0x1f
Reg 0x1f = 0xf7
Reg 0x20 = 0xfd
Reg 0x21 = 0xff
Reg 0x22 = 0x1f
Reg 0x23 = 0xf7
Reg 0x24 = 0xfd
Reg 0x25 = 0xff
Reg 0x26 = 0x1b
Reg 0x27 = 0xb8
Reg 0x28 = 0x28
Reg 0x29 = 0x28
Reg 0x2a = 0xb8
Reg 0x2b = 0x80
Reg 0x2c = 0x00
Reg 0x2d = 0x00
Reg 0x2e = 0x1e
Reg 0x2f = 0x1e
Reg 0x30 = 0x1e
Reg 0x31 = 0x1e
Reg 0x32 = 0x00
Reg 0x33 = 0xaa
Reg 0x34 = 0xaa
I2S configured using following pins:
    BCLK,BCK: 27
    WCLK,LCK: 25
    DOUT: 26
    DIN: 35
rev: 15329, 15329
Reverb is ready!
MIDI listen on Serial with 115200 baud
Setup Serial2 with 31250 baud with rx: IO21 only
Setup MidiPort2 using Serial2
ESP.getFreeHeap() 284460
ESP.getMinFreeHeap() 277252
ESP.getHeapSize() 312632
ESP.getMaxAllocHeap() 110580
Total heap: 312632
Free heap: 284460
Total PSRAM: 4192103
Free PSRAM: 4093195
Firmware started successfully
jordanlewis commented 1 month ago

Here is a video of the ePiano example, compiled without PSRAM: https://photos.app.goo.gl/cNu4yij3VPaYESNw6

Startup log:

mode:DIO, clock div:1
load:0x3fff0030,len:1324
ho 0 tail 12 room 4
load:0x40078000,len:13508
load:0x40080400,len:3604
entry 0x400805f0

Loading data
Firmware started successfully
Initialize Audio Interface
Connect to ES8388 codec...
  SDA: 33
   SCL: 32
  freq: 400000
0x00: 0x05
0x01: 0x40
AdcCh1!
MixChAMPL!
ES8388 setup finished!
Reg 0x00 = 0x05
Reg 0x01 = 0x40
Reg 0x02 = 0x00
Reg 0x03 = 0x00
Reg 0x04 = 0x3c
Reg 0x05 = 0x00
Reg 0x06 = 0x00
Reg 0x07 = 0x7c
Reg 0x08 = 0x00
Reg 0x09 = 0x88
Reg 0x0a = 0x50
Reg 0x0b = 0x02
Reg 0x0c = 0x0c
Reg 0x0d = 0x02
Reg 0x0e = 0x30
Reg 0x0f = 0x20
Reg 0x10 = 0x00
Reg 0x11 = 0x00
Reg 0x12 = 0x16
Reg 0x13 = 0xb0
Reg 0x14 = 0x32
Reg 0x15 = 0x06
Reg 0x16 = 0x00
Reg 0x17 = 0x18
Reg 0x18 = 0x02
Reg 0x19 = 0x22
Reg 0x1a = 0x00
Reg 0x1b = 0x02
Reg 0x1c = 0x08
Reg 0x1d = 0x00
Reg 0x1e = 0x1f
Reg 0x1f = 0xf7
Reg 0x20 = 0xfd
Reg 0x21 = 0xff
Reg 0x22 = 0x1f
Reg 0x23 = 0xf7
Reg 0x24 = 0xfd
Reg 0x25 = 0xff
Reg 0x26 = 0x1b
Reg 0x27 = 0xb8
Reg 0x28 = 0x28
Reg 0x29 = 0x28
Reg 0x2a = 0xb8
Reg 0x2b = 0x80
Reg 0x2c = 0x00
Reg 0x2d = 0x00
Reg 0x2e = 0x1e
Reg 0x2f = 0x1e
Reg 0x30 = 0x1e
Reg 0x31 = 0x1e
Reg 0x32 = 0x00
Reg 0x33 = 0xaa
Reg 0x34 = 0xaa
I2S configured using following pins:
    BCLK,BCK: 27
    WCLK,LCK: 25
    DOUT: 26
    DIN: 35
MIDI listen on Serial with 115200 baud
Setup Serial2 with 31250 baud with rx: IO21 only
Setup MidiPort2 using Serial2
rev: 15329, 15329
Reverb is ready!
ESP.getFreeHeap() 208112
ESP.getMinFreeHeap() 201816
ESP.getHeapSize() 306144
ESP.getMaxAllocHeap() 110580
Total PSRAM: 0
Free PSRAM: 0
Firmware started successfully
[?25lChorus_SetPhaseShift: 0.500
jordanlewis commented 1 month ago

For contrast, here is a video of the organ example playing some sequenced chords: https://photos.app.goo.gl/KocrteWScf5MTCZy6

marcel-licence commented 1 month ago

Thank you very much for the videos. That helps a lot. It definitely sounds like a buffer underrun. It usually happens when the code execution is too slow. I wonder how that happened. I will take a look later and bring the project onto an audio kit.

marcel-licence commented 1 month ago

I tried different versions but no issues with the project (esp32_fm_synth). I tried v1.0.6, v2.0.2 and v2.0.13 I changed only one line to select the ESP32 board.

grafik

jordanlewis commented 1 month ago

Interesting. Could you share which AudioKit revision you are using? The one I have says v2.2 and a417.

jordanlewis commented 1 month ago

Also, is there something I could do to help diagnose which part of the code is running too slowly? For example, I could disable different sub-components to try to bisect where things are going wrong. However, I don't know what hooks there are available for me to try something like that.

marcel-licence commented 1 month ago

Did you try the projects from the example directory or did you try the origin / separated projects? I had issues with updating the submodules. My version is v2.2 2957. Yours is using the ESP32 A1S as-well?

jordanlewis commented 1 month ago

Yes, ESP32 A1s. Here is a picture. I wonder if there is some difference with 2957...

image

jordanlewis commented 1 month ago

I tried the origin / separated projects, I just cloned them individually.

jordanlewis commented 1 month ago

Hi - I solved the problem! Somewhere along the way I had enabled debug core mode in the Arduino UI... without that, there are no stuttering issues. PEBKAC. Thanks again! I hope to post a project video for you when I get a bit further.

jordanlewis commented 1 month ago

Hi @marcel-licence, as an FYI one interesting thing that I ran into is that after pulling the very latest commit ffcdb95bf21790250193fcd8c315bce9ee24f954, the original configuration that I wrote about:

#define ES8388_CFG_I2C  2
#define ES8388_CFG_I2S  6

now produces a build that produces audio but gibberish text.

Reverting to the default I2S config of 4 produces non-gibberish text but no audio.

Using the I2S config 5 produces both text and audio.

marcel-licence commented 1 month ago

Hi @jordanlewis I am very happy to hear that you solved the problem. Good to know that it might be possible to slow everything down with some debug settings.

Between ES8388_CFG_I2S 5 and 6 the MCLK pin is set to 0 or 1. I think 1 is not supported on the ESP32 (only supported on ESP32S3).

Best regards, Marcel

marcel-licence commented 2 weeks ago

@jordanlewis is this issues solved?

jordanlewis commented 1 week ago

Yes thank you!